diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 168408d87..e0c09905d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -12,6 +12,7 @@ on: permissions: contents: write + packages: write concurrency: group: release-${{ github.ref }} diff --git a/.github/workflows/windows-arm64.yml b/.github/workflows/windows-arm64.yml index 76d146e48..3d5312b14 100644 --- a/.github/workflows/windows-arm64.yml +++ b/.github/workflows/windows-arm64.yml @@ -19,8 +19,8 @@ permissions: jobs: windows-arm64: - # Immutable supporting workflow revision until the new install contract lands upstream. - uses: storytold/craft-common-actions/.github/workflows/windows-arm64.yml@8bb4e5ab95443171104ad88113fe1ef9b0e5ce3f + # Intentionally tracked at @main (not SHA-pinned) so org-wide CI improvements, security fixes, and build optimizations propagate without PR churn across all craftapp repos. + uses: storytold/craft-common-actions/.github/workflows/windows-arm64.yml@main with: app-name: gridcraft app-dir: GridCraft diff --git a/AGENTS.md b/AGENTS.md index 289cb12d0..e4fcf5570 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,8 +2,6 @@ GridCraft is a clean-room, open-source, Rust-native spreadsheet targeting Microsoft Excel parity — and superiority (speed, openness, agent control). It runs natively on macOS, Windows, Linux and BSD, and on the web via WASM. Siblings with the same conventions: `../designcraft` (InDesign-class, the closest reference), `../photocraft` (Photoshop), `../vectorcraft` (Illustrator), `../pdfcraft` (Acrobat), `../filmcraft` (Premiere), `../lightcraft` (Lightroom), `../effectcraft` (After Effects). -Standards and learnings shared across the crafting apps live in `../../craftrules` (checked out next to the craft apps, or `storytold/craftrules`). Read its `AGENTS.md` at the start of a session, follow its standards, and contribute reusable learnings back there. Never code: repos don't share code. - ## Start every session here 1. Read `plan/STATUS.md` (current milestone, next task), then the task in `plan/execution-plan.md` and the relevant `plan/architecture.md` section. Behaviour reference: `plan/excel/*.md` (`observed-ui.md` holds measured observations of the running app). 2. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's. @@ -20,10 +18,10 @@ Standards and learnings shared across the crafting apps live in `../../craftrule - The ArtCraft brand files in `docs/brand/` are ArtCraft trademarks under `docs/brand/LICENSE-brand.txt`, the one exception. ## Never crash -People trust GridCraft with their numbers; a crash loses their work. **This outranks feature work.** Standard: [`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md). +People trust GridCraft with their numbers; a crash loses their work. **This outranks feature work.** - **No panics in non-test code:** no `unwrap()`, `expect()`, `panic!`, `unreachable!`, `todo!`, `unimplemented!`; no `unsafe` (`unsafe_code = "forbid"`). - **Errors are `Result`** through the crate's error type and `?`. -- **Input-derived numbers are hostile** (files, formulas, commands, MCP/control params): `get()` instead of indexing, checked/saturating arithmetic, cap input-sized allocations (huge ranges, `REPT`, `SEQUENCE`, whole-column references). +- **Input-derived numbers are hostile** (files, formulas, commands, MCP/control params): validate indexes and ranges before access, use checked access when validity is not established, use checked/saturating arithmetic, and cap input-sized allocations (huge ranges, `REPT`, `SEQUENCE`, whole-column references). - **Bound recursion** (formula nesting, dependency chains, names referring to names). - **Last-resort guard:** `Session::execute` catches escaped panics and restores the workbook. It's a safety net, not a licence. - **Prove it:** every crash fix lands with a regression test (`tests::every_command_survives_empty_params` fuzzes every command; `crates/engine/tests/it/command_sequence_fuzz.rs` fuzzes random command sequences against undo/redo, XLSX round-trip and full-recalc invariants; soak it with `GRIDCRAFT_FUZZ_SEEDS`/`GRIDCRAFT_FUZZ_STEPS`, replay with `GRIDCRAFT_FUZZ_SEED`). @@ -31,8 +29,8 @@ People trust GridCraft with their numbers; a crash loses their work. **This outr ## Non-negotiables - **Clean-room.** Microsoft Excel is installed on the dev machine and may be *observed* black-box: run it, use its UI with synthetic workbooks, take screenshots (by window id) stored only under `plan/excel/screenshots/`. Never read, disassemble or copy anything inside the Office bundle, never commit files produced by Excel, never copy GPL/AGPL/LGPL code (LibreOffice, Gnumeric…). Function behaviour comes from public documentation and observation. -- **Fonts live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts)**, never in this repo. It is an optional build input (`CRAFT_FONTS_DIR`); without it the app uses system fonts and egui's defaults. Standard: craftrules `standards/fonts.md`. -- **Test corpora** (real Excel-authored files) live in a separate corpus repo, fetched pinned and sha256-verified (craftrules `standards/test-corpora.md`). Never commit large binary fixtures; build test workbooks in code. +- **Fonts live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts)**, never in this repo. It is an optional build input (`CRAFT_FONTS_DIR`); without it the app uses system fonts and egui's defaults. +- **Test corpora** (real Excel-authored files) live in a separate corpus repo, fetched pinned and sha256-verified. Never commit large binary fixtures; build test workbooks in code. - **Everything is a command.** User-visible behaviour = a command in `crates/engine/src/cmd/*` (id, label, ribbon path, shortcut, params doc, `enabled`, `run`) + tests. Ids follow Excel's ribbon (`home.bold`, `insert.chart`, `data.sortAscending`) plus primitives (`cell.set`, `selection.set`, `range.setValues`). UI-only commands live in `crates/ui-egui`. The control channel, CLI and MCP reach all of them. - **Programmatic calls never open dialogs**: a command with empty params runs with defaults or returns an error; only menu-style invocation opens a dialog (`UiRequest::Dialog`). - **Layering** is enforced by `cargo xtask layers`: `core`/`numfmt` (L0) → `formula`/`functions` (L1) → `model`/`calc`/`xlsx` (L2) → `chart` (L3) → `engine` (L5) → `ui-egui`/`mcp` (L6) → `apps/*` (L7). Nothing below L6 depends on egui/eframe/winit/rfd. **The UI crate is swappable.** @@ -51,7 +49,7 @@ People trust GridCraft with their numbers; a crash loses their work. **This outr - Parallel agents: separate `CARGO_TARGET_DIR` per agent; edit only the crates you own; delete your target dir when done. ## Roadmap -`ROADMAP.md` (committed) is the one-page summary: stage, headline numbers, dimensions, languages, progress log. It follows craftrules `standards/progress-docs.md` and points at the detail in `docs/`: `target-app-parity.md` (authoritative assessment), `gaps.md` (ranked work list: pick from it), `roadmap.md` (current focus, beta gates, milestones), `architecture.md`, and the area checklists `function-parity.md`, `chart-parity.md`, `file-format-parity.md`, `ui-parity.md`, `hardware-parity.md`, `localization-parity.md`. Update the affected docs (and their timestamp lines and revision history) whenever work lands. `cargo xtask parity` regenerates `docs/parity-checklist.md` from `crates/engine/src/catalog.rs`. +`ROADMAP.md` (committed) is the one-page summary: stage, headline numbers, dimensions, languages, progress log. It points at the detail in `docs/`: `target-app-parity.md` (authoritative assessment), `gaps.md` (ranked work list: pick from it), `roadmap.md` (current focus, beta gates, milestones), `architecture.md`, and the area checklists `function-parity.md`, `chart-parity.md`, `file-format-parity.md`, `ui-parity.md`, `hardware-parity.md`, `localization-parity.md`. Update the affected docs (and their timestamp lines and revision history) whenever work lands. `cargo xtask parity` regenerates `docs/parity-checklist.md` from `crates/engine/src/catalog.rs`. ## Contributor credits (About window) @@ -62,11 +60,13 @@ People trust GridCraft with their numbers; a crash loses their work. **This outr username has a `[people.]` entry in `contributors/people.toml`. If not, ask them once whether they want to be credited by more than their username: a real name, a display name, and/or their public GitHub profile name (`sync_github_name = true`). If yes, add **only their own** entry - (copy the template at the top of the file, or run - `python3 ../../craftrules/scripts/contributors.py --add-me . --real-name "…" --sync-github-name`) + (copy the template at the top of the file) and include it in their PR, committed as them. If no, change nothing: they are credited as `@username` anyway. - Never add, edit, guess or copy anyone else's entry or name (not from git config, commit authors or GitHub profiles). Never hand-edit `contributors.json`. -- Maintainers refresh the stats with `python3 ../../craftrules/scripts/contributors.py .` (it also - re-verifies who wrote each `people.toml` entry; `--check` only verifies). + +## CI workflows +- Workflows in `.github/workflows/` call the shared workflows in [`storytold/craft-common-actions`](https://github.com/storytold/craft-common-actions) at `@main`, with their inputs and secrets. No other jobs: if no shared workflow does what's needed, it doesn't go here. +- Don't add or change CI in a feature or fix PR. If the existing workflows can't build or test a change, link an existing craft-common-actions issue or PR, or open a new issue, or a PR only for a small fix that is safe for every app and does not increase CI time. +- Some workflow files here still run their own jobs. Don't add to them. Moving a job into craft-common-actions is its own task, never part of another PR, and the job stays here until a shared workflow runs it. diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index a1e4d8fbc..2a4b5c96e 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -12,15 +12,15 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra | `epaint_default_fonts::HACK_REGULAR` (dependency font input) | Source Foundry Authors (Hack; DejaVu and Bitstream Vera foundations) | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Hack](https://github.com/source-foundry/Hack) | MIT + Bitstream Vera License (full upstream texts in `NOTICE`) | Hack-Regular.ttf is egui's bundled monospace face and GridCraft's last-resort symbol fallback. Dependency asset unmodified here, no font file copied into this repository. | | `epaint_default_fonts::NOTO_EMOJI_REGULAR` (dependency font input) | The Noto Project Authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Noto Emoji](https://fonts.google.com/noto/specimen/Noto+Emoji) | SIL Open Font License 1.1 (full upstream text in `NOTICE`) | NotoEmoji-Regular.ttf is enabled by egui's `monochrome_emoji_fonts` feature and used as the monochrome emoji fallback. Dependency asset unmodified here, no font file copied into this repository. | | `epaint_default_fonts::UBUNTU_LIGHT` (dependency font input) | Canonical Ltd. and the Ubuntu Font Family authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Ubuntu font](https://design.ubuntu.com/font) | Ubuntu Font Licence 1.0 (full upstream text in `NOTICE`) | Ubuntu-Light.ttf is egui's default proportional face. Dependency asset unmodified here, no font file copied into this repository. | -| `docs/brand/artcraft-logo.svg` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms (`docs/brand/LICENSE-brand.txt`) | Trademark, not open source | -| `docs/brand/artcraft-logo.png` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/artcraft-logo-white.png` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/artcraft-mark.svg` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/artcraft-mark.png` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/artcraft-mark-black.svg` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/artcraft-mark-black.png` | ArtCraft Team | craftrules `assets/brand/` | ArtCraft brand terms | Trademark | -| `docs/brand/LICENSE-brand.txt` | (licence text) | craftrules | — | | +| `docs/brand/artcraft-logo.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms (`docs/brand/LICENSE-brand.txt`) | Trademark, not open source | +| `docs/brand/artcraft-logo.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/artcraft-logo-white.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/artcraft-mark.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/artcraft-mark.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/artcraft-mark-black.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/artcraft-mark-black.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | +| `docs/brand/LICENSE-brand.txt` | (licence text) | project | — | | | `assets/app-icon/gridcraft.svg` | GridCraft contributors | original (hand-written SVG; `packaging/icons.sh`) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) | PLACEHOLDER app icon (ledger page); replaced by the owner's creature art later | | `packaging/macos/dmg/background.svg` | @XusBadia | original (hand-written SVG, derived from the GridCraft app icon `assets/app-icon/gridcraft.svg`, referenced via ``, not copied; text is outlined glyph paths; `packaging/macos/dmg/generate.py` renders it to `background.tiff`) | MIT OR Apache-2.0 (`LICENSE-MIT`, `LICENSE-APACHE`) | GridCraft macOS DMG window background for the Finder window; added 2026-10-08 | | `packaging/macos/dmg/background.tiff` | @XusBadia | Rendered from `packaging/macos/dmg/background.svg` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0 (`LICENSE-MIT`, `LICENSE-APACHE`) | 1x + 2x HiDPI TIFF for the macOS DMG window; includes the app icon (see its row) and LittleCMS's built-in sRGB profile ("No copyright, use freely") | @@ -46,3 +46,5 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra | `docs/images/pivot.png` | GridCraft contributors | screenshot of GridCraft rendered offscreen (`crates/ui-egui/examples/snapshot.rs`) showing an original sample workbook | MIT OR Apache-2.0 | README screenshot; no third-party content | | `docs/images/format-chart.png` | GridCraft contributors | screenshot of GridCraft rendered offscreen (`crates/ui-egui/examples/snapshot.rs`) showing an original sample workbook | MIT OR Apache-2.0 | README screenshot; no third-party content | | `docs/images/page-break.png` | GridCraft contributors | screenshot of GridCraft rendered offscreen (`crates/ui-egui/examples/snapshot.rs`) showing an original sample workbook | MIT OR Apache-2.0 | README screenshot; no third-party content | +| Core 14 AFM metric numbers in `crates/pdf/src/symbol.rs` (Symbol: character codes and advance widths) and `crates/pdf/src/metrics.rs` (Helvetica, Times and Courier advance widths); no font file | Adobe Systems Incorporated | Adobe's Core 14 AFM font metrics files (`Symbol.afm` 001.008 and the Helvetica, Times and Courier AFMs) | Adobe's Core 14 metrics notice: the files may be used, copied and distributed for any purpose and without charge, provided the notice is kept (quoted in `crates/pdf/src/metrics.rs`) | Numbers only, typed in from the public metrics. PDF export embeds no font: Symbol and the Times faces are base-14 fonts every PDF reader supplies, and text outside WinAnsiEncoding is set in a predefined, non-embedded Adobe-GB1 CID font | +| Unicode values of the Symbol encoding in `crates/pdf/src/symbol.rs` (the code-to-character map; numbers only, no file copied) | Unicode, Inc. | `symbol.txt`, "Adobe Symbol Encoding to Unicode", table version 1.0 (2011-07-12), at unicode.org/Public/MAPPINGS/VENDORS/ADOBE/ | Unicode, Inc.'s notice in the file: "Unicode, Inc. hereby grants the right to freely use the information supplied in this file in the creation of products supporting the Unicode Standard, and to make copies of this file in any form for internal or external distribution as long as this notice remains attached." The file is provided as-is, without warranties | Separate from Adobe's Core 14 notice (previous row), which covers the character codes and widths; the Unicode values come from this table | diff --git a/CLAUDE.md b/CLAUDE.md index 8bf527c54..3297f122a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,8 +2,6 @@ GridCraft is a clean-room, open-source, Rust-native spreadsheet targeting Microsoft Excel parity — and superiority (speed, openness, agent control). It runs natively on macOS, Windows, Linux and BSD, and on the web via WASM. Siblings with the same conventions: `../designcraft` (InDesign-class, the closest reference), `../photocraft` (Photoshop), `../vectorcraft` (Illustrator), `../pdfcraft` (Acrobat), `../filmcraft` (Premiere), `../lightcraft` (Lightroom), `../effectcraft` (After Effects). -Standards and learnings shared across the crafting apps live in `../../craftrules` (checked out next to the craft apps, or `storytold/craftrules`). Read its `AGENTS.md` at the start of a session, follow its standards, and contribute reusable learnings back there. Never code: repos don't share code. - ## Start every session here 1. Read `plan/STATUS.md` (current milestone, next task), then the task in `plan/execution-plan.md` and the relevant `plan/architecture.md` section. Behaviour reference: `plan/excel/*.md` (`observed-ui.md` holds measured observations of the running app). 2. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's. @@ -20,19 +18,19 @@ Standards and learnings shared across the crafting apps live in `../../craftrule - The ArtCraft brand files in `docs/brand/` are ArtCraft trademarks under `docs/brand/LICENSE-brand.txt`, the one exception. ## Never crash -People trust GridCraft with their numbers; a crash loses their work. **This outranks feature work.** Standard: [`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md). +People trust GridCraft with their numbers; a crash loses their work. **This outranks feature work.** - **No panics in non-test code:** no `unwrap()`, `expect()`, `panic!`, `unreachable!`, `todo!`, `unimplemented!`; no `unsafe` (`unsafe_code = "forbid"`). - **Errors are `Result`** through the crate's error type and `?`. - **Input-derived numbers are hostile** (files, formulas, commands, MCP/control params): `get()` instead of indexing, checked/saturating arithmetic, cap input-sized allocations (huge ranges, `REPT`, `SEQUENCE`, whole-column references). - **Bound recursion** (formula nesting, dependency chains, names referring to names). - **Last-resort guard:** `Session::execute` catches escaped panics and restores the workbook. It's a safety net, not a licence. - **Prove it:** every crash fix lands with a regression test (`tests::every_command_survives_empty_params` fuzzes every command; `crates/engine/tests/it/command_sequence_fuzz.rs` fuzzes random command sequences against undo/redo, XLSX round-trip and full-recalc invariants; soak it with `GRIDCRAFT_FUZZ_SEEDS`/`GRIDCRAFT_FUZZ_STEPS`, replay with `GRIDCRAFT_FUZZ_SEED`). -- Crates carry `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]`. +- Production crate roots carry `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]`; every new workspace crate must also include `[lints] workspace = true` in `Cargo.toml` to inherit `[workspace.lints]`. ## Non-negotiables - **Clean-room.** Microsoft Excel is installed on the dev machine and may be *observed* black-box: run it, use its UI with synthetic workbooks, take screenshots (by window id) stored only under `plan/excel/screenshots/`. Never read, disassemble or copy anything inside the Office bundle, never commit files produced by Excel, never copy GPL/AGPL/LGPL code (LibreOffice, Gnumeric…). Function behaviour comes from public documentation and observation. -- **Fonts live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts)**, never in this repo. It is an optional build input (`CRAFT_FONTS_DIR`); without it the app uses system fonts and egui's defaults. Standard: craftrules `standards/fonts.md`. -- **Test corpora** (real Excel-authored files) live in a separate corpus repo, fetched pinned and sha256-verified (craftrules `standards/test-corpora.md`). Never commit large binary fixtures; build test workbooks in code. +- **Fonts live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts)**, never in this repo. It is an optional build input (`CRAFT_FONTS_DIR`); without it the app uses system fonts and egui's defaults. +- **Test corpora** (real Excel-authored files) live in a separate corpus repo, fetched pinned and sha256-verified. Never commit large binary fixtures; build test workbooks in code. - **Everything is a command.** User-visible behaviour = a command in `crates/engine/src/cmd/*` (id, label, ribbon path, shortcut, params doc, `enabled`, `run`) + tests. Ids follow Excel's ribbon (`home.bold`, `insert.chart`, `data.sortAscending`) plus primitives (`cell.set`, `selection.set`, `range.setValues`). UI-only commands live in `crates/ui-egui`. The control channel, CLI and MCP reach all of them. - **Programmatic calls never open dialogs**: a command with empty params runs with defaults or returns an error; only menu-style invocation opens a dialog (`UiRequest::Dialog`). - **Layering** is enforced by `cargo xtask layers`: `core`/`numfmt` (L0) → `formula`/`functions` (L1) → `model`/`calc`/`xlsx` (L2) → `chart` (L3) → `engine` (L5) → `ui-egui`/`mcp` (L6) → `apps/*` (L7). Nothing below L6 depends on egui/eframe/winit/rfd. **The UI crate is swappable.** @@ -50,4 +48,9 @@ People trust GridCraft with their numbers; a crash loses their work. **This outr - Parallel agents: separate `CARGO_TARGET_DIR` per agent; edit only the crates you own; delete your target dir when done. ## Roadmap -`ROADMAP.md` (committed) is the one-page summary: stage, headline numbers, dimensions, languages, progress log. It follows craftrules `standards/progress-docs.md` and points at the detail in `docs/`: `target-app-parity.md` (authoritative assessment), `gaps.md` (ranked work list: pick from it), `roadmap.md` (current focus, beta gates, milestones), `architecture.md`, and the area checklists `function-parity.md`, `chart-parity.md`, `file-format-parity.md`, `ui-parity.md`, `hardware-parity.md`, `localization-parity.md`. Update the affected docs (and their timestamp lines and revision history) whenever work lands. `cargo xtask parity` regenerates `docs/parity-checklist.md` from `crates/engine/src/catalog.rs`. +`ROADMAP.md` (committed) is the one-page summary: stage, headline numbers, dimensions, languages, progress log. It points at the detail in `docs/`: `target-app-parity.md` (authoritative assessment), `gaps.md` (ranked work list: pick from it), `roadmap.md` (current focus, beta gates, milestones), `architecture.md`, and the area checklists `function-parity.md`, `chart-parity.md`, `file-format-parity.md`, `ui-parity.md`, `hardware-parity.md`, `localization-parity.md`. Update the affected docs (and their timestamp lines and revision history) whenever work lands. `cargo xtask parity` regenerates `docs/parity-checklist.md` from `crates/engine/src/catalog.rs`. + +## CI workflows +- Workflows in `.github/workflows/` call the shared workflows in [`storytold/craft-common-actions`](https://github.com/storytold/craft-common-actions) at `@main`, with their inputs and secrets. No other jobs: if no shared workflow does what's needed, it doesn't go here. +- Don't add or change CI in a feature or fix PR. If the existing workflows can't build or test a change, link an existing craft-common-actions issue or PR, or open a new issue, or a PR only for a small fix that is safe for every app and does not increase CI time. +- Some workflow files here still run their own jobs. Don't add to them. Moving a job into craft-common-actions is its own task, never part of another PR, and the job stays here until a shared workflow runs it. diff --git a/Cargo.lock b/Cargo.lock index 51d5d31d7..93356b73c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -20,7 +20,7 @@ dependencies = [ "accesskit", "accesskit_consumer", "atspi-common", - "phf", + "phf 0.14.0", "serde", "zvariant", ] @@ -220,7 +220,7 @@ dependencies = [ "objc2-foundation 0.3.2", "parking_lot", "percent-encoding", - "windows-sys 0.61.2", + "windows-sys 0.59.0", "wl-clipboard-rs", "x11rb", ] @@ -474,15 +474,30 @@ version = "1.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" +[[package]] +name = "bit-set" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3" +dependencies = [ + "bit-vec 0.8.0", +] + [[package]] name = "bit-set" version = "0.10.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09ec2f926cc3060f09db9ebc5b52823d85268d24bb917e472c0c4bea35780a7d" dependencies = [ - "bit-vec", + "bit-vec 0.9.1", ] +[[package]] +name = "bit-vec" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" + [[package]] name = "bit-vec" version = "0.9.1" @@ -895,12 +910,12 @@ dependencies = [ [[package]] name = "craft-fonts-build" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0" +source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7" [[package]] name = "craft-i18n" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0" +source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7" dependencies = [ "fixed_decimal", "icu_decimal", @@ -923,9 +938,10 @@ dependencies = [ [[package]] name = "craft-print" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0" +source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7" dependencies = [ "craft-winprint-gdi", + "log", "miniz_oxide 0.9.1", "roxmltree", "tempfile", @@ -936,7 +952,7 @@ dependencies = [ [[package]] name = "craft-ui" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0" +source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7" dependencies = [ "craft-fonts-build", "craft-i18n", @@ -956,13 +972,14 @@ dependencies = [ [[package]] name = "craft-update" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0" +source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7" dependencies = [ "chrono", "dunce", "egui", "flate2", "getrandom 0.3.4", + "image", "log", "plist", "semver", @@ -981,7 +998,7 @@ dependencies = [ [[package]] name = "craft-window" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=92d392cff132eaf53e13afff77f38b5c11b73aa0#92d392cff132eaf53e13afff77f38b5c11b73aa0" +source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7" dependencies = [ "egui", "objc2 0.6.5", @@ -1461,7 +1478,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -1499,6 +1516,17 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "fancy-regex" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e24cb5a94bcae1e5408b0effca5cd7172ea3c5755049c5f3af4cd283a165298" +dependencies = [ + "bit-set 0.8.0", + "regex-automata", + "regex-syntax", +] + [[package]] name = "fastrand" version = "2.5.0" @@ -1941,7 +1969,7 @@ dependencies = [ [[package]] name = "gridcraft" -version = "0.12.1" +version = "0.15.0" dependencies = [ "arboard", "chrono", @@ -1968,7 +1996,7 @@ dependencies = [ [[package]] name = "gridcraft-calc" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-core", "gridcraft-formula", @@ -1980,7 +2008,7 @@ dependencies = [ [[package]] name = "gridcraft-chart" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-calc", "gridcraft-core", @@ -1991,7 +2019,7 @@ dependencies = [ [[package]] name = "gridcraft-cli" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-engine", "gridcraft-mcp", @@ -2003,7 +2031,7 @@ dependencies = [ [[package]] name = "gridcraft-core" -version = "0.12.1" +version = "0.15.0" dependencies = [ "chrono", "js-sys", @@ -2014,13 +2042,14 @@ dependencies = [ [[package]] name = "gridcraft-engine" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-calc", "gridcraft-chart", "gridcraft-core", "gridcraft-formula", "gridcraft-functions", + "gridcraft-math", "gridcraft-model", "gridcraft-numfmt", "gridcraft-pdf", @@ -2030,12 +2059,13 @@ dependencies = [ "serde", "serde_json", "thiserror 2.0.21", + "web-time", "zip 3.0.0", ] [[package]] name = "gridcraft-formula" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-core", "serde", @@ -2044,15 +2074,25 @@ dependencies = [ [[package]] name = "gridcraft-functions" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-core", "regex", ] +[[package]] +name = "gridcraft-math" +version = "0.15.0" +dependencies = [ + "ratex-font", + "ratex-layout", + "ratex-parser", + "ratex-types", +] + [[package]] name = "gridcraft-mcp" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-engine", "log", @@ -2061,10 +2101,11 @@ dependencies = [ [[package]] name = "gridcraft-model" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-core", "gridcraft-formula", + "gridcraft-math", "serde", "serde_json", "sha2 0.11.1", @@ -2073,18 +2114,18 @@ dependencies = [ [[package]] name = "gridcraft-numfmt" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-core", ] [[package]] name = "gridcraft-pdf" -version = "0.12.1" +version = "0.15.0" [[package]] name = "gridcraft-ui-egui" -version = "0.12.1" +version = "0.15.0" dependencies = [ "craft-ui", "craft-update", @@ -2103,12 +2144,13 @@ dependencies = [ "sys-locale", "toml", "web-sys", + "web-time", "zbus", ] [[package]] name = "gridcraft-web" -version = "0.12.1" +version = "0.15.0" dependencies = [ "eframe", "egui", @@ -2125,12 +2167,13 @@ dependencies = [ [[package]] name = "gridcraft-xlsx" -version = "0.12.1" +version = "0.15.0" dependencies = [ "aes", "gridcraft-core", "gridcraft-formula", "gridcraft-functions", + "gridcraft-math", "gridcraft-model", "quick-xml 0.41.0", "sha1 0.11.0", @@ -2969,7 +3012,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a616d2fb8c89516ac2723a581f69d6c18576046bed761bd6b305e5618e6ae130" dependencies = [ "arrayvec", - "bit-set", + "bit-set 0.10.0", "bitflags 2.13.2", "cfg-if", "cfg_aliases", @@ -3464,7 +3507,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7d8fae84b431384b68627d0f9b3b1245fcf9f46f6c0e3dc902e9dce64edd1967" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -3545,17 +3588,37 @@ dependencies = [ "indexmap", ] +[[package]] +name = "phf" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078" +dependencies = [ + "phf_macros 0.11.3", + "phf_shared 0.11.3", +] + [[package]] name = "phf" version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "010378780309880b08997fae13be7834dba947d36393bd372f2b1556deb2a2f6" dependencies = [ - "phf_macros", - "phf_shared", + "phf_macros 0.14.0", + "phf_shared 0.14.0", "serde", ] +[[package]] +name = "phf_generator" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d" +dependencies = [ + "phf_shared 0.11.3", + "rand 0.8.8", +] + [[package]] name = "phf_generator" version = "0.14.0" @@ -3563,7 +3626,20 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "aeb62e0959d5a1bebc965f4d15d9e2b7cea002b6b0f5ba8cde6cc26738467100" dependencies = [ "fastrand", - "phf_shared", + "phf_shared 0.14.0", +] + +[[package]] +name = "phf_macros" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f84ac04429c13a7ff43785d75ad27569f2951ce0ffd30a3321230db2fc727216" +dependencies = [ + "phf_generator 0.11.3", + "phf_shared 0.11.3", + "proc-macro2", + "quote", + "syn 2.0.119", ] [[package]] @@ -3572,13 +3648,22 @@ version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5fa8d0ca26d424d27630da600c6624696e7dec8bf7b3b492b383c5dc49e5e085" dependencies = [ - "phf_generator", - "phf_shared", + "phf_generator 0.14.0", + "phf_shared 0.14.0", "proc-macro2", "quote", "syn 2.0.119", ] +[[package]] +name = "phf_shared" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5" +dependencies = [ + "siphasher", +] + [[package]] name = "phf_shared" version = "0.14.0" @@ -3854,7 +3939,7 @@ dependencies = [ "bytes", "getrandom 0.4.3", "lru-slab", - "rand", + "rand 0.10.3", "rand_pcg", "ring", "rustc-hash 2.1.3", @@ -3878,7 +3963,7 @@ dependencies = [ "once_cell", "socket2", "tracing", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -3902,6 +3987,15 @@ version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" +[[package]] +name = "rand" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e058c7de0b26af77780c769414d6257830bb240f3c38477dbc2c16e5f54d6d4c" +dependencies = [ + "rand_core 0.6.4", +] + [[package]] name = "rand" version = "0.10.3" @@ -3940,6 +4034,66 @@ version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ca45419789ae5a7899559e9512e58ca889e41f04f1f2445e9f4b290ceccd1d08" +[[package]] +name = "ratex-font" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a7b73531dabe934a3e725ed5c90d18dacf43778b7a208381390943a9a8c24db" +dependencies = [ + "phf 0.11.3", + "ratex-types", +] + +[[package]] +name = "ratex-layout" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1792738e2e36683cd20de742296a5233759565efc8cd03831064a0ccfb1fe0f7" +dependencies = [ + "ratex-font", + "ratex-parser", + "ratex-types", + "serde_json", +] + +[[package]] +name = "ratex-lexer" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f704add86d7bf4c4ecc7767f45ab38a44f1bbfa4b49725b2ae53c32e83b8a61e" +dependencies = [ + "ratex-types", + "serde", + "thiserror 1.0.69", +] + +[[package]] +name = "ratex-parser" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "079ec2e7ac6bc76137a6de093a173d1f901139a51318589ab9ce17139c21a6f0" +dependencies = [ + "fancy-regex", + "ratex-font", + "ratex-lexer", + "ratex-types", + "regex", + "regex-lite", + "serde", + "serde_json", + "thiserror 1.0.69", + "unicode-normalization", +] + +[[package]] +name = "ratex-types" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0cf0be3d050525f0f7d37af55a443d7648c222ccc617639950e3de1f8e6f9900" +dependencies = [ + "serde", +] + [[package]] name = "raw-window-handle" version = "0.6.2" @@ -4039,6 +4193,12 @@ dependencies = [ "regex-syntax", ] +[[package]] +name = "regex-lite" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973" + [[package]] name = "regex-syntax" version = "0.8.11" @@ -4188,7 +4348,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys 0.12.1", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -4247,7 +4407,7 @@ dependencies = [ "security-framework", "security-framework-sys", "webpki-root-certs", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -4630,7 +4790,7 @@ dependencies = [ "const-oid", "der", "hex", - "rand", + "rand 0.10.3", "reqwest", "rustls-pki-types", "rustls-webpki", @@ -4998,7 +5158,7 @@ dependencies = [ "getrandom 0.4.3", "once_cell", "rustix 1.1.5", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -5390,6 +5550,15 @@ version = "1.0.26" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954" +[[package]] +name = "unicode-normalization" +version = "0.1.26" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75cd3fb03d5288b3ba9a1ee103ac7e9cbb59f0b03d64f8ddfcc04dd42758a85d" +dependencies = [ + "tinyvec", +] + [[package]] name = "unicode-properties" version = "0.1.4" @@ -5830,8 +5999,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "14c018fce9b6270aa203c2fdd56f3cce996713534bd757e4ea58c8560b121f14" dependencies = [ "arrayvec", - "bit-set", - "bit-vec", + "bit-set 0.10.0", + "bit-vec 0.9.1", "bitflags 2.13.2", "bytemuck", "cfg_aliases", @@ -5903,7 +6072,7 @@ dependencies = [ "android_system_properties", "arrayvec", "ash", - "bit-set", + "bit-set 0.10.0", "bitflags 2.13.2", "block2 0.6.2", "bytemuck", @@ -5982,7 +6151,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -6449,7 +6618,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7" [[package]] name = "xtask" -version = "0.12.1" +version = "0.15.0" dependencies = [ "gridcraft-engine", "serde_json", diff --git a/Cargo.toml b/Cargo.toml index 32823c84a..8aa7ec8e7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"] default-members = ["crates/*", "apps/gridcraft", "apps/gridcraft-cli", "xtask"] [workspace.package] -version = "0.12.1" +version = "0.15.0" edition = "2024" license = "MIT OR Apache-2.0" rust-version = "1.95" @@ -24,9 +24,11 @@ gridcraft-engine = { path = "crates/engine" } gridcraft-ui-egui = { path = "crates/ui-egui" } gridcraft-mcp = { path = "crates/mcp" } gridcraft-pdf = { path = "crates/pdf" } +gridcraft-math = { path = "crates/math" } # Every craft-libs crate (craft-update and craft-window here, craft-ui in ui-egui, craft-print in # the app) comes from one storytold/craft-libs revision. -craft-update = { git = "https://github.com/storytold/craft-libs", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0", version = "=0.1.0", features = ["egui"] } +# craft-update: app-specific pretend versions apply to startup and manual checks. +craft-update = { git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7", version = "=0.1.0", features = ["egui"] } # foundation thiserror = "2" @@ -34,7 +36,8 @@ serde = { version = "1", features = ["derive", "rc"] } serde_json = { version = "1", features = ["float_roundtrip"] } log = "0.4" # All date/time conversion goes through chrono (no hand-rolled calendar code beyond the 400-year range extension; AGENTS.md). -chrono = { version = "0.4.45", default-features = false, features = ["std"] } +chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] } +web-time = "1.1.0" rayon = "1" regex = { version = "1", default-features = false, features = ["std", "unicode", "perf"] } quick-xml = "0.41" @@ -45,6 +48,13 @@ aes = { version = "0.9", default-features = false } sha1 = { version = "0.11", default-features = false } image = { version = "0.25", default-features = false, features = ["png", "jpeg"] } +# equations: RaTeX (MIT) lays LaTeX out as positioned glyphs, rules and paths. Exact pins: it is +# young (0.1.x), and the gate recorded in docs/decisions.md vetted this release. +ratex-parser = "=0.1.14" +ratex-layout = "=0.1.14" +ratex-types = "=0.1.14" +ratex-font = "=0.1.14" + # ui egui = "0.37" egui_extras = { version = "0.37", default-features = false } @@ -61,16 +71,17 @@ skrifa = { version = "0.44", default-features = false, features = ["std"] } epaint_default_fonts = "0.37" # One native window per workbook: the window list, Window menu titles and the immediate-viewport # driver. Same craft-libs revision as craft-update above. -craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0", default-features = false, features = ["egui"] } +craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7", default-features = false, features = ["egui"] } [workspace.lints.rust] unsafe_code = "forbid" [workspace.lints.clippy] +cognitive_complexity = "deny" +too_many_arguments = "deny" +wildcard_imports = "deny" dbg_macro = "warn" todo = "warn" -too_many_arguments = "allow" -type_complexity = "allow" [profile.dev.package."*"] opt-level = 2 diff --git a/README.md b/README.md index d9dc5eb36..f2d9a930e 100644 --- a/README.md +++ b/README.md @@ -298,7 +298,15 @@ stand on its own. | | **PdfCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/pdfcraft) | [Website](https://getartcraft.com/apps/pdfcraft) | | | **EffectCraft** | Motion graphics and visual effects | [GitHub](https://github.com/storytold/effectcraft) | [Website](https://getartcraft.com/apps/effectcraft) | | | **DesignCraft** | Page layout and publishing | [GitHub](https://github.com/storytold/designcraft) | [Website](https://getartcraft.com/apps/designcraft) | -| | **GridCraft** | **Spreadsheets and data · you are here** | [GitHub](https://github.com/storytold/gridcraft) | [Website](https://getartcraft.com/apps/gridcraft) | +| | **FigCraft** | Interface design and prototyping | [GitHub](https://github.com/storytold/figcraft) | [Website](https://getartcraft.com/apps/figcraft) | +| | **WordCraft** | Word processing and documents | [GitHub](https://github.com/storytold/wordcraft) | [Website](https://getartcraft.com/apps/wordcraft) | +| | **SoundCraft** | Audio recording and editing | [GitHub](https://github.com/storytold/soundcraft) | [Website](https://getartcraft.com/apps/soundcraft) | +| | **DeckCraft** | Presentations and slide decks | [GitHub](https://github.com/storytold/deckcraft) | [Website](https://getartcraft.com/apps/deckcraft) | +| | **CADCraft** | Computer-aided design | [GitHub](https://github.com/storytold/cadcraft) | [Website](https://getartcraft.com/apps/cadcraft) | +| | **GridCraft** | Spreadsheets and data | [GitHub](https://github.com/storytold/gridcraft) | [Website](https://getartcraft.com/apps/gridcraft) | +| | **TextureCraft** | 3D texture painting | [GitHub](https://github.com/storytold/texturecraft) | [Website](https://getartcraft.com/apps/texturecraft) | +| | **Craft Anything** | See how any software works, then build your own | [GitHub](https://github.com/storytold/craft-anything) | [Website](https://getartcraft.com/apps/craft-anything) | +| | **SolidCraft** | Parametric 3D CAD by Tangency Labs (community app) | [GitHub](https://github.com/tangency-labs/solidcraft) | | And [**ArtCraft**](https://getartcraft.com/) itself, our AI image and video studio for artists who want real control. diff --git a/ROADMAP.md b/ROADMAP.md index 6b9b0fce7..9acd2e13f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-25 · **Change:** minor (AutoSum over a selected row puts its total to the right, and fills a selection's empty last row and column) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -22,7 +22,7 @@ This page is the summary. The detail is in: | Number | Value | Kind | |---|---|---| | 2026-10-14 | Editing a cell with text that would read back as a number, boolean, date or formula (entered with a leading apostrophe) shows the apostrophe in the editor and formula bar, so Enter unchanged keeps it text (#1362) | -| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 274 / 290 (94.5%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | +| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 277 / 292 (94.9%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | | **Ready for real work** | **~50%** | estimated from written weights (51.6); [target-app-parity.md](docs/target-app-parity.md) | | **Mainstream practitioner** | **~50%** | estimated: typical analyst/finance user, weekly features, × 0.90 interaction × 0.92 stability × 0.88 file exchange; [method](docs/target-app-parity.md#mainstream-practitioner) | | **Essentials user** | **~63%** | estimated: core features only, × 0.92 launch/stability × 0.95 clarity × 0.90 opening sent files; [method](docs/target-app-parity.md#essentials-user) | @@ -86,7 +86,7 @@ collaboration, mobile, AI). | Get & Transform, external data, data model | ~15% | ~10% | 60–100 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | Insert objects (pictures, shapes, ink, SmartArt, equations) | ~55% | ~45% | 25–40 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | Review, comments, protection, collaboration | ~55% | ~40% | 35–55 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | -| Automation (recorded scripts; no VBA) | ~30% | ~20% | 12–20 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | +| Automation (recorded scripts; typed macros with a recorder, a Macros dialog and a step editor; no VBA) | ~30% | ~20% | 12–20 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | ## Languages @@ -135,7 +135,19 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | What landed | |---|---| +| 2026-11-02 | Python mode in the formula bar: Ctrl+Alt+Shift+P, `=PY` then Tab, or editing a Python cell edits its code in the formula bar, with Enter for a new line, Tab to indent and Ctrl+Enter to commit through `python.set`. The code still doesn't run | +| 2026-11-02 | Python cells as data: `python.set` stores code in a cell, Python cells copy, move, fill and undo like formulas, and `.xlsx` reads and writes Python cells as [MS-XLSX] specifies (`_xlfn._xlws.PY(index,0 or 1)`, the code in the workbook's Python part) with cached values and spills, so such workbooks open with their results; nothing runs Python (new code shows `#BLOCKED!`), and opening a file limits code size, result size and spill ranges | +| 2026-11-02 | Equations: Insert › Equation (a split button with a gallery of examples) and an editor with quick-insert buttons, the error underlined at its position and a live preview; `insert.equation` and `equation.edit`; LaTeX laid out by RaTeX inside the new `gridcraft-math` crate, capped and panic-guarded, and cached on the shape so that painting never runs the layout engine; xlsx writes Office Math with a plain-text fallback and the source in `descr`, and reads Office Math back into LaTeX; PDF prints equations with the standard Symbol font and `[?]` for anything else | +| 2026-11-02 | Solver writes a Sensitivity report: Simplex LP shadow prices, reduced costs and allowable ranges from the final tableau; GRG Nonlinear reduced gradients and Lagrange multipliers from difference gradients, at a Karush-Kuhn-Tucker point only; Solver Results offers it where it applies (no score change) | +| 2026-11-02 | Floating objects: charts, pictures, shapes and slicers share one selection with a frame of eight resize handles (each moves only its own edges, also on right-to-left sheets); Delete or Backspace deletes the selected object and Escape drops the selection; slicers follow row and column edits and sorts like the other objects; chart series follow cut-moves and sheet renames and deletes; a copied sheet gives its objects fresh ids; `ui.inspect` reports `selectedObject` and `ui.object.select` selects any object (#190, #213) | +| 2026-11-01 | Chart data tables are available from Chart Design › Add Chart Element and Format Chart Area. Tables follow live source values and number formats, keep blank cells blank, offer legend keys and border settings, and survive XLSX save/reopen; the shared renderer includes them in print and PDF. Readiness estimates unchanged. | +| 2026-11-01 | XLSX keeps the first visible workbook window's active sheet and display settings together, falling back to the first view when all are hidden. Tab ratios are normalized to 0–1000 on import and export; invalid numeric text uses 600. Synthetic round trips cover isolated attributes, missing views, and active tabs after sheet deletion/reordering; firstSheet remains separate #2264 work (#3057). | +| 2026-10-31 | Esc in File › Info, Print or Options first leaves the text field; the next Esc returns to the sheet (#3035) | +| 2026-10-31 | CSV and text open detects each column's date order: a value that reads only day first (`31/05/2026`) makes the column day first, a value that reads only month first (`12/31/2026`) makes it month first, an all-ambiguous column keeps month first with a warning, and a column with both kinds stays text with a warning (#2765) | +| 2026-10-31 | Recently Used lists the functions most recently entered in a formula from any source (ribbon, control channel, MCP, scripts), kept with the preferences and returned by `formulas.recentlyUsed`; the Insert Function dialog takes Excel's title and a working category list (Most Recently Used, All, each category) | +| 2026-10-31 | Formulas › Function Library as a big Insert Function button, the AutoSum split button and three columns of labelled small buttons, each category book in its own colour; category menus end with Insert Function… on that category; More Functions cascades into the remaining categories; Home › Editing shares the AutoSum menu; the MR key tip opens Recently Used | | 2026-10-25 | AutoSum over a selection does what Excel does: a selected row gets one total to its right (in its last cell if that is empty), not a one-cell total under each cell; a block whose last column is empty gets each row's total there, one whose last row is empty each column's, and with both the corner gets the total of the block; otherwise totals go under each column as before | +| 2026-10-24 | Macro interface ([macros.md](docs/macros.md#in-the-window)): an Automate › Macros group (Record Macro, which turns into Stop Recording, Use Relative References, Run Macro with recent macros); "Recording · N steps" with Stop in the status bar; a Record Macro dialog (checked name, free Ctrl+Shift shortcut, description); the Macros dialog lists macros beside Automate scripts and a VBA project, with Run, Step Into, Edit, Create, Delete and Options; Ctrl+Shift+letter macro shortcuts; questions that offer Run, Overwrite, Shift Down or Shift Right; a step editor with a breakpoint gutter, drag to reorder, a command picker, reference boxes that switch between relative tokens and absolute references, Inputs and Output tabs, a debug bar (Check, Run, Step, Continue, Run to Here, Stop with keep or undo) and grid overlays for inputs, writes, conflicts and the current step | | 2026-10-24 | Go To lists the named ranges of every sheet (`Sheet2!Name` for a name scoped to another sheet; hidden names, constants and formulas left out) and the references entered before, a double-click goes to an entry, and the Reference box takes several names or references separated by commas (`Price, Type`, `B14:C22,F19:G30`) | | 2026-10-24 | Status bar: a right-click opens a menu that sets which selection statistics show (Average, Count and Sum by default; Numerical Count, Minimum and Maximum are new), kept with the preferences (`view.statusBar`) | | 2026-10-24 | Ctrl+Shift+U (Control+Shift+U on a Mac) expands and collapses the formula bar, on the grid and while editing (`view.expandFormulaBar`); on Windows and Linux it used to underline | @@ -166,6 +178,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-14 | Lookups (MATCH, XMATCH, VLOOKUP, HLOOKUP, XLOOKUP, LOOKUP) compare numbers exactly like Excel, so a computed key such as `=0.1+0.2` no longer finds 0.3; `=` and COUNTIF/SUMIF keep their 15-digit comparison | | 2026-10-14 | Functions: EOMONTH, EDATE, WORKDAY, NETWORKDAYS, YEARFRAC, WEEKNUM, the bond and coupon functions, DISC, BIN2DEC, IM* and the other Analysis ToolPak functions give `#VALUE!` for a reference to more than one cell in a value argument, as Excel 365 does, instead of spilling; array constants and computed arrays still lift | | 2026-10-14 | Protect Sheet on a protected sheet: Home › Format, its keytips (HOP, RPS) and the tab menu now read Unprotect Sheet… and unprotect (straight away without a password); a protected sheet that forbids selecting locked (or unlocked) cells is enforced for clicks, arrows and Go To, and the Protect Sheet dialog has the select options (#1346) | +| 2026-10-14 | `.xls` (Excel 97-2003) files open with their formatting: fonts, solid and pattern fills, borders (with diagonals), alignment (rotation, indent, wrap, shrink), locked and hidden cells, the workbook's custom colour palette, its Normal font and its user-defined cell styles | | 2026-10-14 | XLSX: formulas opened with dynamic-array metadata (`cm`) keep it on save, with `xl/metadata.xml`. Before, a formula returning one value lost it (11 of 40 corpus files that have the part lost it entirely), and Excel recalculated some of them with implicit intersection (`SUMPRODUCT(IFERROR(...))` changed value) | | 2026-10-14 | Recalculation: a dynamic array's anchor given a new formula that reads its old spill area (`A1` was `=SEQUENCE(3)`, becomes `=SUM(A2:A3)+1` or `=SEQUENCE(1,A2+1)`) sees that area empty instead of the old values | | 2026-10-14 | Recalculation: when two arrays of one recalculation want the same empty cells, the formulas that read the one laid out second (at its anchor or over the cells it would have spilled into) read `#SPILL!` and empty cells instead of its values | @@ -179,6 +192,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-13 | XLSX: `[Content_Types].xml`, `_rels/.rels`, the workbook rels and `xl/workbook.xml` are the first zip entries (document properties last), so libmagic and PHP `finfo` detect the file type even with large properties (#1090) | | 2026-10-13 | Conditional formatting: a rule gathers range statistics only when it reads them (top/bottom, above average, duplicates, scales, bars, icons) and borrows them instead of copying them per cell, so the per-frame lookup of an expression or cell-value rule no longer visits its whole range (#1098, part 1) | | 2026-10-12 | Next/previous sheet navigation stops at the first/last visible worksheet instead of activating a hidden boundary sheet; the current selection is retained | +| 2026-10-21 | Macro engine ([macros.md](docs/macros.md)): typed, declarative macros saved in the workbook (a JSON field; hidden names in xlsx); declared inputs checked with messages that name the cells; tokens (`Input.col(2)`, `Row`, `Output`, `Name:Total`, `{{…}}` in formulas); one undo step with full rollback; a footprint dry run with ask, fail, overwrite, shift down or shift right; a policy that keeps macros away from files and the app, with approval for macros in opened files; Repeat, If and Stop steps; a debugger; a relative recorder; `macro.*` commands beside the unchanged Automate scripts | | 2026-10-12 | Security: the `--control` server requires an auth token (random, kept in a 0600 file or chosen with `GRIDCRAFT_CONTROL_TOKEN`) on each connection's first line and closes bad connections (#777) | | 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) | | 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) | @@ -231,7 +245,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-11 | PivotCharts (`insert.pivotChart`): charts bound to a PivotTable's report, following refreshes and layout changes (#311) | | 2026-10-11 | FORECAST.ETS, FORECAST.ETS.CONFINT, FORECAST.ETS.SEASONALITY, FORECAST.ETS.STAT: AAA exponential smoothing forecasts (#314) | | 2026-10-11 | File tab Backstage view (Home, New, Open, Info, Save As, Print preview, Export, Options) and `file.options` (#174) | -| 2026-10-10 | Ribbon translated into Japanese, Chinese, Korean, Brazilian Portuguese and Russian with system-language default (#31, #104, #82); CJK/Thai/Hebrew font fallback (#38); clear messages for encrypted and `.xls` files (#17); smooth line charts (#62). Progress docs restructured to the craftrules standard; full re-measure (this page). Initial ribbon key tips (#43), in-place text box editing (#84), copy as HTML table (#36). ODS and XLSB data import (#100, #101); recalc linear in the number of range formulas (#66); RISC-V Linux build; Flatpak repository on Pages; web cell-commit trap fixed | +| 2026-10-10 | Ribbon translated into Japanese, Chinese, Korean, Brazilian Portuguese and Russian with system-language default (#31, #104, #82); CJK/Thai/Hebrew font fallback (#38); clear messages for encrypted and `.xls` files (#17); smooth line charts (#62). Progress docs restructured; full re-measure (this page). Initial ribbon key tips (#43), in-place text box editing (#84), copy as HTML table (#36). ODS and XLSB data import (#100, #101); recalc linear in the number of range formulas (#66); RISC-V Linux build; Flatpak repository on Pages; web cell-commit trap fixed | | 2026-10-09 | Community fixes: spill range operator `A1#`, sheet-scoped names, protection refuses locked-cell edits, Insert/Delete Cells adjust references, `DATE(1900,2,29)`, SUMIF sizing, `ROUND` at 15 digits, `_xlpm.` LAMBDA parameters in XLSX, vertical scrollbar, system appearance, border previews; v0.4.0 | | 2026-10-08 | v0.2.0 and v0.3.0 releases; Flatpak, AppImage zsync, branded DMG; desktop log file; DirectX 12 default on Windows | | 2026-10-07 | About window with contributor and model credits; macOS release runner | @@ -242,7 +256,19 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| +| 2026-11-02 | minor | Progress log: Python mode in the formula bar | +| 2026-11-02 | minor | Progress log: Python cells as data | +| 2026-11-02 | minor | Progress log: equations (Insert › Equation, the editor, gridcraft-math) | +| 2026-11-02 | minor | What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear (no score change) | +| 2026-11-02 | minor | Progress log: one selection and eight resize handles for every floating object | +| 2026-11-01 | minor | Chart data tables are available from Chart Design › Add Chart Element and Format Chart Area. Tables follow live source values and number formats, keep blank cells blank, offer legend keys and border settings, and survive XLSX save/reopen; the shared renderer includes them in print and PDF. Readiness estimates unchanged. | +| 2026-11-01 | trivial | Workbook views: selected window settings stay together, tab ratios are bounded, and regressions cover isolated settings and sheet edits; readiness estimates unchanged (#3057) | +| 2026-10-31 | minor | File-page text fields keep the view open when Esc cancels an edit (#3035) | +| 2026-10-31 | minor | Progress log: Recently Used and Insert Function categories | +| 2026-10-31 | minor | Progress log: Function Library | | 2026-10-25 | minor | Progress log: AutoSum over a selected row, and a selection's empty last row and column | +| 2026-10-24 | minor | Macro interface on the Automate tab; progress log row | +| 2026-10-24 | minor | Macro engine and `macro.*` commands; catalog 275 / 291 → 276 / 292 (View › Macros › Record Macro); progress log row | | 2026-10-23 | minor | Progress log: accounting negatives in number text | | 2026-10-23 | minor | Progress log: INDEX with one index on a 2-D reference is #REF! | | 2026-10-22 | trivial | Progress log: formula hardening (#2163, #2162) | @@ -258,6 +284,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-16 | minor | Progress log: floating pictures' textures bounded and kept per workbook | | 2026-10-16 | trivial | Progress log: web preferences kept between visits (#1562) | | 2026-10-16 | minor | Progress log: interface fonts held once instead of twice | +| 2026-10-14 | minor | Progress log: `.xls` import reads cell formatting | | 2026-10-14 | minor | Progress log: lookups compare numbers exactly | | 2026-10-14 | minor | Progress log: Analysis ToolPak value arguments reject multi-cell references | | 2026-10-14 | minor | XLSX dynamic-array `cm` metadata kept on save; progress log row | @@ -292,5 +319,5 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-10 | minor | Readiness-by-audience table with hours to ~95% per audience; full number unchanged (additive weighted sum) | | 2026-10-10 | minor | Added Mainstream practitioner (~50%) and Essentials user (~63%); languages updated for the 10-10 merges (ribbon in ja, zh, ko, pt-BR, ru); localization 5%→10%; full number re-checked (51.6, unchanged) | | 2026-10-10 | minor | Checked against the alpha gate (six core workflows pass): stays alpha | -| 2026-10-10 | major | Re-measured against Excel for Mac 16.113.4: stage alpha, breadth ~75%, ready ~50%, ~160–240 h to beta, ~700–1,100 h to full parity; restructured to the progress-docs standard; detail moved to `docs/` | +| 2026-10-10 | major | Re-measured against Excel for Mac 16.113.4: stage alpha, breadth ~75%, ready ~50%, ~160–240 h to beta, ~700–1,100 h to full parity; progress docs restructured; detail moved to `docs/` | | 2026-10-07 | major | First estimates: catalog 89%, functions ~93%, depth ~65%, ~35 h to alpha, ~215 h to full parity | diff --git a/apps/gridcraft-cli/src/check/fidelity.rs b/apps/gridcraft-cli/src/check/fidelity.rs index 446a8949a..d8def2747 100644 --- a/apps/gridcraft-cli/src/check/fidelity.rs +++ b/apps/gridcraft-cli/src/check/fidelity.rs @@ -1,6 +1,6 @@ use super::calc::formula_text; use super::package::{REBUILT_KINDS, relationship_kinds, validate_package}; -use super::*; +use super::{BTreeSet, Category, Cell, CellError, CellRef, Diffs, Options, RoundTrip, Sheet, Value, Workbook, number_text, quote_sheet}; // ---------------------------------------------------------------- round trip diff --git a/apps/gridcraft-cli/src/check/package.rs b/apps/gridcraft-cli/src/check/package.rs index bcbbef10f..6b97fa495 100644 --- a/apps/gridcraft-cli/src/check/package.rs +++ b/apps/gridcraft-cli/src/check/package.rs @@ -20,7 +20,9 @@ fn local_name(name: &[u8]) -> &[u8] { } /// `(element local name, attributes)` for every start/empty element of an XML part. -fn xml_elements(xml: &[u8]) -> Result)>, String> { +type XmlElement = (String, Vec<(String, String)>); + +fn xml_elements(xml: &[u8]) -> Result, String> { use quick_xml::events::Event; let mut reader = quick_xml::Reader::from_reader(xml); let mut buf = Vec::new(); diff --git a/apps/gridcraft-cli/src/check/report.rs b/apps/gridcraft-cli/src/check/report.rs index 54227e797..6c0d6b36d 100644 --- a/apps/gridcraft-cli/src/check/report.rs +++ b/apps/gridcraft-cli/src/check/report.rs @@ -131,7 +131,7 @@ fn fit(s: &str, w: usize) -> String { if n <= w { s.to_string() } else { format!("…{}", s.chars().skip(n + 1 - w).collect::()) } } -pub(super) fn print_human(reports: &[FileReport], opts: &Options) { +fn print_file_reports(reports: &[FileReport]) { for r in reports { outln!("== {}", r.path); if let Some(e) = &r.error { @@ -186,7 +186,9 @@ pub(super) fn print_human(reports: &[FileReport], opts: &Options) { } } } - outln!(""); +} + +fn print_file_table(reports: &[FileReport]) { let w = reports.iter().map(|r| r.path.chars().count()).max().unwrap_or(4).clamp(4, 50); outln!( "{:8} {:>8} {:>8} {:>8} {:>9} {:>8} round trip", @@ -220,7 +222,9 @@ pub(super) fn print_human(reports: &[FileReport], opts: &Options) { c.volatile ); } - let t = totals(reports); +} + +fn print_rankings(t: &Totals, opts: &Options) { let top = ranked(&t.functions); if !top.is_empty() { outln!(""); @@ -239,6 +243,9 @@ pub(super) fn print_human(reports: &[FileReport], opts: &Options) { outln!(" {n:>8} {sig}"); } } +} + +fn print_totals(t: &Totals) { outln!(""); outln!("files: {} ({} opened, {} failed to open)", t.files, t.opened, t.failed); outln!( @@ -262,3 +269,12 @@ pub(super) fn print_human(reports: &[FileReport], opts: &Options) { if cats.is_empty() { String::new() } else { format!("; categories (files): {}", cats.join(", ")) } ); } + +pub(super) fn print_human(reports: &[FileReport], opts: &Options) { + print_file_reports(reports); + outln!(""); + print_file_table(reports); + let t = totals(reports); + print_rankings(&t, opts); + print_totals(&t); +} diff --git a/apps/gridcraft-cli/tests/it/main.rs b/apps/gridcraft-cli/tests/it/main.rs index 17f8db87f..5906b3967 100644 --- a/apps/gridcraft-cli/tests/it/main.rs +++ b/apps/gridcraft-cli/tests/it/main.rs @@ -1,5 +1,3 @@ //! The crate's integration tests, one module per file, built as one test binary. - -#[allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] mod cli; mod text_to_columns_quotes; diff --git a/apps/gridcraft-web/src/web.rs b/apps/gridcraft-web/src/web.rs index 43923bde1..10e17dc44 100644 --- a/apps/gridcraft-web/src/web.rs +++ b/apps/gridcraft-web/src/web.rs @@ -230,8 +230,9 @@ fn services(inbox: Inbox, ctx: egui::Context) -> Services { && let gridcraft_engine::links::LinkKind::External(url) = gridcraft_engine::links::classify(url) { // With `noopener` the new browsing context can't hold a reference back to the - // editor window and navigate it while the user reads the linked page. - let _ = w.open_with_url_and_target(&url, "_blank,noopener"); + // editor window and navigate it while the user reads the linked page. It must be a + // window *feature*: inside the target string it would only name the new window. + let _ = w.open_with_url_and_target_and_features(&url, "_blank", "noopener,noreferrer"); } })), inbox: Some(inbox), diff --git a/apps/gridcraft/Cargo.toml b/apps/gridcraft/Cargo.toml index 58868556e..aaa012934 100644 --- a/apps/gridcraft/Cargo.toml +++ b/apps/gridcraft/Cargo.toml @@ -34,7 +34,7 @@ winit = { version = "0.30", default-features = false, features = ["x11", "waylan [target.'cfg(not(target_arch = "wasm32"))'.dependencies] # Same craft-libs revision as the workspace's craft-update and craft-window (see the root # Cargo.toml). -craft-print = { git = "https://github.com/storytold/craft-libs", rev = "92d392cff132eaf53e13afff77f38b5c11b73aa0" } +craft-print = { git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7" } craft-update = { workspace = true } rfd = { workspace = true } arboard = { workspace = true } @@ -56,6 +56,10 @@ muda = { version = "0.21", default-features = false } # by PdfCraft and PhotoCraft) registers the handler (Objective-C interop needs unsafe, which our # workspace forbids) behind a safe API. Pinned exactly: review the diff before bumping. fmv-macos-events = "=0.1.0" +# apple_events.rs brings the app forward for a Dock quit's question. Already in Cargo.lock (muda, +# winit); pinned exactly to the same versions. +objc2 = "=0.6.5" +objc2-app-kit = { version = "=0.3.2", default-features = false, features = ["std", "NSApplication", "NSResponder", "NSWindow"] } # tests/mac_menu_appkit.rs: reads AppKit's menu bar back. Already in Cargo.lock (muda, winit); # pinned exactly to the same versions. diff --git a/apps/gridcraft/src/apple_events.rs b/apps/gridcraft/src/apple_events.rs index 4d596f957..b0898eced 100644 --- a/apps/gridcraft/src/apple_events.rs +++ b/apps/gridcraft/src/apple_events.rs @@ -11,6 +11,8 @@ use fmv_macos_events::{Event, Inbox, Registration}; use gridcraft_ui_egui::SheetApp; +use objc2::MainThreadMarker; +use objc2_app_kit::{NSApplication, NSRequestUserAttentionType}; /// Keeps the Apple-event handlers registered; hold it until the event loop returns. pub struct AppleEvents { @@ -44,7 +46,32 @@ pub fn poll(inbox: &Inbox, app: &mut SheetApp, ctx: &egui::Context) { } } // Quit from the Dock or the system: each unsaved workbook is asked about in its window. - Event::Quit => app.request_quit(ctx), + Event::Quit => { + app.request_quit(ctx); + if app.dialog.is_some() { + bring_app_forward(); + } + } } } } + +/// A Dock quit reaches GridCraft behind other apps, hidden or minimized, and its question has to +/// be seen. egui's Focus goes through winit, which skips a minimized window or a hidden app, and +/// macOS may refuse an activation it didn't expect, so ask AppKit directly and bounce the Dock +/// icon until the user switches over. +fn bring_app_forward() { + let Some(mtm) = MainThreadMarker::new() else { return }; + let ns_app = NSApplication::sharedApplication(mtm); + ns_app.unhide(None); + for window in ns_app.windows().to_vec() { + if window.isMiniaturized() { + window.deminiaturize(None); + } + } + #[allow(deprecated)] // `activate` needs macOS 14; this works on every supported version. + ns_app.activateIgnoringOtherApps(true); + if !ns_app.isActive() { + ns_app.requestUserAttention(NSRequestUserAttentionType::CriticalRequest); + } +} diff --git a/apps/gridcraft/src/control_server.rs b/apps/gridcraft/src/control_server.rs index cc5a80c10..7cba6db2c 100644 --- a/apps/gridcraft/src/control_server.rs +++ b/apps/gridcraft/src/control_server.rs @@ -12,16 +12,47 @@ use std::io::{BufRead, BufReader, Read, Write}; use std::net::{TcpListener, TcpStream}; use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::mpsc::{Receiver, Sender, channel}; -use std::time::Duration; +use std::time::{Duration, Instant}; use gridcraft_mcp::control_auth; use gridcraft_ui_egui::ControlRequest; use serde_json::{Value as Json, json}; const MAX_LINE: usize = 64 * 1024 * 1024; -/// How long a new connection may take to send its token. +/// How long a new connection may take to send its token, in total. const AUTH_TIMEOUT: Duration = Duration::from_secs(10); +/// How many connections may be waiting to authenticate at once. Anyone on the machine can +/// connect without the token, so past this a new connection is closed before anything is read: +/// each waiting one holds a thread and a socket. +const MAX_PENDING_AUTH: usize = 16; + +/// How the server treats connections that haven't authenticated yet. +#[derive(Clone, Copy)] +struct AuthLimits { + timeout: Duration, + max_pending: usize, +} + +const AUTH_LIMITS: AuthLimits = AuthLimits { timeout: AUTH_TIMEOUT, max_pending: MAX_PENDING_AUTH }; + +/// A connection that is still waiting to authenticate; dropping it frees its place. +struct Pending(Arc); + +impl Pending { + /// A place among the `max` waiting connections, or `None` when they are all taken. + fn claim(waiting: &Arc, max: usize) -> Option { + waiting.try_update(Ordering::AcqRel, Ordering::Acquire, |n| (n < max).then_some(n + 1)).ok()?; + Some(Pending(Arc::clone(waiting))) + } +} + +impl Drop for Pending { + fn drop(&mut self) { + self.0.fetch_sub(1, Ordering::AcqRel); + } +} /// The secret of this run: the one the user chose (`--control-token`, `GRIDCRAFT_CONTROL_TOKEN`), /// else 256 random bits from the operating system, written to a file only the user can read. @@ -51,35 +82,75 @@ pub fn start(port: u16, chosen_token: Option, ctx: egui::Context) -> std /// Accept connections on `listener`; requests arrive on the returned receiver. fn serve_listener(listener: TcpListener, token: Arc, ctx: egui::Context) -> std::io::Result> { + serve_listener_with(listener, token, ctx, AUTH_LIMITS) +} + +fn serve_listener_with(listener: TcpListener, token: Arc, ctx: egui::Context, limits: AuthLimits) -> std::io::Result> { let (tx, rx) = channel(); + let waiting = Arc::new(AtomicUsize::new(0)); std::thread::Builder::new().name("control".into()).spawn(move || { - for stream in listener.incoming().flatten() { + for stream in listener.incoming() { + let Ok(stream) = stream else { + // Out of sockets or handles: wait for some to free instead of spinning on the error. + std::thread::sleep(Duration::from_millis(50)); + continue; + }; + // Dropping the stream closes it before anything is read. + let Some(place) = Pending::claim(&waiting, limits.max_pending) else { continue }; let tx = tx.clone(); let ctx = ctx.clone(); let token = Arc::clone(&token); - let _ = std::thread::Builder::new().name("control-conn".into()).spawn(move || serve(stream, &token, tx, ctx)); + let _ = + std::thread::Builder::new().name("control-conn".into()).spawn(move || serve(stream, &token, tx, ctx, limits.timeout, Some(place))); } })?; Ok(rx) } /// Read the connection's first line (at most [`control_auth::MAX_AUTH_LINE`] bytes, within -/// [`AUTH_TIMEOUT`]) and check the token. `true` when the peer may go on. -fn authenticate(reader: &mut BufReader, out: &mut TcpStream, token: &str) -> bool { - let _ = out.set_read_timeout(Some(AUTH_TIMEOUT)); - let mut first = String::new(); - let read = reader.by_ref().take(control_auth::MAX_AUTH_LINE as u64).read_line(&mut first); - let verdict = match read { - Ok(n) if n > 0 && first.ends_with('\n') => control_auth::check_first_line(&first, token), +/// `timeout` in total) and check the token. `true` when the peer may go on. +#[allow(clippy::disallowed_methods)] // the desktop app (native only): wasm never runs it +fn authenticate(reader: &mut BufReader, out: &mut TcpStream, token: &str, timeout: Duration) -> bool { + let deadline = Instant::now().checked_add(timeout).unwrap_or_else(Instant::now); + let mut first = Vec::new(); + let read = read_auth_line(reader, &mut first, deadline); + let verdict = match (read, std::str::from_utf8(&first)) { + (Ok(()), Ok(line)) if line.ends_with('\n') => control_auth::check_first_line(line, token), _ => Err(json!({"ok": false, "error": "authentication required"})), }; let allowed = verdict.is_ok(); let reply = verdict.unwrap_or_else(|e| e); let _ = writeln!(out, "{reply}"); - let _ = out.set_read_timeout(None); + let _ = reader.get_ref().set_read_timeout(None); allowed } +/// Read the first line, at most [`control_auth::MAX_AUTH_LINE`] bytes, by `deadline`. +/// +/// The timeout goes on the handle the reader reads: on Windows a `try_clone` of the stream keeps +/// its own, so one set on the writing half never fired (a silent peer held its thread forever). +/// It is shortened before every read, so a peer sending one byte at a time can't stretch it. +#[allow(clippy::disallowed_methods)] // the desktop app (native only): wasm never runs it +fn read_auth_line(reader: &mut BufReader, line: &mut Vec, deadline: Instant) -> std::io::Result<()> { + let max = control_auth::MAX_AUTH_LINE; + while line.len() < max && line.last() != Some(&b'\n') { + let left = deadline.saturating_duration_since(Instant::now()); + if left.is_zero() { + return Err(std::io::ErrorKind::TimedOut.into()); + } + reader.get_ref().set_read_timeout(Some(left))?; + let buf = reader.fill_buf()?; + if buf.is_empty() { + break; + } + let room = max.saturating_sub(line.len()); + let take = buf.iter().take(room).position(|b| *b == b'\n').map_or(buf.len().min(room), |i| i + 1); + line.extend(buf.iter().take(take)); + reader.consume(take); + } + Ok(()) +} + /// Read one `\n`-terminated line of at most `max` bytes into `buf` (newline stripped). /// `Ok(false)` at end of stream; `Err` when the line is too long or the read fails. fn read_bounded_line(reader: &mut impl BufRead, buf: &mut Vec, max: usize) -> Result { @@ -148,11 +219,14 @@ fn is_http_request_line(text: &str) -> bool { || text.rsplit_once(' ').is_some_and(|(_, version)| version.starts_with("HTTP/")) } -fn serve(stream: TcpStream, token: &str, tx: Sender, ctx: egui::Context) { +fn serve(stream: TcpStream, token: &str, tx: Sender, ctx: egui::Context, auth_timeout: Duration, place: Option) { let Ok(read) = stream.try_clone() else { return }; let mut out = stream; let mut reader = BufReader::new(read); - if !authenticate(&mut reader, &mut out, token) { + let allowed = authenticate(&mut reader, &mut out, token, auth_timeout); + // Authenticated or refused, the connection no longer counts as waiting. + drop(place); + if !allowed { return; } let mut buf = Vec::new(); @@ -294,7 +368,7 @@ mod tests { std::thread::spawn(move || { for stream in listener.incoming().flatten() { let tx = tx.clone(); - std::thread::spawn(move || serve(stream, TOKEN, tx, egui::Context::default())); + std::thread::spawn(move || serve(stream, TOKEN, tx, egui::Context::default(), AUTH_TIMEOUT, None)); } }); addr @@ -346,6 +420,81 @@ mod tests { assert_eq!(got[1]["ok"], json!(false)); } + /// Whether the server closed `c` (end of stream, or a refusal and then end of stream) within + /// `within`, and how long it took. + #[allow(clippy::disallowed_methods)] // a native-only test of the desktop app + fn closed_within(c: TcpStream, within: Duration) -> (bool, Duration) { + let start = Instant::now(); + c.set_read_timeout(Some(within)).unwrap(); + let mut r = BufReader::new(c); + let mut line = String::new(); + loop { + line.clear(); + match r.read_line(&mut line) { + Ok(0) => return (true, start.elapsed()), + Ok(_) => continue, + Err(e) if matches!(e.kind(), std::io::ErrorKind::WouldBlock | std::io::ErrorKind::TimedOut) => return (false, start.elapsed()), + Err(_) => return (true, start.elapsed()), + } + } + } + + /// The token deadline is for the whole first line: neither a silent peer nor one sending a + /// byte at a time keeps the connection (and its thread) past it. + #[test] + fn a_silent_or_slow_peer_is_closed_at_the_auth_deadline() { + let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap(); + let addr = listener.local_addr().unwrap(); + let limits = AuthLimits { timeout: Duration::from_millis(300), max_pending: MAX_PENDING_AUTH }; + let _rx = serve_listener_with(listener, TOKEN.into(), egui::Context::default(), limits).unwrap(); + + let silent = TcpStream::connect(addr).unwrap(); + let mut slow = TcpStream::connect(addr).unwrap(); + let mut drip = slow.try_clone().unwrap(); + std::thread::spawn(move || { + for b in control_auth::auth_line(1, TOKEN).bytes() { + if drip.write_all(&[b]).is_err() { + break; + } + std::thread::sleep(Duration::from_millis(50)); + } + }); + let (closed, took) = closed_within(silent, Duration::from_secs(5)); + assert!(closed && took < Duration::from_secs(3), "silent peer kept for {took:?}"); + let (closed, took) = closed_within(slow.try_clone().unwrap(), Duration::from_secs(5)); + assert!(closed && took < Duration::from_secs(3), "byte-at-a-time peer kept for {took:?}"); + let _ = slow.flush(); + } + + /// Peers without the token can't pile up threads: past the cap a new connection is closed at + /// once, and a waiting one that authenticates frees its place. + #[test] + fn connections_waiting_to_authenticate_are_capped() { + let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap(); + let addr = listener.local_addr().unwrap(); + let limits = AuthLimits { timeout: Duration::from_secs(20), max_pending: 2 }; + let _rx = serve_listener_with(listener, TOKEN.into(), egui::Context::default(), limits).unwrap(); + + let mut first = TcpStream::connect(addr).unwrap(); + let _second = TcpStream::connect(addr).unwrap(); + std::thread::sleep(Duration::from_millis(200)); + let (closed, took) = closed_within(TcpStream::connect(addr).unwrap(), Duration::from_secs(5)); + assert!(closed && took < Duration::from_secs(3), "a third waiting connection was kept"); + + first.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + first.write_all(format!("{}\n", control_auth::auth_line(1, TOKEN)).as_bytes()).unwrap(); + let mut reply = String::new(); + BufReader::new(first.try_clone().unwrap()).read_line(&mut reply).unwrap(); + assert_eq!(serde_json::from_str::(&reply).unwrap()["ok"], json!(true), "{reply}"); + std::thread::sleep(Duration::from_millis(200)); + let mut next = TcpStream::connect(addr).unwrap(); + next.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + next.write_all(format!("{}\n", control_auth::auth_line(1, TOKEN)).as_bytes()).unwrap(); + let mut reply = String::new(); + BufReader::new(next).read_line(&mut reply).unwrap(); + assert_eq!(serde_json::from_str::(&reply).unwrap()["ok"], json!(true), "a freed place was not reused: {reply}"); + } + #[test] fn a_chosen_token_must_be_long_enough() { assert!(make_token(1, Some("short".into())).is_err()); diff --git a/apps/gridcraft/src/graphics.rs b/apps/gridcraft/src/graphics.rs index 83478edde..93ec2e4c9 100644 --- a/apps/gridcraft/src/graphics.rs +++ b/apps/gridcraft/src/graphics.rs @@ -1,4 +1,5 @@ -//! Windows graphics instance configuration, applied before wgpu can initialize any driver. +//! Graphics instance configuration, applied before wgpu can initialize any driver: Windows +//! backends and the DirectX 12 shader compiler, and the Intel Mac indirect-validation skip. //! //! Vulkan is never initialized unless `WGPU_BACKEND` asks for it: enumerating Vulkan adapters //! crashes inside Intel's `igvk64.dll` on some UHD Graphics drivers (#120), and that access @@ -8,12 +9,38 @@ //! whichever `dxcompiler.dll` the DLL search path happens to find (#339, #354): see //! [`dx12_compiler`]. +// On macOS only the indirect-validation items are used outside tests; on Windows the other way +// round, so those two carry their own `allow`. +#![cfg_attr(target_os = "macos", allow(dead_code))] + use eframe::{ NativeOptions, egui_wgpu::WgpuSetup, - wgpu::{Backends, DeviceType, Dx12Compiler}, + wgpu::{Backends, DeviceType, Dx12Compiler, InstanceFlags}, }; +/// wgpu compiles its indirect-call validation compute shaders inside `request_device`. On Intel +/// Macs (Iris 5100 in a Mac mini 2014) Metal's compiler aborts on them ("Compiler encountered an +/// internal error"), the device is lost and eframe exits before the window opens. GridCraft issues +/// no indirect draws or dispatches, so Intel Macs skip them; every other GPU keeps wgpu's default. +/// Same as WordCraft (storytold/wordcraft#2652) and PhotoCraft (storytold/PhotoCraft#3171). +#[cfg_attr(not(target_os = "macos"), allow(dead_code))] +pub fn skips_indirect_validation(macos: bool, intel: bool) -> bool { + macos && intel +} + +/// Leaves wgpu's indirect-call validation out of the instance eframe creates when `skip`. +/// `WGPU_VALIDATION_INDIRECT_CALL=1` still turns it back on. +#[cfg_attr(not(target_os = "macos"), allow(dead_code))] +pub fn configure_indirect_validation(options: &mut NativeOptions, skip: bool) { + if !skip { + return; + } + if let WgpuSetup::CreateNew(create) = &mut options.wgpu_options.wgpu_setup { + create.instance_descriptor.flags = create.instance_descriptor.flags.difference(InstanceFlags::VALIDATION_INDIRECT_CALL).with_env(); + } +} + /// What the start-up probe learned about one DirectX 12 adapter. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub struct Dx12Adapter { @@ -141,6 +168,55 @@ fn probe_dx12(options: &NativeOptions) -> Vec { mod tests { use super::*; + fn instance_flags(options: &NativeOptions) -> InstanceFlags { + let WgpuSetup::CreateNew(create) = &options.wgpu_options.wgpu_setup else { + panic!("default native options must create a wgpu instance"); + }; + create.instance_descriptor.flags + } + + #[test] + fn intel_macs_skip_the_indirect_validation_shaders() { + // Metal's Intel compiler aborted on them, losing the device before the window opened. + assert!(skips_indirect_validation(true, true)); + // Apple silicon, Windows and Linux keep wgpu's validation. + for (macos, intel) in [(true, false), (false, true), (false, false)] { + assert!(!skips_indirect_validation(macos, intel), "macos={macos} intel={intel}"); + } + let mut options = NativeOptions::default(); + let before = instance_flags(&options); + configure_indirect_validation(&mut options, false); + assert_eq!(instance_flags(&options), before, "flags untouched when not skipping"); + if std::env::var_os("WGPU_VALIDATION_INDIRECT_CALL").is_none() { + configure_indirect_validation(&mut options, true); + assert!(!instance_flags(&options).contains(InstanceFlags::VALIDATION_INDIRECT_CALL)); + } + } + + /// `WGPU_VALIDATION_INDIRECT_CALL=1` turns the validation back on even when it is skipped. + /// Tests can't set environment variables (`set_var` is unsafe, and unsafe code is forbidden), + /// so this runs itself again as a child process with the variable set. + #[test] + fn the_environment_turns_indirect_validation_back_on() { + const NAME: &str = "graphics::tests::the_environment_turns_indirect_validation_back_on"; + if std::env::var_os("GRIDCRAFT_INDIRECT_VALIDATION_CHILD").is_some() { + let mut options = NativeOptions::default(); + configure_indirect_validation(&mut options, true); + assert!(instance_flags(&options).contains(InstanceFlags::VALIDATION_INDIRECT_CALL)); + return; + } + let exe = std::env::current_exe().unwrap(); + let out = std::process::Command::new(exe) + .args(["--exact", NAME, "--test-threads=1"]) + .env("GRIDCRAFT_INDIRECT_VALIDATION_CHILD", "1") + .env("WGPU_VALIDATION_INDIRECT_CALL", "1") + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&out.stdout); + // `--exact` matching nothing would pass too, so the child must report this one test. + assert!(out.status.success() && stdout.contains("1 passed"), "child test failed:\n{stdout}{}", String::from_utf8_lossy(&out.stderr)); + } + fn configured_backends(backends: Backends) -> Backends { let mut options = NativeOptions::default(); configure_backends(&mut options, backends); diff --git a/apps/gridcraft/src/logging.rs b/apps/gridcraft/src/logging.rs index 413797378..ca7d05717 100644 --- a/apps/gridcraft/src/logging.rs +++ b/apps/gridcraft/src/logging.rs @@ -268,9 +268,10 @@ impl log::Log for AppLogger { } // Formatted before the sink is locked, so a message that logs while it is formatted can't deadlock. let thread = std::thread::current(); - #[allow(clippy::disallowed_methods)] // the desktop app (native only): wasm never runs it - let line = - format_line(×tamp(SystemTime::now()), record.level(), thread.name().unwrap_or("?"), record.target(), &record.args().to_string()); + let logged_at = gridcraft_engine::core::date::unix_now_ms() + .and_then(|milliseconds| SystemTime::UNIX_EPOCH.checked_add(std::time::Duration::from_millis(milliseconds.max(0.0) as u64))) + .unwrap_or(SystemTime::UNIX_EPOCH); + let line = format_line(×tamp(logged_at), record.level(), thread.name().unwrap_or("?"), record.target(), &record.args().to_string()); if self.stderr { // No terminal (a Windows GUI build, a closed pipe) is not an error worth reporting. let _ = std::io::stderr().write_all(line.as_bytes()); diff --git a/apps/gridcraft/src/main.rs b/apps/gridcraft/src/main.rs index 988bf738b..3aab1ccfc 100644 --- a/apps/gridcraft/src/main.rs +++ b/apps/gridcraft/src/main.rs @@ -12,7 +12,7 @@ #[cfg(target_os = "macos")] mod apple_events; mod control_server; -#[cfg(any(target_os = "windows", test))] +#[cfg(any(target_os = "windows", target_os = "macos", test))] mod graphics; mod logging; #[cfg(target_os = "macos")] @@ -256,7 +256,7 @@ fn main() -> eframe::Result<()> { })); let args: Vec = std::env::args().skip(1).collect(); // The restart helper a self-update spawns: it finishes the swap and relaunches, then exits. - if let Some(done) = craft_update::self_update::finish(&craft_update::suite::GRIDCRAFT, &std::env::args_os().skip(1).collect::>()) { + if let Some(done) = craft_update::self_update::finish(&gridcraft_ui_egui::UPDATE_APP, &std::env::args_os().skip(1).collect::>()) { if let Err(e) = done { eprintln!("GridCraft update helper failed: {e}"); std::process::exit(1); @@ -311,15 +311,16 @@ fn main() -> eframe::Result<()> { { log::warn!("sample {s}: {e}"); } - if session.documents().is_empty() { - session.new_workbook(); - } let mut app = SheetApp::new(session, services()); app.watch_desktop_theme(); app.start_update_check(); - app.after_engine(); // Show warnings from the sample. let control_port = control_port; load_prefs(&mut app); + // Stamp the startup workbook's author from the persisted user name. + if app.session.documents().is_empty() { + app.session.new_workbook(); + } + app.after_engine(); // Show warnings from the sample. // Files named on the command line (a double-click on Windows and Linux) open as File › Open // opens them: a password-protected one asks for its password, and errors are shown, not // only logged. Each replaces the untouched blank Book1, as a file opened from Finder does. @@ -340,8 +341,11 @@ fn main() -> eframe::Result<()> { } let window_builder = viewport.clone(); let options = eframe::NativeOptions { viewport, ..Default::default() }; - #[cfg(all(unix, not(target_os = "macos")))] + #[cfg(unix)] let mut options = options; + // Intel Macs: Metal's compiler aborts on wgpu's indirect-validation shaders (see graphics.rs). + #[cfg(target_os = "macos")] + graphics::configure_indirect_validation(&mut options, graphics::skips_indirect_validation(true, cfg!(target_arch = "x86_64"))); // winit has no file drag-and-drop on Wayland (only on X11), so dropping files from the file // manager showed a "no" cursor. Run through XWayland when it's there; GRIDCRAFT_WAYLAND=1 // keeps the native Wayland backend. diff --git a/apps/gridcraft/src/native_menu.rs b/apps/gridcraft/src/native_menu.rs index 522f4fd68..04139297d 100644 --- a/apps/gridcraft/src/native_menu.rs +++ b/apps/gridcraft/src/native_menu.rs @@ -390,11 +390,12 @@ impl NativeMenu { #[cfg(test)] mod tests { - use super::*; + // No `use super::*`: tests/mac_menu_appkit.rs builds this file with cfg(test) but no test + // harness, so the #[test] functions are left out and a module-level import would be unused. #[test] fn file_menu_has_export_as_pdf_dispatching_the_command() { - let tree = tree(); + let tree = super::tree(); let (_, file) = tree.iter().find(|(title, _)| *title == "File").expect("File menu"); let item = file.iter().find(|e| e.0 == "Export as PDF…").expect("Export as PDF… in the File menu"); assert_eq!(item.1, "file.exportPdf"); @@ -403,23 +404,23 @@ mod tests { #[test] fn help_menu_checks_for_updates_through_the_registered_command() { - let tree = tree(); + let tree = super::tree(); let (_, help) = tree.iter().find(|(title, _)| *title == "Help").expect("Help menu"); let item = help.iter().find(|e| e.0 == "Check for Updates…").expect("Check for Updates… in the Help menu"); assert_eq!(item.1, "app.checkForUpdates"); - let mut app = SheetApp::new(gridcraft_engine::Session::new(), Default::default()); + let mut app = super::SheetApp::new(gridcraft_engine::Session::new(), Default::default()); assert!(app.run(item.1, serde_json::json!({})).is_ok()); } #[test] fn theme_choices_use_the_shared_preference_command() { - let mut app = SheetApp::new(gridcraft_engine::Session::new(), Default::default()); + let mut app = super::SheetApp::new(gridcraft_engine::Session::new(), Default::default()); for mode in ["dark", "system", "light"] { - assert!(select_theme(&mut app, &format!("theme:{mode}"))); + assert!(super::select_theme(&mut app, &format!("theme:{mode}"))); assert_eq!(app.ui.theme_mode(), mode); assert!(app.message.is_none()); } - assert!(!select_theme(&mut app, "view.formulaBar")); + assert!(!super::select_theme(&mut app, "view.formulaBar")); assert_eq!(app.ui.theme_mode(), "light"); } } diff --git a/apps/gridcraft/tests/mac_menu_appkit.rs b/apps/gridcraft/tests/mac_menu_appkit.rs index e713f6699..ca1eef509 100644 --- a/apps/gridcraft/tests/mac_menu_appkit.rs +++ b/apps/gridcraft/tests/mac_menu_appkit.rs @@ -7,7 +7,6 @@ // unit tests compile here too without a harness to run them). #[cfg(target_os = "macos")] #[path = "../src/native_menu.rs"] -#[allow(unused_imports, dead_code)] mod native_menu; #[cfg(target_os = "macos")] @@ -20,12 +19,23 @@ fn main() { nsapp.finishLaunching(); let ctx = egui::Context::default(); - let _menu = native_menu::NativeMenu::install(&ctx); + let mut menu = native_menu::NativeMenu::install(&ctx); let bar = nsapp.mainMenu().expect("a main menu"); let tops: Vec = (0..bar.numberOfItems()).filter_map(|i| bar.itemAtIndex(i)?.submenu()).map(|m| m.title().to_string()).collect(); assert_eq!(tops, ["GridCraft", "File", "Edit", "View", "Insert", "Format", "Tools", "Data", "Window", "Help"]); - assert_eq!(nsapp.windowsMenu().map(|m| m.title().to_string()).as_deref(), Some("Window"), "AppKit knows the Window menu"); + let windows_menu = nsapp.windowsMenu().expect("AppKit knows the Window menu"); + assert_eq!(windows_menu.title().to_string(), "Window"); + + // A poll lists the open workbook windows at the end of the Window menu (#716). + let mut session = gridcraft_engine::Session::new(); + session.new_workbook(); + let mut app = gridcraft_ui_egui::SheetApp::new(session, Default::default()); + menu.poll(&mut app, &ctx, egui::ViewportId::ROOT); + let open = app.open_windows(); + assert_eq!(open.len(), 1, "one workbook window"); + let last = windows_menu.itemAtIndex(windows_menu.numberOfItems() - 1).expect("a Window menu item"); + assert_eq!(last.title().to_string(), open[0].label(), "the Window menu lists the open workbook"); println!("mac_menu_appkit: ok"); } diff --git a/assets/app-icon/README.md b/assets/app-icon/README.md index fdf0b0e09..349dcfd5a 100644 --- a/assets/app-icon/README.md +++ b/assets/app-icon/README.md @@ -1,7 +1,7 @@ # GridCraft app icon (PLACEHOLDER) **This icon is temporary.** Every Crafting App's icon is an engraved creature portrait that the -product owner makes (craftrules `standards/icon-design.md`); GridCraft's hasn't been made yet. +product owner makes; GridCraft's hasn't been made yet. Until it is, this placeholder stands in: a simple ledger page with a header row, a row-number column, a selected cell with its fill handle and a double rule under a total, on a full-bleed GridCraft-green tile. It is original geometry written by hand for this repository, deliberately diff --git a/assets/app-icon/gridcraft.svg b/assets/app-icon/gridcraft.svg index fd53f9e1b..38e3e802a 100644 --- a/assets/app-icon/gridcraft.svg +++ b/assets/app-icon/gridcraft.svg @@ -1,7 +1,7 @@ diff --git a/assets/app-icon/hicolor/scalable/apps/ai.storyteller.gridcraft.svg b/assets/app-icon/hicolor/scalable/apps/ai.storyteller.gridcraft.svg index fd53f9e1b..38e3e802a 100644 --- a/assets/app-icon/hicolor/scalable/apps/ai.storyteller.gridcraft.svg +++ b/assets/app-icon/hicolor/scalable/apps/ai.storyteller.gridcraft.svg @@ -1,7 +1,7 @@ diff --git a/clippy.toml b/clippy.toml index de0f11db4..c6fbddafd 100644 --- a/clippy.toml +++ b/clippy.toml @@ -14,3 +14,4 @@ disallowed-methods = [ disallowed-types = [ { path = "egui::ComboBox", reason = "use craft_ui::ComboBox: egui's combo box menu can't grow back while open (egui#5225)" }, ] +cognitive-complexity-threshold = 30 diff --git a/contributors/people.toml b/contributors/people.toml index 5ce80fe59..00f6ce76a 100644 --- a/contributors/people.toml +++ b/contributors/people.toml @@ -1,13 +1,10 @@ -# Contributor names for the About window (craftrules standards/contributors.md). +# Contributor names for the About window. # # Everyone who contributes is credited by GitHub username automatically. Anything more is up to # you: add YOUR OWN entry below, in your own PR (commit it yourself). Never add someone else; a # maintainer may add a person only when that person asked, recording where in consent_source. # -# Easiest: python3 ../../craftrules/scripts/contributors.py --add-me . \ -# [--real-name "Ada Lovelace"] [--display-name "Ada"] [--sync-github-name] -# -# Or copy this block, replacing your-github-username (every field is optional): +# Copy this block, replacing your-github-username (every field is optional): # # [people.your-github-username] # real_name = "Your Name" # shown when the viewer picks "Real name" @@ -103,3 +100,12 @@ real_name = "Helen Kwok" [people.dotCooCoo] real_name = "Robert Lee" sync_github_name = true + +[people.dylanmtaylor] +sync_github_name = true + +[people.Plutonium-fream] +sync_github_name = true + +[people.maxm11] +sync_github_name = true diff --git a/crates/calc/src/eval.rs b/crates/calc/src/eval.rs index 45d81ccca..4972623fc 100644 --- a/crates/calc/src/eval.rs +++ b/crates/calc/src/eval.rs @@ -7,7 +7,7 @@ use std::sync::Arc; use gridcraft_core::{Array, CellError, CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value, compare}; -use gridcraft_formula::{BinOp, Expr, RefKind, Reference, SheetSel, StructItem, StructRef, UnOp}; +use gridcraft_formula::{BinOp, Expr, Reference, SheetSel, StructItem, StructRef, UnOp}; use gridcraft_functions::{Arg, Ctx}; use gridcraft_model::Workbook; diff --git a/crates/calc/src/eval/array.rs b/crates/calc/src/eval/array.rs index b17a8c70f..67662c83c 100644 --- a/crates/calc/src/eval/array.rs +++ b/crates/calc/src/eval/array.rs @@ -1,4 +1,4 @@ -use super::*; +use super::{Array, BinOp, CellError, MAX_CELLS, Value, compare}; /// The aggregates that can fold a reference without building an array. #[derive(Clone, Copy, PartialEq, Eq)] @@ -89,8 +89,14 @@ pub fn zip2(a: &Value, b: &Value, f: impl Fn(&Value, &Value) -> Value) -> Value /// Cell count of a `rows`×`cols` broadcast result, or `None` when it would exceed [`MAX_CELLS`]. /// Each operand passes its own size cap, but their broadcast product can be far larger /// (`SEQUENCE(1048576)*SEQUENCE(1,16384)` is 17 billion cells), so it is checked before allocating. +/// The product is formed in `u64`: on wasm32 `usize` is 32 bits, where a `usize` product above +/// `u32::MAX` would wrap (2^32 cells → 0) and slip past the cap. pub(super) fn broadcast_len(rows: usize, cols: usize) -> Option { - rows.checked_mul(cols).filter(|&n| n as u64 <= MAX_CELLS) + let cells = u64::try_from(rows).ok()?.checked_mul(u64::try_from(cols).ok()?)?; + if cells > MAX_CELLS { + return None; + } + usize::try_from(cells).ok() } fn broadcast_dim(a: usize, b: usize) -> usize { @@ -124,7 +130,7 @@ pub(super) fn zip3(c: &Array, t: &Value, f: &Value, g: impl Fn(&Value, &Value, & /// can't nest, so the whole helper gives `#CALC!`, as in Excel. pub(super) fn single(v: Value) -> Option { match v { - Value::Array(a) if a.rows * a.cols > 1 => None, + Value::Array(a) if a.data.len() > 1 => None, v => Some(v.scalar()), } } @@ -178,3 +184,24 @@ fn binary_scalar(op: BinOp, a: &Value, b: &Value, sys: gridcraft_core::DateSyste BinOp::Range | BinOp::Union | BinOp::Intersect => Value::Error(CellError::Value), } } + +#[cfg(test)] +mod tests { + use super::{MAX_CELLS, broadcast_len}; + + #[test] + fn broadcast_len_caps_at_max_cells() { + assert_eq!(broadcast_len(4_000, 4_000), Some(MAX_CELLS as usize)); + assert_eq!(broadcast_len(4_000, 4_001), None); + assert_eq!(broadcast_len(1, MAX_CELLS as usize), Some(MAX_CELLS as usize)); + } + + /// `65_536 * 65_536` is 2^32. Multiplied as `usize` on wasm32 it wraps to 0, which a + /// `> MAX_CELLS` test would let through; the guard must reject the true product. + #[test] + fn broadcast_len_rejects_a_product_that_wraps_to_zero_on_32_bits() { + assert_eq!(65_536u32.wrapping_mul(65_536), 0); + assert_eq!(broadcast_len(65_536, 65_536), None); + assert_eq!(broadcast_len(1_048_576, 16_384), None); + } +} diff --git a/crates/calc/src/eval/call.rs b/crates/calc/src/eval/call.rs index 9013ca882..08d563d43 100644 --- a/crates/calc/src/eval/call.rs +++ b/crates/calc/src/eval/call.rs @@ -1,40 +1,13 @@ use super::array::{Acc, Agg, zip2, zip3}; use super::format::same_kind; use super::special::{ev_shape, resized_sum_range, resizes_sum_range}; -use super::*; +use super::{Area, Arg, Array, CellError, Ev, Evaluator, Expr, FnCtx, Lambda, Pipe, Value, compare, err}; +use std::sync::Arc; impl<'h> Evaluator<'h> { pub(super) fn call(&mut self, name: &str, args: &[Expr]) -> Ev { match name { - "IF" => { - if args.is_empty() || args.len() > 3 { - return err(CellError::Value); - } - let c = self.arg_val(args, 0).unwrap_or_default(); - if let Value::Array(arr) = &c { - // Array condition: element-wise. - let t = self.arg_val(args, 1).unwrap_or(Value::Bool(true)); - let f = if args.len() > 2 { self.arg_val(args, 2).unwrap_or(Value::Bool(false)) } else { Value::Bool(false) }; - return Ev::V(zip3(arr, &t, &f, |c, t, f| match c.to_bool() { - Ok(true) => t.clone(), - Ok(false) => f.clone(), - Err(e) => Value::Error(e), - })); - } - match c.to_bool() { - Ok(true) => match args.get(1) { - Some(Expr::Missing) => Ev::V(Value::Number(0.0)), - Some(e) => self.eval(e), - None => Ev::V(Value::Bool(true)), - }, - Ok(false) => match args.get(2) { - Some(Expr::Missing) => Ev::V(Value::Number(0.0)), - Some(e) => self.eval(e), - None => Ev::V(Value::Bool(false)), - }, - Err(e) => err(e), - } - } + "IF" => self.if_call(args), "IFS" => { if args.len() < 2 || !args.len().is_multiple_of(2) { return err(CellError::Value); @@ -50,101 +23,10 @@ impl<'h> Evaluator<'h> { } err(CellError::NA) } - "IFERROR" | "IFNA" => { - if args.len() != 2 { - return err(CellError::Value); - } - let v = self.arg_ev(args, 0).unwrap_or(Ev::V(Value::Empty)); - let na_only = name == "IFNA"; - let catch = |e: CellError| if na_only { e == CellError::NA } else { true }; - match &v { - Ev::V(Value::Error(e)) if catch(*e) => self.eval(&args[1]), - Ev::V(Value::Array(a)) if a.data.iter().any(|x| x.as_error().is_some_and(catch)) => { - let alt = self.arg_val(args, 1).unwrap_or_default(); - let data = a.data.iter().map(|x| if x.as_error().is_some_and(catch) { alt.scalar() } else { x.clone() }).collect(); - Ev::V(Array::new(a.rows, a.cols, data).map(Value::from).unwrap_or(Value::Error(CellError::Value))) - } - Ev::R(_) => { - let val = self.deref(v.clone()); - match val { - Value::Error(e) if catch(e) => self.eval(&args[1]), - Value::Array(ref a) if a.data.iter().any(|x| x.as_error().is_some_and(catch)) => { - let alt = self.arg_val(args, 1).unwrap_or_default(); - let data = a.data.iter().map(|x| if x.as_error().is_some_and(catch) { alt.scalar() } else { x.clone() }).collect(); - Ev::V(Array::new(a.rows, a.cols, data).map(Value::from).unwrap_or(Value::Error(CellError::Value))) - } - _ => v, - } - } - _ => v, - } - } - "CHOOSE" => { - if args.len() < 2 { - return err(CellError::Value); - } - let index = self.value(&args[0]); - if let Value::Array(_) = index { - // An array of indexes picks element by element: `CHOOSE({1,2},A1:A3,B1:B3)` - // is a two-column array. The library CHOOSE lifts over its arguments. - let Some(spec) = gridcraft_functions::lookup(name) else { return err(CellError::Name) }; - let mut evaluated = vec![Arg { value: index, from_ref: false }]; - for a in &args[1..] { - let value = self.value(a); - evaluated.push(Arg { value, from_ref: false }); - } - let mut ctx = FnCtx { ev: self, criteria: None, lambda: None }; - return gridcraft_functions::call(spec, &evaluated, &mut ctx).pipe(Ev::V); - } - let i = match index.to_number() { - Ok(n) => n.trunc() as i64, - Err(e) => return err(e), - }; - if i < 1 || i as usize >= args.len() { - return err(CellError::Value); - } - self.eval(&args[i as usize]) - } - "SWITCH" => { - if args.len() < 3 { - return err(CellError::Value); - } - let v = self.arg_val(args, 0).unwrap_or_default().scalar(); - if let Value::Error(e) = v { - return err(e); - } - let rest = &args[1..]; - let mut i = 0; - while i + 1 < rest.len() { - let cand = self.value(&rest[i]).scalar(); - if !cand.is_error() && compare(&v, &cand) == std::cmp::Ordering::Equal && same_kind(&v, &cand) { - return self.eval(&rest[i + 1]); - } - i += 2; - } - if rest.len() % 2 == 1 { rest.last().map_or(err(CellError::NA), |d| self.eval(d)) } else { err(CellError::NA) } - } - "ROW" | "COLUMN" => { - let is_row = name == "ROW"; - if args.is_empty() || matches!(args.first(), Some(Expr::Missing)) { - return Ev::V(Value::number(if is_row { self.at.row + 1 } else { self.at.col + 1 } as f64)); - } - match self.eval(&args[0]) { - Ev::R(a) => { - let Some(a) = a.first() else { return err(CellError::Ref) }; - let r = self.trim_lines(a); - if is_row { - let rows: Vec = (r.start.row..=r.end.row).map(|x| Value::number(x as f64 + 1.0)).collect(); - if rows.len() == 1 { Ev::V(rows.into_iter().next().unwrap_or_default()) } else { Ev::V(Array::column(rows).into()) } - } else { - let cols: Vec = (r.start.col..=r.end.col).map(|x| Value::number(x as f64 + 1.0)).collect(); - if cols.len() == 1 { Ev::V(cols.into_iter().next().unwrap_or_default()) } else { Ev::V(Array::row(cols).into()) } - } - } - Ev::V(Value::Error(e)) => err(e), - _ => err(CellError::Value), - } - } + "IFERROR" | "IFNA" => self.if_error_call(name, args), + "CHOOSE" => self.choose_call(args), + "SWITCH" => self.switch_call(args), + "ROW" | "COLUMN" => self.row_column_call(name, args), "ROWS" | "COLUMNS" => { if args.len() != 1 { return err(CellError::Value); @@ -180,28 +62,7 @@ impl<'h> Evaluator<'h> { } } } - "SHEET" | "SHEETS" => { - let wb = self.host.workbook(); - if args.is_empty() { - return Ev::V(Value::number(if name == "SHEET" { self.sheet + 1 } else { wb.sheets.len() } as f64)); - } - match self.eval(&args[0]) { - Ev::R(a) => { - if name == "SHEET" { - a.first().map_or(err(CellError::Ref), |a| Ev::V(Value::number(a.sheet as f64 + 1.0))) - } else { - let mut s: Vec = a.iter().map(|x| x.sheet).collect(); - s.dedup(); - Ev::V(Value::number(s.len() as f64)) - } - } - Ev::V(Value::Text(t)) if name == "SHEET" => { - let wb = self.host.workbook(); - wb.sheet_index(&t).map_or(err(CellError::NA), |i| Ev::V(Value::number(i as f64 + 1.0))) - } - _ => err(CellError::NA), - } - } + "SHEET" | "SHEETS" => self.sheet_call(name, args), "OFFSET" => self.offset(args), "INDIRECT" => self.indirect(args), "INDEX" => self.index(args), @@ -273,6 +134,156 @@ impl<'h> Evaluator<'h> { } } + fn if_call(&mut self, args: &[Expr]) -> Ev { + if args.is_empty() || args.len() > 3 { + return err(CellError::Value); + } + let condition = self.arg_val(args, 0).unwrap_or_default(); + if let Value::Array(array) = &condition { + let when_true = self.arg_val(args, 1).unwrap_or(Value::Bool(true)); + let when_false = if args.len() > 2 { self.arg_val(args, 2).unwrap_or(Value::Bool(false)) } else { Value::Bool(false) }; + return Ev::V(zip3(array, &when_true, &when_false, |condition, when_true, when_false| match condition.to_bool() { + Ok(true) => when_true.clone(), + Ok(false) => when_false.clone(), + Err(error) => Value::Error(error), + })); + } + match condition.to_bool() { + Ok(true) => match args.get(1) { + Some(Expr::Missing) => Ev::V(Value::Number(0.0)), + Some(expression) => self.eval(expression), + None => Ev::V(Value::Bool(true)), + }, + Ok(false) => match args.get(2) { + Some(Expr::Missing) => Ev::V(Value::Number(0.0)), + Some(expression) => self.eval(expression), + None => Ev::V(Value::Bool(false)), + }, + Err(error) => err(error), + } + } + + fn if_error_call(&mut self, name: &str, args: &[Expr]) -> Ev { + if args.len() != 2 { + return err(CellError::Value); + } + let value = self.arg_ev(args, 0).unwrap_or(Ev::V(Value::Empty)); + let na_only = name == "IFNA"; + let catches = |error: CellError| !na_only || error == CellError::NA; + match &value { + Ev::V(Value::Error(error)) if catches(*error) => self.eval(&args[1]), + Ev::V(Value::Array(array)) if array.data.iter().any(|item| item.as_error().is_some_and(catches)) => { + let alternate = self.arg_val(args, 1).unwrap_or_default(); + let data = + array.data.iter().map(|item| if item.as_error().is_some_and(catches) { alternate.scalar() } else { item.clone() }).collect(); + Ev::V(Array::new(array.rows, array.cols, data).map(Value::from).unwrap_or(Value::Error(CellError::Value))) + } + Ev::R(_) => self.if_error_reference(value.clone(), args, catches), + _ => value, + } + } + + fn if_error_reference(&mut self, value: Ev, args: &[Expr], catches: impl Fn(CellError) -> bool + Copy) -> Ev { + match self.deref(value.clone()) { + Value::Error(error) if catches(error) => self.eval(&args[1]), + Value::Array(ref array) if array.data.iter().any(|item| item.as_error().is_some_and(catches)) => { + let alternate = self.arg_val(args, 1).unwrap_or_default(); + let data = + array.data.iter().map(|item| if item.as_error().is_some_and(catches) { alternate.scalar() } else { item.clone() }).collect(); + Ev::V(Array::new(array.rows, array.cols, data).map(Value::from).unwrap_or(Value::Error(CellError::Value))) + } + _ => value, + } + } + + fn choose_call(&mut self, args: &[Expr]) -> Ev { + if args.len() < 2 { + return err(CellError::Value); + } + let index = self.value(&args[0]); + if let Value::Array(_) = index { + let Some(spec) = gridcraft_functions::lookup("CHOOSE") else { return err(CellError::Name) }; + let mut evaluated = vec![Arg { value: index, from_ref: false }]; + for argument in &args[1..] { + evaluated.push(Arg { value: self.value(argument), from_ref: false }); + } + let mut context = FnCtx { ev: self, criteria: None, lambda: None }; + return gridcraft_functions::call(spec, &evaluated, &mut context).pipe(Ev::V); + } + let index = match index.to_number() { + Ok(number) => number.trunc() as i64, + Err(error) => return err(error), + }; + if index < 1 || index as usize >= args.len() { + return err(CellError::Value); + } + self.eval(&args[index as usize]) + } + + fn switch_call(&mut self, args: &[Expr]) -> Ev { + if args.len() < 3 { + return err(CellError::Value); + } + let value = self.arg_val(args, 0).unwrap_or_default().scalar(); + if let Value::Error(error) = value { + return err(error); + } + let choices = &args[1..]; + for pair in choices.as_chunks::<2>().0 { + let (candidate, result) = (&pair[0], &pair[1]); + let candidate = self.value(candidate).scalar(); + if !candidate.is_error() && compare(&value, &candidate) == std::cmp::Ordering::Equal && same_kind(&value, &candidate) { + return self.eval(result); + } + } + if let Some(default) = choices.get(choices.len().saturating_sub(1)).filter(|_| choices.len() % 2 == 1) { + self.eval(default) + } else { + err(CellError::NA) + } + } + + fn row_column_call(&mut self, name: &str, args: &[Expr]) -> Ev { + let is_row = name == "ROW"; + if args.is_empty() || matches!(args.first(), Some(Expr::Missing)) { + return Ev::V(Value::number(if is_row { self.at.row + 1 } else { self.at.col + 1 } as f64)); + } + let areas = match self.eval(&args[0]) { + Ev::R(areas) => areas, + Ev::V(Value::Error(error)) => return err(error), + _ => return err(CellError::Value), + }; + let Some(area) = areas.first() else { return err(CellError::Ref) }; + let range = self.trim_lines(area); + if is_row { + let values: Vec = (range.start.row..=range.end.row).map(|row| Value::number(row as f64 + 1.0)).collect(); + if values.len() == 1 { Ev::V(values.into_iter().next().unwrap_or_default()) } else { Ev::V(Array::column(values).into()) } + } else { + let values: Vec = (range.start.col..=range.end.col).map(|col| Value::number(col as f64 + 1.0)).collect(); + if values.len() == 1 { Ev::V(values.into_iter().next().unwrap_or_default()) } else { Ev::V(Array::row(values).into()) } + } + } + + fn sheet_call(&mut self, name: &str, args: &[Expr]) -> Ev { + let is_sheet = name == "SHEET"; + if args.is_empty() { + let count = if is_sheet { self.sheet + 1 } else { self.host.workbook().sheets.len() }; + return Ev::V(Value::number(count as f64)); + } + match self.eval(&args[0]) { + Ev::R(areas) if is_sheet => areas.first().map_or(err(CellError::Ref), |area| Ev::V(Value::number(area.sheet as f64 + 1.0))), + Ev::R(areas) => { + let mut sheets: Vec = areas.iter().map(|area| area.sheet).collect(); + sheets.dedup(); + Ev::V(Value::number(sheets.len() as f64)) + } + Ev::V(Value::Text(name)) if is_sheet => { + self.host.workbook().sheet_index(&name).map_or(err(CellError::NA), |index| Ev::V(Value::number(index as f64 + 1.0))) + } + _ => err(CellError::NA), + } + } + pub(super) fn builtin(&mut self, name: &str, args: &[Expr]) -> Ev { let Some(spec) = gridcraft_functions::lookup(name) else { // A defined name holding a LAMBDA, or a LET-bound lambda. diff --git a/crates/calc/src/eval/format.rs b/crates/calc/src/eval/format.rs index a4c52b492..a288b410e 100644 --- a/crates/calc/src/eval/format.rs +++ b/crates/calc/src/eval/format.rs @@ -1,4 +1,5 @@ -use super::*; +use super::{CellError, Value}; +use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS}; /// CELL("format") code for a number format. pub(super) fn cell_format_code(code: &str) -> String { diff --git a/crates/calc/src/eval/lambda.rs b/crates/calc/src/eval/lambda.rs index 45f99b488..dfb30029a 100644 --- a/crates/calc/src/eval/lambda.rs +++ b/crates/calc/src/eval/lambda.rs @@ -1,5 +1,7 @@ +use super::MAX_LAMBDA_CALLS; use super::array::{broadcast_len, single}; -use super::*; +use super::{Array, CellError, Ev, Evaluator, Expr, Lambda, MAX_CELLS, Value, err}; +use std::sync::Arc; impl<'h> Evaluator<'h> { pub fn apply_lambda(&mut self, l: &Lambda, args: &[Expr]) -> Ev { diff --git a/crates/calc/src/eval/reference_fns.rs b/crates/calc/src/eval/reference_fns.rs index a21797c23..4f5154c2d 100644 --- a/crates/calc/src/eval/reference_fns.rs +++ b/crates/calc/src/eval/reference_fns.rs @@ -1,5 +1,7 @@ use super::format::{cell_format_code, r1c1_to_a1}; -use super::*; +use super::{Area, Arg, Array, CellError, Ev, Evaluator, Expr, FnCtx, MAX_CELLS, Value, err}; +use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS, RangeRef}; +use std::sync::Arc; impl<'h> Evaluator<'h> { /// Whole rows/columns trimmed for ROW()/COLUMN() arrays. diff --git a/crates/calc/src/eval/special.rs b/crates/calc/src/eval/special.rs index f68badbfc..0151ea2f9 100644 --- a/crates/calc/src/eval/special.rs +++ b/crates/calc/src/eval/special.rs @@ -1,4 +1,5 @@ -use super::*; +use super::{Area, BinOp, Ev, Expr, Reference, SheetSel, Value, Workbook}; +use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS, RangeRef}; /// SUMIF and AVERAGEIF with a sum range: Excel sizes the sum range like the criteria range. pub(super) fn resizes_sum_range(name: &str, args: usize) -> bool { @@ -162,6 +163,3 @@ pub const SPECIAL_FUNCTIONS: &[&str] = &[ pub fn is_known_function(n: &str) -> bool { is_special(&n.to_ascii_uppercase()) || gridcraft_functions::lookup(n).is_some() } - -#[allow(dead_code)] -fn unused(_: RefKind) {} diff --git a/crates/calc/src/recalc.rs b/crates/calc/src/recalc.rs index b7f403131..63ac533de 100644 --- a/crates/calc/src/recalc.rs +++ b/crates/calc/src/recalc.rs @@ -10,14 +10,13 @@ //! and the formulas that read one are evaluated again until the loop settles //! (`PassHost::iterate_loops`). -use std::collections::{BTreeSet, VecDeque}; -use std::hash::{BuildHasherDefault, Hasher}; +use std::collections::VecDeque; use gridcraft_core::{Array, CellError, CellRef, RangeRef, Value}; use gridcraft_formula::Expr; use gridcraft_model::{CalcMode, Workbook}; -use crate::eval::{Area, Evaluator, Host, precedents, stack_position}; +use crate::eval::{Evaluator, Host, precedents, stack_position}; mod graph; mod pass_host; @@ -136,6 +135,10 @@ impl Calc { if cell.value == Value::Error(CellError::Circ) { circ.push((si, c)); } + } else if cell.py.is_some() { + // A Python cell lays out its kept result, so it is calculated like a formula + // that reads nothing: its spill is blocked and unblocked like a formula's. + self.graph.insert((si, c), Node { areas: Vec::new(), dynamic: false }); } } } @@ -207,11 +210,29 @@ impl Calc { let mut cleared: Vec<(usize, RangeRef)> = Vec::new(); for &k in changed { let formula = wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.formula.clone()); + let py = wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.py.as_ref()); + let python = py.is_some(); + // A Python cell whose result (if any) is not an array spills nothing. + let spills = py.and_then(|p| p.result_value()).is_some_and(|v| matches!(v, Value::Array(_))); match formula.and_then(|f| f.expr()) { Some(e) => { let (areas, dynamic) = precedents(wb, k.0, &e); self.graph.insert(k, Node { areas, dynamic }); } + // See `rebuild`: its spill is laid out again from its result when it is calculated. + None if python => { + self.graph.insert(k, Node { areas: Vec::new(), dynamic: false }); + // New code has no result yet: the spill the cell had (its old result's, or a + // formula's) goes, and what read it is recalculated, as for a formula + // replaced by a constant. + if !spills + && let Some(sh) = wb.sheet_mut(k.0) + && let Some(r) = sh.spill_ranges.remove(&k.1) + { + sh.clear_spill(r); + cleared.push((k.0, r)); + } + } None => { self.graph.remove(k); // A formula replaced by a constant (or cleared) takes its spilled values with it. @@ -339,11 +360,15 @@ impl Calc { exprs.insert(*k, e); } } + let python: HashMap = dirty + .iter() + .filter_map(|k| wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.py.as_ref()).and_then(|p| p.result_value()).map(|v| (*k, v))) + .collect(); let now = self.now(); let mut rng = self.rng; let tp = prof_now(); let (results, spills, cycles) = { - let mut host = PassHost::new(wb, &exprs, &mut rng, now); + let mut host = PassHost::new(wb, &exprs, &mut rng, now).with_python(python); host.iterate = wb.calc.iterative; for k in &dirty { host.calculate(*k); @@ -368,105 +393,13 @@ impl Calc { self.circular.retain(|k| !evaluated.contains(k)); } self.circular.extend(cycles); - // Write back. - let mut spill_changes: Vec = Vec::new(); - let mut freed: Vec<(usize, RangeRef)> = Vec::new(); - for (k, v) in results { - let Some(sheet) = wb.sheet_mut(k.0) else { continue }; - if let Some(cell) = sheet.cells.get_mut(k.1) - && cell.value != v - { - cell.value = v; - } - } - for k in &dirty { - let new = spills.get(k); - let Some(sheet) = wb.sheet_mut(k.0) else { continue }; - let old = sheet.spill_ranges.get(&k.1).copied(); - if old.is_none() && new.is_none() { - continue; - } - // An array that spills exactly as before changes nothing its dependents read, so - // they aren't queued again. Otherwise two arrays that read each other's spill areas - // queue each other in every round, up to the round limit. - if let (Some(r), Some(arr)) = (old, new) - && spills_as_before(sheet, k.1, r, arr) - { - continue; - } - if let Some(r) = old { - for c in r.iter() { - sheet.spill.remove(&c); - } - sheet.spill_ranges.remove(&k.1); - } - let mut changed_range = old; - if let Some(arr) = new { - let end = CellRef::new(k.1.row.saturating_add(arr.rows as u32 - 1), k.1.col.saturating_add(arr.cols as u32 - 1)); - let range = RangeRef::new(k.1, end); - let blocked = end.row >= gridcraft_core::MAX_ROWS - || end.col >= gridcraft_core::MAX_COLS - || range.iter().any(|c| c != k.1 && (sheet.cells.has(c) || sheet.spill.contains_key(&c))) - || sheet.merges.iter().any(|m| m.intersects(&range)) - || sheet.tables.iter().any(|t| t.range.contains(k.1)); - if blocked { - if let Some(cell) = sheet.cells.get_mut(k.1) { - cell.value = Value::Error(CellError::Spill); - } - // Not blocked while the pass ran (it would have been #SPILL! there), so - // blocked by another array of this pass: the formulas that read this one, - // at its anchor or over the area it would have spilled into, read #SPILL! - // or empty cells now. - let mut deps = Vec::new(); - self.graph.dependents_of_range(k.0, range, &mut deps); - spill_changes.extend(deps.into_iter().filter(|d| d != k)); - } else { - for (i, c) in range.iter().enumerate() { - if c != k.1 { - sheet.spill.insert(c, arr.data.get(i).cloned().unwrap_or_default()); - } - } - sheet.spill_ranges.insert(k.1, range); - changed_range = Some(changed_range.map_or(range, |o| o.union(&range))); - } - } - if let Some(r) = changed_range { - let mut deps = Vec::new(); - self.graph.dependents_of_range(k.0, r, &mut deps); - // Not the anchor itself (that would read its own array), unless its old array - // is gone or partly freed: its new formula may have read the old values, still - // laid out while it was evaluated. - let freed = match (old, sheet.spill_ranges.get(&k.1)) { - (Some(o), Some(n)) => !(n.contains(o.start) && n.contains(o.end)), - (_, None) => true, - (None, Some(_)) => false, - }; - spill_changes.extend(deps.into_iter().filter(|d| d != k || freed)); - } - if let Some(o) = old - && !sheet.spill_ranges.get(&k.1).is_some_and(|n| n.contains(o.start) && n.contains(o.end)) - { - freed.push((k.0, o)); - } - } + let mut spill_changes = self.write_back(wb, &dirty, results, &spills); // Formulas with references only known while evaluating (INDIRECT, OFFSET) may have read // a spill area before it was laid out: they are evaluated once more. if dynamic_again && !spills.is_empty() { dynamic_again = false; spill_changes.extend(self.graph.dynamic.iter().copied()); } - // An area a formula no longer spills over may unblock another formula's array. - for (si, o) in freed { - for &(s, c) in self.graph.nodes.keys() { - if s == si - && c.row <= o.end.row - && c.col <= o.end.col - && wb.sheet(s).is_some_and(|sh| sh.value(c) == Value::Error(CellError::Spill)) - { - spill_changes.push((s, c)); - } - } - } if full { // Everything was already evaluated once; spill dependents need another pass. } @@ -488,6 +421,105 @@ impl Calc { self.circular.retain(|k| self.graph.nodes.contains_key(k)); } } + + fn write_back( + &mut self, + wb: &mut Workbook, + dirty: &[Key], + results: HashMap, + spills: &HashMap>, + ) -> Vec { + let mut spill_changes = Vec::new(); + let mut freed = Vec::new(); + for (key, value) in results { + let Some(sheet) = wb.sheet_mut(key.0) else { continue }; + if let Some(cell) = sheet.cells.get_mut(key.1) + && cell.value != value + { + cell.value = value; + } + } + for key in dirty { + let new = spills.get(key); + let Some(sheet) = wb.sheet_mut(key.0) else { continue }; + let old = sheet.spill_ranges.get(&key.1).copied(); + if old.is_none() && new.is_none() { + continue; + } + // An unchanged spill does not alter any values its dependents read. + if let (Some(range), Some(array)) = (old, new) + && spills_as_before(sheet, key.1, range, array) + { + continue; + } + if let Some(range) = old { + for cell in range.iter() { + sheet.spill.remove(&cell); + } + sheet.spill_ranges.remove(&key.1); + } + let mut changed_range = old; + if let Some(array) = new { + let end = CellRef::new(key.1.row.saturating_add(array.rows as u32 - 1), key.1.col.saturating_add(array.cols as u32 - 1)); + let range = RangeRef::new(key.1, end); + let blocked = spill_is_blocked(sheet, key.1, range, end); + if blocked { + if let Some(cell) = sheet.cells.get_mut(key.1) { + cell.value = Value::Error(CellError::Spill); + } + let mut blocked_dependents = Vec::new(); + self.graph.dependents_of_range(key.0, range, &mut blocked_dependents); + spill_changes.extend(blocked_dependents.into_iter().filter(|dependent| dependent != key)); + } else { + fill_spill(sheet, key.1, range, array); + changed_range = Some(changed_range.map_or(range, |previous| previous.union(&range))); + } + } + if let Some(range) = changed_range { + let mut dependents = Vec::new(); + self.graph.dependents_of_range(key.0, range, &mut dependents); + let freed_old_range = match (old, sheet.spill_ranges.get(&key.1)) { + (Some(previous), Some(current)) => !(current.contains(previous.start) && current.contains(previous.end)), + (_, None) => true, + (None, Some(_)) => false, + }; + spill_changes.extend(dependents.into_iter().filter(|dependent| dependent != key || freed_old_range)); + } + if let Some(previous) = old + && !sheet.spill_ranges.get(&key.1).is_some_and(|current| current.contains(previous.start) && current.contains(previous.end)) + { + freed.push((key.0, previous)); + } + } + for (sheet_index, range) in freed { + for &(sheet, cell) in self.graph.nodes.keys() { + let was_blocked_cell = cell.row <= range.end.row + && cell.col <= range.end.col + && wb.sheet(sheet).is_some_and(|sheet| sheet.value(cell) == Value::Error(CellError::Spill)); + if sheet == sheet_index && was_blocked_cell { + spill_changes.push((sheet, cell)); + } + } + } + spill_changes + } +} + +fn spill_is_blocked(sheet: &gridcraft_model::Sheet, anchor: CellRef, range: RangeRef, end: CellRef) -> bool { + end.row >= gridcraft_core::MAX_ROWS + || end.col >= gridcraft_core::MAX_COLS + || range.iter().any(|cell| cell != anchor && (sheet.cells.has(cell) || sheet.spill.contains_key(&cell))) + || sheet.merges.iter().any(|merge| merge.intersects(&range)) + || sheet.tables.iter().any(|table| table.range.contains(anchor)) +} + +fn fill_spill(sheet: &mut gridcraft_model::Sheet, anchor: CellRef, range: RangeRef, array: &Array) { + for (index, cell) in range.iter().enumerate() { + if cell != anchor { + sheet.spill.insert(cell, array.data.get(index).cloned().unwrap_or_default()); + } + } + sheet.spill_ranges.insert(anchor, range); } /// Current date-time as a serial (1900 system), from the system clock (browser clock on wasm). @@ -513,21 +545,13 @@ pub fn evaluate_expr(wb: &Workbook, sheet: usize, at: CellRef, expr: &Expr) -> V ev.value(expr) } -#[allow(clippy::disallowed_methods)] // the clock is read only off wasm fn prof_now() -> f64 { - #[cfg(not(target_arch = "wasm32"))] - { - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs_f64() * 1000.0).unwrap_or(0.0) - } - #[cfg(target_arch = "wasm32")] - { - 0.0 - } + gridcraft_core::date::unix_now_ms().unwrap_or(0.0) } #[cfg(test)] pub(crate) mod tests { - use std::time::{Duration, Instant}; + use std::time::Duration; use gridcraft_core::{CellRef, RangeRef, Value}; use gridcraft_model::{Cell, Formula, Workbook}; @@ -549,6 +573,48 @@ pub(crate) mod tests { v } + /// A 1,000 x 1,000 Python result laid out from A1 in a pass, with the same values already + /// spilled on the sheet from the last calculation, as a full recalculation finds them. + fn big_python_spill() -> (Workbook, HashMap) { + let n = 1_000usize; + let data: Vec = (0..n * n).map(|i| Value::Number(i as f64)).collect(); + let array = gridcraft_core::Array::new(n, n, data.clone()).unwrap(); + let mut wb = Workbook::new(); + let sh = wb.sheet_mut(0).unwrap(); + let anchor = CellRef::new(0, 0); + let py = gridcraft_model::PyCode::new("big"); + sh.set_cell(anchor, Cell { value: Value::Number(0.0), py: Some(std::sync::Arc::new(py)), ..Cell::default() }); + let area = RangeRef::new(anchor, CellRef::new(n as u32 - 1, n as u32 - 1)); + sh.spill_ranges.insert(anchor, area); + for (i, c) in area.iter().enumerate().skip(1) { + sh.spill.insert(c, data[i].clone()); + } + let mut python = HashMap::default(); + python.insert((0, anchor), Value::Array(std::sync::Arc::new(array))); + (wb, python) + } + + #[test] + fn a_read_beside_a_large_spill_walks_none_of_it() { + let (wb, python) = big_python_spill(); + let exprs = HashMap::default(); + let mut rng = 1u64; + let mut host = PassHost::new(&wb, &exprs, &mut rng, 0.0).with_python(python); + assert_eq!(host.cell_value(0, CellRef::new(0, 0)), Value::Number(0.0)); + // Beside the spill (column ALM, the 1,001st), over the rows it covers. + let beside = RangeRef::parse("ALM1:ALM1000").unwrap(); + host.spill_visits = 0; + assert!(host.range_values(0, beside).iter().all(Value::is_empty)); + assert!(host.visit_range(0, beside, &mut |_| true), "a disjoint read takes the fast path"); + assert_eq!(host.spill_visits, 0, "nothing of the spill is walked"); + // A read inside it still sees the spilled values, walking only the cells it reads. + let inside = RangeRef::parse("B2:C3").unwrap(); + host.spill_visits = 0; + assert_eq!(host.range_values(0, inside), [1_001.0, 1_002.0, 2_001.0, 2_002.0].map(Value::Number).to_vec()); + assert!(host.spill_visits <= 8, "{}", host.spill_visits); + assert!(!host.visit_range(0, inside, &mut |_| true), "an overlapping read leaves the block to range_values"); + } + #[test] fn intervals_match_a_linear_scan() { let mut iv: Intervals = Intervals::default(); @@ -640,11 +706,10 @@ pub(crate) mod tests { } /// Opening (full recalc), changing every value of a column, and re-entering every formula. - #[allow(clippy::disallowed_methods)] // a native-only timing test: the clock is read off wasm fn open_and_bulk_edit(rows: u32) -> Duration { let mut wb = sum_rows(rows); let mut calc = Calc::new(); - let t = Instant::now(); + let started = gridcraft_core::date::unix_now_ms().unwrap_or(0.0); calc.recalc_all(&mut wb); let values: Vec = (0..rows).map(|r| (0, CellRef::new(r, 4))).collect(); if let Some(sh) = wb.sheet_mut(0) { @@ -655,7 +720,8 @@ pub(crate) mod tests { calc.cells_changed(&mut wb, &values); let formulas: Vec = (0..rows).map(|r| (0, CellRef::new(r, 10))).collect(); calc.cells_changed(&mut wb, &formulas); - let elapsed = t.elapsed(); + let ended = gridcraft_core::date::unix_now_ms().unwrap_or(started); + let elapsed = Duration::from_secs_f64((ended - started).max(0.0) / 1000.0); let last = wb.sheet(0).map(|s| s.value(CellRef::new(rows - 1, 10))); let r = rows - 1; assert_eq!(last, Some(Value::Number(f64::from(1 + (5..10).map(|c| r % 97 + c).sum::())))); @@ -670,7 +736,6 @@ pub(crate) mod tests { #[test] #[ignore = "timing-sensitive; run with --ignored in release"] - #[allow(clippy::disallowed_methods)] fn recalc_with_range_formulas_scales_linearly() { // Each row's SUM reads a range in the same columns. Recalculating on open used to walk // every row per range read (the used range), and finding a cell's dependents scanned diff --git a/crates/calc/src/recalc/graph.rs b/crates/calc/src/recalc/graph.rs index aee4b46d5..ab034f8cc 100644 --- a/crates/calc/src/recalc/graph.rs +++ b/crates/calc/src/recalc/graph.rs @@ -1,4 +1,8 @@ -use super::*; +use super::RangeRef; +use crate::eval::Area; +use gridcraft_core::CellRef; +use std::collections::BTreeSet; +use std::hash::{BuildHasherDefault, Hasher}; /// A fast, non-cryptographic hasher for cell keys (FxHash-style multiply-rotate). #[derive(Default, Clone, Copy)] @@ -179,6 +183,9 @@ pub struct Graph { unsettled_wide: HashSet, } +/// A formula's precedents split by shape: single cells, column intervals and wide ranges. +type Entries = (Vec, Vec<(usize, u32, u32, u32)>, Vec<(usize, RangeRef)>); + impl Graph { pub fn len(&self) -> usize { self.nodes.len() @@ -189,7 +196,7 @@ impl Graph { /// Index entries of a formula's precedents, without duplicates: single cells, column /// intervals and wide ranges. - fn entries(areas: &[Area]) -> (Vec, Vec<(usize, u32, u32, u32)>, Vec<(usize, RangeRef)>) { + fn entries(areas: &[Area]) -> Entries { let (mut cells, mut cols, mut wide) = (Vec::new(), Vec::new(), Vec::new()); for a in areas { if a.range.is_single() { diff --git a/crates/calc/src/recalc/pass_host.rs b/crates/calc/src/recalc/pass_host.rs index 00e92e82b..5909369fd 100644 --- a/crates/calc/src/recalc/pass_host.rs +++ b/crates/calc/src/recalc/pass_host.rs @@ -1,5 +1,7 @@ use super::graph::{HashMap, HashSet, Key}; -use super::*; +use super::{Evaluator, Host, Workbook, stack_position}; +use gridcraft_core::{Array, CellError, CellRef, RangeRef, Value}; +use gridcraft_formula::Expr; /// Formulas a formula may calculate on demand one inside the other (each needing one another's /// value before it is done). Each level takes stack, so a chain deeper than this is suspended and @@ -43,6 +45,10 @@ pub(super) struct PassHost<'a> { /// Suspensions so far (tests check that a wide read at the limit needs one, not one per cell). #[cfg(test)] pub(super) suspensions: usize, + /// Spilled values looked at by range reads (tests check that a read outside a large spill + /// looks at none of it). + #[cfg(test)] + pub(super) spill_visits: usize, /// The formulas in `cycles`. cycle_seen: HashSet, /// Iterative calculation: a circular reference reads the cell's previous value instead of @@ -62,6 +68,10 @@ pub(super) struct PassHost<'a> { stack_top: usize, /// Whether an evaluation ran past the stack budget since the innermost formula started. out_of_stack: bool, + /// The kept results of the Python cells in the pass: a Python cell isn't evaluated, its + /// result is laid out (an array spills from the cell, as a formula's does). Each one is in + /// `pending` until it is laid out, so a formula that reads it first gets its new value. + python: HashMap, } impl<'a> PassHost<'a> { @@ -85,6 +95,8 @@ impl<'a> PassHost<'a> { also: Vec::new(), #[cfg(test)] suspensions: 0, + #[cfg(test)] + spill_visits: 0, cycle_seen: HashSet::default(), iterate: false, prev: HashMap::default(), @@ -94,8 +106,17 @@ impl<'a> PassHost<'a> { taint: false, stack_top: 0, out_of_stack: false, + python: HashMap::default(), } } + + /// Adds the dirty Python cells of the pass and their kept results: like the formulas, they + /// are pending until laid out, so every read path calculates them before reading them. + pub(super) fn with_python(mut self, python: HashMap) -> Self { + self.pending.extend(python.keys().copied()); + self.python = python; + self + } } /// Whether an array result at `k` can't spill: it is in a table, or its area runs off the sheet, @@ -109,7 +130,10 @@ fn spill_blocked(host: &PassHost<'_>, (sheet, anchor): Key, a: &Array) -> bool { let own = sh.spill_ranges.get(&anchor); let spilled_over = |c: CellRef| { !own.is_some_and(|o| o.contains(c)) - && sh.spill_ranges.iter().any(|(o, r)| *o != anchor && r.contains(c) && !host.exprs.contains_key(&(sheet, *o))) + && sh.spill_ranges.iter().any(|(o, r)| { + // A formula or Python cell recalculated in this pass lays its array out again. + *o != anchor && r.contains(c) && !host.exprs.contains_key(&(sheet, *o)) && !host.python.contains_key(&(sheet, *o)) + }) }; end.row >= gridcraft_core::MAX_ROWS || end.col >= gridcraft_core::MAX_COLS @@ -133,6 +157,42 @@ pub(super) fn spills_as_before(sheet: &gridcraft_model::Sheet, anchor: CellRef, .all(|(i, c)| c == anchor || (!sheet.cells.has(c) && sheet.spill.get(&c) == Some(arr.data.get(i).unwrap_or(&Value::Empty)))) } +/// The area an array anchored at `anchor` covers. +fn array_area(anchor: CellRef, a: &Array) -> RangeRef { + let end = + CellRef::new(anchor.row.saturating_add((a.rows as u32).saturating_sub(1)), anchor.col.saturating_add((a.cols as u32).saturating_sub(1))); + RangeRef::new(anchor, end) +} + +/// The spilled values `sheet` stores inside `range`, passed to `f` until it returns false. Values +/// outside the range are not walked: with fewer rows in the range than spilled values, each row +/// is looked up on its own; otherwise one ordered scan visits each spilled value at most once. +/// Returns how many values were looked at. +fn stored_spill_in<'s>(sheet: &'s gridcraft_model::Sheet, range: RangeRef, f: &mut dyn FnMut(CellRef, &'s Value) -> bool) -> usize { + let mut seen = 0usize; + if sheet.spill.is_empty() { + return 0; + } + if (range.height() as usize) < sheet.spill.len() { + for row in range.start.row..=range.end.row { + for (c, v) in sheet.spill.range(CellRef::new(row, range.start.col)..=CellRef::new(row, range.end.col)) { + seen += 1; + if !f(*c, v) { + return seen; + } + } + } + } else { + for (c, v) in sheet.spill.range(range.start..=range.end) { + seen += 1; + if range.contains(*c) && !f(*c, v) { + return seen; + } + } + } + seen +} + impl PassHost<'_> { /// Calculates `k` (and the formulas it reads) with bounded stack: when [`Self::compute`] would /// go deeper than [`MAX_NESTED`], it suspends what is in progress, the formula it needed is @@ -186,6 +246,9 @@ impl PassHost<'_> { } return v; } + if let Some(result) = self.python.get(&k).cloned() { + return self.python_result(k, result); + } let Some(expr) = self.exprs.get(&k) else { return self.wb.sheet(k.0).map(|s| s.value(k.1)).unwrap_or_default(); }; @@ -303,6 +366,36 @@ impl PassHost<'_> { v } + /// A Python cell's kept result as its value: an array spills from the cell unless it is + /// blocked, malformed or larger than [`gridcraft_model::python::MAX_RESULT_CELLS`] (only a + /// crafted file holds one), which shows `#SPILL!`. + fn python_result(&mut self, k: Key, result: Value) -> Value { + self.pending.remove(&k); + let v = match result { + Value::Array(a) => { + let sane = a.rows >= 1 + && a.cols >= 1 + && a.rows.checked_mul(a.cols).is_some_and(|n| n == a.data.len() && n <= gridcraft_model::python::MAX_RESULT_CELLS); + if sane { + self.spills.insert(k, a.clone()); + a.data.first().cloned().unwrap_or_default() + } else { + Value::Error(CellError::Spill) + } + } + v => v, + }; + let v = match self.spills.get(&k) { + Some(a) if spill_blocked(self, k, a) => { + self.spills.remove(&k); + Value::Error(CellError::Spill) + } + _ => v, + }; + self.results.insert(k, v.clone()); + v + } + /// A formula's previous value for iterative calculation: from the last iteration, else the /// value it shows. Blank, and the `#CIRC!` of a loop calculated with iteration off, are 0. fn previous(&self, k: Key) -> Value { @@ -443,26 +536,29 @@ impl Host for PassHost<'_> { let (h, w) = (range.height() as usize, range.width() as usize); let mut out = vec![Value::Empty; h * w]; let Some(sh) = self.wb.sheet(sheet) else { return out }; - // Sparse walk: only stored cells; formula cells that are (or were) dirty go through - // `compute`, constants are copied straight from the store. + // Sparse walk: only stored cells; formula and Python cells that are (or were) dirty go + // through `compute`, constants are copied straight from the store. let mut formulas: Vec<(usize, CellRef)> = Vec::new(); for (c, cell) in sh.cells.iter_range(range) { let i = (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize; - if cell.formula.is_some() { + if cell.formula.is_some() || cell.py.is_some() { formulas.push((i, c)); } else if let Some(slot) = out.get_mut(i) { *slot = cell.value.clone(); } } - for (c, v) in sh.spill.range(range.start..=range.end) { - if range.contains(*c) { - let i = (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize; - if let Some(slot) = out.get_mut(i) - && slot.is_empty() - { - *slot = v.clone(); - } + let _seen = stored_spill_in(sh, range, &mut |c, v| { + let i = (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize; + if let Some(slot) = out.get_mut(i) + && slot.is_empty() + { + *slot = v.clone(); } + true + }); + #[cfg(test)] + { + self.spill_visits += _seen; } for (i, c) in formulas { let v = self.cell_value(sheet, c); @@ -471,26 +567,31 @@ impl Host for PassHost<'_> { } } if !self.spills.is_empty() { - let new: Vec<(usize, Value)> = self - .spills - .iter() - .filter(|(a, _)| a.0 == sheet) - .flat_map(|(a, arr)| { - let mut v = Vec::new(); - for r in 0..arr.rows { - for cc in 0..arr.cols { - let c = CellRef::new(a.1.row.saturating_add(r as u32), a.1.col.saturating_add(cc as u32)); - if c != a.1 && range.contains(c) { - v.push(( - (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize, - arr.get(r, cc).cloned().unwrap_or_default(), - )); - } + // Only the part of each array inside the range is walked: a large array beside a small + // read costs nothing. + let mut new: Vec<(usize, Value)> = Vec::new(); + for (a, arr) in self.spills.iter().filter(|(a, _)| a.0 == sheet) { + let area = array_area(a.1, arr); + if !area.intersects(&range) { + continue; + } + let (r0, r1) = (area.start.row.max(range.start.row), area.end.row.min(range.end.row)); + let (c0, c1) = (area.start.col.max(range.start.col), area.end.col.min(range.end.col)); + for row in r0..=r1 { + for col in c0..=c1 { + let c = CellRef::new(row, col); + if c == a.1 { + continue; } + let v = arr.get((row - a.1.row) as usize, (col - a.1.col) as usize).cloned().unwrap_or_default(); + new.push(((row - range.start.row) as usize * w + (col - range.start.col) as usize, v)); } - v - }) - .collect(); + } + } + #[cfg(test)] + { + self.spill_visits += new.len(); + } // Iterating: an array spilled by a formula that read a previous value. if self.iterate && self.spills.iter().any(|(a, arr)| { @@ -514,21 +615,31 @@ impl Host for PassHost<'_> { fn visit_range(&mut self, sheet: usize, range: RangeRef, f: &mut dyn FnMut(&Value) -> bool) -> bool { let wb = self.wb; let Some(sh) = wb.sheet(sheet) else { return false }; - // Formula cells first, in row-major order like `range_values`: reading one calculates it - // and the pass remembers the value, so the walk below gets the same value again without - // this method holding any. + // Formula and Python cells first, in row-major order like `range_values`: reading one + // calculates it (a Python cell lays out its result) and the pass remembers the value, so + // the walk below gets the same value again without this method holding any. for (c, cell) in sh.cells.iter_range(range) { - if cell.formula.is_some() { + if cell.formula.is_some() || cell.py.is_some() { let _ = self.cell_value(sheet, c); } } // Spilled values sit between the stored cells (and the formulas just calculated may have // produced some): leave such blocks to `range_values`. - if self.spills.keys().any(|a| a.0 == sheet) || sh.spill.range(range.start..=range.end).any(|(c, _)| range.contains(*c)) { + let arrays = self.spills.iter().any(|(a, arr)| a.0 == sheet && array_area(a.1, arr).intersects(&range)); + let mut stored = false; + let _seen = stored_spill_in(sh, range, &mut |_, _| { + stored = true; + false + }); + #[cfg(test)] + { + self.spill_visits += _seen; + } + if arrays || stored { return false; } for (c, cell) in sh.cells.iter_range(range) { - let go = if cell.formula.is_some() { f(&self.cell_value(sheet, c)) } else { f(&cell.value) }; + let go = if cell.formula.is_some() || cell.py.is_some() { f(&self.cell_value(sheet, c)) } else { f(&cell.value) }; if !go { break; } diff --git a/crates/calc/src/tests.rs b/crates/calc/src/tests.rs index 5b9e90085..d63590737 100644 --- a/crates/calc/src/tests.rs +++ b/crates/calc/src/tests.rs @@ -614,6 +614,25 @@ fn lambda_calls_spend_one_work_budget_per_cell() { assert_eq!(t.num("A2"), 4000.0 * 4001.0 / 2.0); } +#[test] +fn lifted_lookups_stop_past_their_work_budget() { + // Security: each needle searched the whole lookup vector, so this one cell ran for hours + // (about 1.1e12 compares) and hung opening the file that held it. It is #NUM! at once now. + let mut t = T::new(); + t.set("A1", "=ROWS(XMATCH(SEQUENCE(1048576),-SEQUENCE(1048576)))"); + assert_eq!(t.get("A1"), Value::Error(CellError::Num)); + // Ordinary lifted lookups are unaffected. + for (i, k) in [10, 20, 30, 40, 50].into_iter().enumerate() { + t.set(&format!("B{}", i + 1), &k.to_string()); + t.set(&format!("C{}", i + 1), &(i + 1).to_string()); + } + t.set("D1", "=SUM(XMATCH({30;50;10},B1:B5))"); + t.set("D2", "=SUM(XLOOKUP({20;40},B1:B5,C1:C5))"); + t.set("D3", "=SUM(VLOOKUP({30,10},B1:C5,2,FALSE))"); + t.set("D4", "=SUM(MATCH({25,50},B1:B5))"); + assert_eq!([t.num("D1"), t.num("D2"), t.num("D3"), t.num("D4")], [9.0, 6.0, 4.0, 7.0]); +} + #[test] fn accounting_negative_text_coerces() { let mut t = T::new(); @@ -1232,6 +1251,25 @@ fn array_broadcast_result_is_capped() { assert_eq!(t.num("C3"), 5.0); } +/// MAP refuses a result past `MAX_CELLS` by its checked product, before allocating it (the +/// 2^32-cell wrap that a 32-bit `usize` product would turn into 0 is pinned in +/// `broadcast_len_rejects_a_product_that_wraps_to_zero_on_32_bits`; `SEQUENCE` tops out at +/// 16384 columns, so the widest product it can build is 16384 squared). +#[test] +fn map_caps_its_result_size() { + let mut t = T::new(); + for (at, formula) in + [("A1", "=MAP(SEQUENCE(4001),SEQUENCE(1,4000),LAMBDA(a,b,a))"), ("A2", "=MAP(SEQUENCE(16384),SEQUENCE(1,16384),LAMBDA(a,b,a))")] + { + t.set(at, formula); + assert_eq!(t.get(at), Value::Error(CellError::Num), "{formula}"); + } + // Within the cap the broadcast shape is unchanged. + t.set("A3", "=ROWS(MAP(SEQUENCE(3),SEQUENCE(1,2),LAMBDA(a,b,a+b)))"); + t.set("A4", "=COLUMNS(MAP(SEQUENCE(3),SEQUENCE(1,2),LAMBDA(a,b,a+b)))"); + assert_eq!((t.num("A3"), t.num("A4")), (3.0, 2.0)); +} + #[test] fn text_function() { let mut t = T::new(); @@ -2274,3 +2312,63 @@ fn loan_schedule_residue_keeps_excels_sign() { } assert_eq!(t.get("P7"), Value::Text("".into())); } + +/// Makes `at` a Python cell showing `cached` (its value before this calculation), with `result` +/// as its kept result, and returns the cell's key for `Calc::cells_changed`. +fn plant_python(t: &mut T, at: &str, cached: Value, result: Value) -> (usize, CellRef) { + use gridcraft_model::python::{PyCode, PyOutput}; + let output = PyOutput { value: result, type_name: "list".into(), preview: String::new(), convertible: true }; + let py = PyCode { output: Some(output), ..PyCode::new("[5, 6]") }; + let cell = Cell { value: cached, formula: None, style: gridcraft_model::StyleId::default(), py: Some(std::sync::Arc::new(py)) }; + t.wb.sheet_mut(0).unwrap().cells.set(c(at), cell); + (0, c(at)) +} + +fn column(values: &[f64]) -> Value { + let data = values.iter().map(|v| Value::Number(*v)).collect(); + Value::Array(std::sync::Arc::new(gridcraft_core::Array::new(values.len(), 1, data).unwrap())) +} + +#[test] +fn formulas_read_a_dirty_python_cell_after_it_is_laid_out() { + // B1's result can't spill (B2 holds a value), so it is #SPILL!. A1, A2 and A3 come before + // it in the pass; each must lay it out first rather than read its old value, 1. + let build = |t: &mut T| { + t.set("B2", "10"); + t.set("B3", "100"); + t.set("A1", "=B1"); + t.set("A2", "=MAX(B1:B3)"); + t.set("A3", "=SUM(B1:B3)"); + t.set("A4", "=COUNTA(B1:B3)"); + t.set("A5", "=SUMPRODUCT(--ISERROR(B1:B3))"); + }; + let check = |t: &T, how: &str| { + assert_eq!(t.get("B1"), Value::Error(CellError::Spill), "{how}"); + assert_eq!(t.get("A1"), Value::Error(CellError::Spill), "{how}: a reference"); + assert_eq!(t.get("A2"), Value::Error(CellError::Spill), "{how}: a range read"); + assert_eq!(t.get("A3"), Value::Error(CellError::Spill), "{how}: a range walked cell by cell"); + assert_eq!(t.num("A4"), 3.0, "{how}"); + assert_eq!(t.num("A5"), 1.0, "{how}: the error is seen in the range"); + }; + + // A full recalculation. + let mut t = T::new(); + build(&mut t); + plant_python(&mut t, "B1", Value::Number(1.0), column(&[5.0, 6.0])); + t.calc.recalc_all(&mut t.wb); + check(&t, "full"); + + // An incremental one: B1 becomes a Python cell after its readers. + let mut t = T::new(); + t.set("B1", "1"); + build(&mut t); + assert_eq!((t.num("A1"), t.num("A3")), (1.0, 111.0)); + let k = plant_python(&mut t, "B1", Value::Number(1.0), column(&[5.0, 6.0])); + t.calc.cells_changed(&mut t.wb, &[k]); + check(&t, "incremental"); + + // Unblocked, the readers see the new array. + t.set("B2", ""); + assert_eq!((t.num("B1"), t.num("B2")), (5.0, 6.0)); + assert_eq!((t.num("A1"), t.num("A2"), t.num("A3")), (5.0, 100.0, 111.0)); +} diff --git a/crates/chart/src/render.rs b/crates/chart/src/render.rs index 76a1619e3..7e50e78f6 100644 --- a/crates/chart/src/render.rs +++ b/crates/chart/src/render.rs @@ -1,9 +1,8 @@ //! Chart layout: title, legend, axes and every chart kind, drawn into [`Prim`]s. -use std::collections::HashSet; use std::f32::consts::{PI, TAU}; -use gridcraft_model::{Chart, ChartKind, LegendPos, LineDash, QuartileMethod, Theme, TrendlineKind}; +use gridcraft_model::{Chart, ChartKind, LineDash, QuartileMethod, Theme, TrendlineKind}; use crate::axis::{Fmt, Scale, clean, fixed_scale, nice_scale}; use crate::resolve::{MAX_POINTS, MAX_SERIES, MAX_TRENDLINES, series_color}; @@ -15,6 +14,7 @@ mod frame; mod kinds; mod legend; mod lines; +mod table; mod trend; use cartesian::{cartesian, scatter}; @@ -70,8 +70,7 @@ impl Ctx<'_> { } String::new() } - #[allow(clippy::too_many_arguments)] - fn text(&mut self, x: f32, y: f32, s: &str, size: f32, color: Rgba, align: HAlign, valign: VAlign) { + fn text(&mut self, x: f32, y: f32, s: &str, size: f32, color: Rgba, (align, valign): (HAlign, VAlign)) { if s.is_empty() { return; } @@ -135,7 +134,7 @@ pub fn render(chart: &Chart, data: &ChartData, w: f32, h: f32, m: &dyn Measure) if let Some(t) = chart_title(chart, &data) { let size = TITLE_SIZE.min(h * 0.12).max(4.0); let t = ctx.fit(&t, size, area.w); - ctx.text(w / 2.0, area.y, &t, size, TEXT, HAlign::Center, VAlign::Top); + ctx.text(w / 2.0, area.y, &t, size, TEXT, (HAlign::Center, VAlign::Top)); let used = size * 1.3 + 4.0; area.y += used; area.h -= used; diff --git a/crates/chart/src/render/cartesian.rs b/crates/chart/src/render/cartesian.rs index 85a577b08..ce0bbebb0 100644 --- a/crates/chart/src/render/cartesian.rs +++ b/crates/chart/src/render/cartesian.rs @@ -2,7 +2,10 @@ use super::frame::{Bx, CatAxis, ValAxis, frame, on, value_range}; use super::legend::{cat_label, plot_kind}; use super::lines::{decimate, draw_markers, draw_runs}; use super::trend::{draw_trend_labels, draw_trendlines, max_backward, max_forward, max_forward_slots, supports_trendline}; -use super::*; +use super::{ + Chart, ChartData, ChartKind, Ctx, Fmt, HAlign, LABEL_SIZE, MAX_CATS, MAX_LABELS, MAX_LINE_PTS, MAX_MARKERS, Prim, Scale, SeriesData, TEXT, + VAlign, WHITE, clean, fixed_scale, nice_scale, +}; // ---------------------------------------------------------------- cartesian (column/bar/line/area/combo) @@ -18,6 +21,9 @@ enum Fam { Area100, } +/// Stack accumulator: (axis, family) key with positive and negative running sums. +type Stack = ((bool, Fam), Vec, Vec); + fn family(k: ChartKind) -> Fam { use ChartKind::*; match k { @@ -45,16 +51,7 @@ fn is_line(f: Fam) -> bool { matches!(f, Fam::Line | Fam::LineStack) } -pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx, horizontal: bool) { - let ncat = data.categories.len().max(data.series.iter().map(|s| s.values.len()).max().unwrap_or(0)).min(MAX_CATS); - if ncat == 0 { - return; - } - let kinds: Vec = data.series.iter().map(|s| if horizontal { chart.kind } else { plot_kind(chart, s) }).collect(); - let fams: Vec = kinds.iter().map(|k| family(*k)).collect(); - let any_primary = data.series.iter().any(|s| !s.secondary); - let sec: Vec = data.series.iter().map(|s| s.secondary && any_primary && !horizontal).collect(); - +fn series_spans(data: &ChartData, fams: &[Fam], sec: &[bool], ncat: usize) -> Vec>> { // Totals for 100% stacking, per axis and family (100% columns and 100% areas stack apart). let mut totals: Vec<((bool, Fam), Vec)> = Vec::new(); for (i, s) in data.series.iter().enumerate() { @@ -81,7 +78,7 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx if t > 0.0 { v / t } else { 0.0 } }; // Stack accumulators keyed by (axis, family): positive and negative running sums. - let mut stacks: Vec<((bool, Fam), Vec, Vec)> = Vec::new(); + let mut stacks: Vec = Vec::new(); let mut spans: Vec>> = Vec::with_capacity(data.series.len()); for (i, s) in data.series.iter().enumerate() { let fam = fams.get(i).copied().unwrap_or(Fam::Col); @@ -126,6 +123,10 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx spans.push(row); } + spans +} + +fn axis_scales(data: &ChartData, fams: &[Fam], sec: &[bool], spans: &[Vec>]) -> Vec<(Scale, Fmt)> { // Axis scales. let mut axes: Vec<(Scale, Fmt)> = Vec::new(); for axis in [false, true] { @@ -151,35 +152,13 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx break; } } - let Some((s1, f1)) = axes.first() else { return }; - let y1 = ValAxis { scale: *s1, fmt: f1 }; - let y2 = axes.get(1).map(|(s, f)| ValAxis { scale: *s, fmt: f }); - // A forward forecast on a category axis adds empty category slots after the data. - let fwd = data - .series - .iter() - .enumerate() - .filter(|(i, _)| kinds.get(*i).is_some_and(|k| supports_trendline(*k))) - .map(|(_, s)| max_forward(s)) - .fold(0.0, f64::max); - let ncat_slots = ncat.saturating_add(fwd.min(max_forward_slots(ncat)).ceil() as usize).min(MAX_CATS.max(ncat)); - let labels: Vec = (0..ncat_slots).map(|j| if j < ncat { cat_label(data, j) } else { String::new() }).collect(); - // Outer levels padded to the slots (forecast slots stay in the last group). - let outer: Vec> = - data.category_levels.iter().map(|l| (0..ncat_slots).map(|j| l.get(j).cloned().unwrap_or_default()).collect()).collect(); - let plot = frame(ctx, area, CatAxis::Labels(&labels, &outer), &y1, y2.as_ref(), horizontal, chart.gridlines); - if !plot.ok() { - return; - } - let scale_of = |i: usize| -> Scale { if sec.get(i).copied().unwrap_or(false) { y2.as_ref().map(|a| a.scale).unwrap_or(*s1) } else { *s1 } }; - let cat_len = if horizontal { plot.h } else { plot.w }; - let cw = cat_len / ncat_slots as f32; - let center = |j: usize| if horizontal { plot.bottom() - (j as f32 + 0.5) * cw } else { plot.x + (j as f32 + 0.5) * cw }; - let vpos = |v: f64, sc: &Scale| if horizontal { plot.x + sc.frac(v) * plot.w } else { plot.bottom() - sc.frac(v) * plot.h }; + axes +} +fn column_slots(fams: &[Fam], sec: &[bool]) -> (Vec, f32) { // Column slots: each clustered series gets one, each stack group shares one. let mut slot_keys: Vec<(bool, Fam, usize)> = Vec::new(); - let mut slot_of = vec![0usize; data.series.len()]; + let mut slot_of = vec![0usize; fams.len()]; for (i, f) in fams.iter().enumerate() { if !is_col(*f) { continue; @@ -197,7 +176,51 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx *s = k; } } - let nslots = slot_keys.len().max(1) as f32; + (slot_of, slot_keys.len().max(1) as f32) +} + +pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx, horizontal: bool) { + let ncat = data.categories.len().max(data.series.iter().map(|s| s.values.len()).max().unwrap_or(0)).min(MAX_CATS); + if ncat == 0 { + return; + } + let kinds: Vec = data.series.iter().map(|s| if horizontal { chart.kind } else { plot_kind(chart, s) }).collect(); + let fams: Vec = kinds.iter().map(|k| family(*k)).collect(); + let any_primary = data.series.iter().any(|s| !s.secondary); + let sec: Vec = data.series.iter().map(|s| s.secondary && any_primary && !horizontal).collect(); + + let spans = series_spans(data, &fams, &sec, ncat); + + let axes = axis_scales(data, &fams, &sec, &spans); + let Some((s1, f1)) = axes.first() else { return }; + let y1 = ValAxis { scale: *s1, fmt: f1 }; + let y2 = axes.get(1).map(|(s, f)| ValAxis { scale: *s, fmt: f }); + // A forward forecast on a category axis adds empty category slots after the data. + let fwd = data + .series + .iter() + .enumerate() + .filter(|(i, _)| kinds.get(*i).is_some_and(|k| supports_trendline(*k))) + .map(|(_, s)| max_forward(s)) + .fold(0.0, f64::max); + let ncat_slots = ncat.saturating_add(fwd.min(max_forward_slots(ncat)).ceil() as usize).min(MAX_CATS.max(ncat)); + let labels: Vec = (0..ncat_slots).map(|j| if j < ncat { cat_label(data, j) } else { String::new() }).collect(); + // Outer levels padded to the slots (forecast slots stay in the last group). + let outer: Vec> = + data.category_levels.iter().map(|l| (0..ncat_slots).map(|j| l.get(j).cloned().unwrap_or_default()).collect()).collect(); + let (area, table) = super::table::reserve(ctx, chart, data, area, ncat_slots); + let category_axis = super::table::category_axis(&table, &labels, &outer); + let plot = frame(ctx, area, category_axis, &y1, y2.as_ref(), horizontal, chart.gridlines); + if !plot.ok() { + return; + } + let scale_of = |i: usize| -> Scale { if sec.get(i).copied().unwrap_or(false) { y2.as_ref().map(|a| a.scale).unwrap_or(*s1) } else { *s1 } }; + let cat_len = if horizontal { plot.h } else { plot.w }; + let cw = cat_len / ncat_slots as f32; + let center = |j: usize| if horizontal { plot.bottom() - (j as f32 + 0.5) * cw } else { plot.x + (j as f32 + 0.5) * cw }; + let vpos = |v: f64, sc: &Scale| if horizontal { plot.x + sc.frac(v) * plot.w } else { plot.bottom() - sc.frac(v) * plot.h }; + + let (slot_of, nslots) = column_slots(&fams, &sec); let bw = cw / (nslots + 1.5); let labels_on = chart.data_labels && ncat <= MAX_LABELS; // Area labels wait until every area is filled, so a later (stacked) area can't paint over them. @@ -275,20 +298,20 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx if horizontal { let y = c + nslots * bw / 2.0 - (k + 0.5) * bw; if stacked { - ctx.text(mid, y, &t, LABEL_SIZE, on(s.color), HAlign::Center, VAlign::Middle); + ctx.text(mid, y, &t, LABEL_SIZE, on(s.color), (HAlign::Center, VAlign::Middle)); } else if v >= 0.0 { - ctx.text(pb.max(pt) + 3.0, y, &t, LABEL_SIZE, TEXT, HAlign::Left, VAlign::Middle); + ctx.text(pb.max(pt) + 3.0, y, &t, LABEL_SIZE, TEXT, (HAlign::Left, VAlign::Middle)); } else { - ctx.text(lo - 3.0, y, &t, LABEL_SIZE, TEXT, HAlign::Right, VAlign::Middle); + ctx.text(lo - 3.0, y, &t, LABEL_SIZE, TEXT, (HAlign::Right, VAlign::Middle)); } } else { let x = c - nslots * bw / 2.0 + (k + 0.5) * bw; if stacked { - ctx.text(x, mid, &t, LABEL_SIZE, on(s.color), HAlign::Center, VAlign::Middle); + ctx.text(x, mid, &t, LABEL_SIZE, on(s.color), (HAlign::Center, VAlign::Middle)); } else if v >= 0.0 { - ctx.text(x, lo - 2.0, &t, LABEL_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(x, lo - 2.0, &t, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); } else { - ctx.text(x, lo + len + 2.0, &t, LABEL_SIZE, TEXT, HAlign::Center, VAlign::Top); + ctx.text(x, lo + len + 2.0, &t, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Top)); } } } @@ -304,7 +327,7 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx if labels_on { for (j, p) in pts.iter().enumerate() { if let (Some(p), Some(v)) = (p, s.values.get(j).copied().flatten()) { - ctx.text(p[0], p[1] - 5.0, &fmt.format(v, None), LABEL_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(p[0], p[1] - 5.0, &fmt.format(v, None), LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); } } } @@ -312,7 +335,7 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx } } for (x, y, t) in area_labels.drain(..) { - ctx.text(x, y, &t, LABEL_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(x, y, &t, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); } } // Trendlines on top: x is the category number 1..n. @@ -334,6 +357,7 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx draw_trendlines(ctx, s, &pts, &frac, &px, &mut trend_labels, (0.5, ncat_slots as f64 + 0.5)); } draw_trend_labels(ctx, plot, &trend_labels); + super::table::draw(ctx, chart, data, plot, table.as_ref()); } // ---------------------------------------------------------------- scatter & bubble @@ -412,7 +436,7 @@ pub(super) fn scatter(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx) if chart.data_labels && pts.len() <= MAX_LABELS { for (p, raw) in px.iter().zip(pts) { if let (Some(p), Some((_, y))) = (p, raw) { - ctx.text(p[0], p[1] - 6.0, &fy.format(*y, None), LABEL_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(p[0], p[1] - 6.0, &fy.format(*y, None), LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); } } } diff --git a/crates/chart/src/render/frame.rs b/crates/chart/src/render/frame.rs index 7fa2a54a1..92d8dd9e3 100644 --- a/crates/chart/src/render/frame.rs +++ b/crates/chart/src/render/frame.rs @@ -1,4 +1,4 @@ -use super::*; +use super::{AXIS, AXIS_TITLE_SIZE, Chart, ChartData, Ctx, DARK_TEXT, Fmt, GRID, HAlign, LABEL_SIZE, PI, Prim, Rgba, Scale, TEXT, VAlign, WHITE}; #[derive(Clone, Copy, Debug)] pub(super) struct Bx { @@ -44,7 +44,7 @@ pub(super) fn axis_titles(ctx: &mut Ctx, chart: &Chart, mut area: Bx) -> Bx { let lh = AXIS_TITLE_SIZE * 1.4 + 2.0; if let Some(t) = chart.x_title.as_deref().map(str::trim).filter(|t| !t.is_empty()) { let t = ctx.fit(t, AXIS_TITLE_SIZE, area.w); - ctx.text(area.cx(), area.bottom(), &t, AXIS_TITLE_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(area.cx(), area.bottom(), &t, AXIS_TITLE_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); area.h -= lh; } if let Some(t) = chart.y_title.as_deref().map(str::trim).filter(|t| !t.is_empty()) { @@ -74,12 +74,14 @@ pub(super) enum CatAxis<'a> { /// Category labels and, for a multi-level axis, the outer levels (nearest the labels first; /// one entry per category, non-empty where a group starts). Labels(&'a [String], &'a [Vec]), + /// A data table supplies the labels below a vertical chart. Its stub must clear the plot. + Table(&'a [String], f32), Numeric(Scale, &'a Fmt), } /// Groups of an outer category level: `(first, end)` index pairs, a group starting at every /// non-empty entry (and at the first category). -fn level_groups(level: &[String], n: usize) -> Vec<(usize, usize)> { +pub(super) fn level_groups(level: &[String], n: usize) -> Vec<(usize, usize)> { let mut starts: Vec = (0..n).filter(|&j| j == 0 || level.get(j).is_some_and(|t| !t.is_empty())).collect(); starts.push(n); starts.windows(2).filter_map(|w| Some((*w.first()?, *w.get(1)?))).collect() @@ -101,9 +103,13 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti let ls = LABEL_SIZE; let lh = ls * 1.4; let ylabels = y.labels(); + let stub = match &cat { + CatAxis::Table(_, width) => *width, + _ => 0.0, + }; let maxw = |ctx: &Ctx, l: &[(f64, String)]| l.iter().map(|(_, t)| ctx.tw(t, ls)).fold(0.0f32, f32::max); if !horizontal { - let left = (maxw(ctx, &ylabels) + 6.0).min(area.w * 0.4); + let left = (maxw(ctx, &ylabels) + 6.0).min(area.w * 0.4).max(stub); let y2labels = y2.map(ValAxis::labels); let right = match (&y2labels, &cat) { (Some(l), _) => (maxw(ctx, l) + 6.0).min(area.w * 0.3), @@ -112,7 +118,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti }; let levels = match &cat { CatAxis::Labels(_, o) => o.len() as f32, - CatAxis::Numeric(..) => 0.0, + CatAxis::Numeric(..) | CatAxis::Table(..) => 0.0, }; let plot = Bx { x: area.x + left, y: area.y + lh * 0.5, w: area.w - left - right, h: area.h - lh * 1.5 - 4.0 - lh * levels }; if !plot.ok() { @@ -123,18 +129,19 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti if grid { ctx.hline(plot.x, plot.right(), yy, GRID, 0.75); } - ctx.text(plot.x - 4.0, yy, t, ls, TEXT, HAlign::Right, VAlign::Middle); + ctx.text(plot.x - 4.0, yy, t, ls, TEXT, (HAlign::Right, VAlign::Middle)); } if let (Some(a), Some(l)) = (y2, &y2labels) { for (v, t) in l { let yy = plot.bottom() - a.scale.frac(*v) * plot.h; - ctx.text(plot.right() + 4.0, yy, t, ls, TEXT, HAlign::Left, VAlign::Middle); + ctx.text(plot.right() + 4.0, yy, t, ls, TEXT, (HAlign::Left, VAlign::Middle)); } } let zero = if y.scale.min <= 0.0 && y.scale.max >= 0.0 { 0.0 } else { y.scale.min }; let y0 = plot.bottom() - y.scale.frac(zero) * plot.h; ctx.hline(plot.x, plot.right(), y0, AXIS, 0.75); match cat { + CatAxis::Table(..) => {} CatAxis::Labels(l, outer) => { let n = l.len().max(1); let cw = plot.w / n as f32; @@ -142,7 +149,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti let k = (((lw + 6.0) / cw.max(1e-3)).ceil() as usize).clamp(1, n); for (j, t) in l.iter().enumerate().step_by(k) { let t = ctx.fit(t, ls, cw * k as f32 - 2.0); - ctx.text(plot.x + (j as f32 + 0.5) * cw, plot.bottom() + 3.0, &t, ls, TEXT, HAlign::Center, VAlign::Top); + ctx.text(plot.x + (j as f32 + 0.5) * cw, plot.bottom() + 3.0, &t, ls, TEXT, (HAlign::Center, VAlign::Top)); } // Outer levels, one row each below the labels: a group's name centred under its // categories, and a separator at each group boundary down through the rows. @@ -152,7 +159,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti let (x0, x1) = (plot.x + first as f32 * cw, plot.x + end as f32 * cw); if let Some(t) = level.get(first).filter(|t| !t.is_empty()) { let t = ctx.fit(t, ls, x1 - x0 - 2.0); - ctx.text((x0 + x1) / 2.0, top, &t, ls, TEXT, HAlign::Center, VAlign::Top); + ctx.text((x0 + x1) / 2.0, top, &t, ls, TEXT, (HAlign::Center, VAlign::Top)); } if first > 0 { ctx.vline(x0, plot.bottom(), top + lh - 2.0, AXIS, 0.75); @@ -169,7 +176,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti let k = (((lw + 6.0) / slot.max(1e-3)).ceil() as usize).max(1); for (v, t) in ticks.iter().zip(&labels).step_by(k) { let xx = plot.x + s.frac(*v) * plot.w; - ctx.text(xx, plot.bottom() + 3.0, t, ls, TEXT, HAlign::Center, VAlign::Top); + ctx.text(xx, plot.bottom() + 3.0, t, ls, TEXT, (HAlign::Center, VAlign::Top)); } if s.min <= 0.0 && s.max >= 0.0 { let xx = plot.x + s.frac(0.0) * plot.w; @@ -181,6 +188,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti } else { let (labels, outer): (Vec, &[Vec]) = match cat { CatAxis::Labels(l, o) => (l.to_vec(), o), + CatAxis::Table(l, _) => (l.to_vec(), &[]), CatAxis::Numeric(..) => (vec![], &[]), }; let lw = (labels.iter().take(2000).map(|t| ctx.tw(t, ls)).fold(0.0f32, f32::max) + 6.0).min(area.w * 0.35); @@ -189,7 +197,8 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti outer.iter().map(|l| (l.iter().take(2000).map(|t| ctx.tw(t, ls)).fold(0.0f32, f32::max) + 6.0).min(area.w * 0.15)).collect(); let levels_w: f32 = level_w.iter().sum(); let right = ylabels.last().map(|(_, t)| ctx.tw(t, ls) / 2.0).unwrap_or(0.0).min(area.w * 0.2) + 2.0; - let plot = Bx { x: area.x + lw + levels_w, y: area.y + 2.0, w: area.w - lw - levels_w - right, h: area.h - lh - 6.0 }; + let left = (lw + levels_w).max(stub); + let plot = Bx { x: area.x + left, y: area.y + 2.0, w: area.w - left - right, h: area.h - lh - 6.0 }; if !plot.ok() { return plot; } @@ -202,7 +211,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti ctx.vline(xx, plot.y, plot.bottom(), GRID, 0.75); } if i % k == 0 { - ctx.text(xx, plot.bottom() + 3.0, t, ls, TEXT, HAlign::Center, VAlign::Top); + ctx.text(xx, plot.bottom() + 3.0, t, ls, TEXT, (HAlign::Center, VAlign::Top)); } } let zero = if y.scale.min <= 0.0 && y.scale.max >= 0.0 { 0.0 } else { y.scale.min }; @@ -213,7 +222,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti let k = ((lh / ch.max(1e-3)).ceil() as usize).clamp(1, n); for (j, t) in labels.iter().enumerate().step_by(k) { let t = ctx.fit(t, ls, lw - 6.0); - ctx.text(plot.x - 4.0, plot.bottom() - (j as f32 + 0.5) * ch, &t, ls, TEXT, HAlign::Right, VAlign::Middle); + ctx.text(plot.x - 4.0, plot.bottom() - (j as f32 + 0.5) * ch, &t, ls, TEXT, (HAlign::Right, VAlign::Middle)); } // Outer levels: a group's name centred beside its categories, separators at the // boundaries running right to the axis. @@ -224,7 +233,7 @@ pub(super) fn frame(ctx: &mut Ctx, area: Bx, cat: CatAxis, y: &ValAxis, y2: Opti let (y0, y1) = (plot.bottom() - end as f32 * ch, plot.bottom() - first as f32 * ch); if let Some(t) = level.get(first).filter(|t| !t.is_empty()) { let t = ctx.fit(t, ls, w - 4.0); - ctx.text((x0 + x1) / 2.0, (y0 + y1) / 2.0, &t, ls, TEXT, HAlign::Center, VAlign::Middle); + ctx.text((x0 + x1) / 2.0, (y0 + y1) / 2.0, &t, ls, TEXT, (HAlign::Center, VAlign::Middle)); } if first > 0 { ctx.hline(x0, plot.x, y1, AXIS, 0.75); diff --git a/crates/chart/src/render/kinds.rs b/crates/chart/src/render/kinds.rs index 2ffbfa3dd..54ce6bec8 100644 --- a/crates/chart/src/render/kinds.rs +++ b/crates/chart/src/render/kinds.rs @@ -1,6 +1,9 @@ use super::frame::{Bx, CatAxis, ValAxis, frame, on, value_range}; use super::legend::{cat_label, point_color}; -use super::*; +use super::{ + AXIS, Chart, ChartData, Ctx, DOWN, Fmt, GRID, HAlign, LABEL_SIZE, MAX_CATS, MAX_LABELS, MAX_MARKERS, Prim, QuartileMethod, STOCK, SeriesData, + TAU, TEXT, UP, VAlign, WHITE, clean, nice_scale, +}; // ---------------------------------------------------------------- histogram @@ -47,7 +50,7 @@ pub(super) fn histogram(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx let x = plot.x + b as f32 * cw; ctx.rect(x, top, cw, plot.bottom() - top, s.color, Some((WHITE, 0.75))); if chart.data_labels { - ctx.text(x + cw / 2.0, top - 2.0, &c.to_string(), LABEL_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(x + cw / 2.0, top - 2.0, &c.to_string(), LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); } } } @@ -110,7 +113,7 @@ pub(super) fn waterfall(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx } if chart.data_labels && ncat <= MAX_LABELS { let v = if *kind == 0 { *t } else { t - b }; - ctx.text(x + bw / 2.0, yb.min(yt) - 2.0, &f.format(v, None), LABEL_SIZE, TEXT, HAlign::Center, VAlign::Bottom); + ctx.text(x + bw / 2.0, yb.min(yt) - 2.0, &f.format(v, None), LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); } } } @@ -150,10 +153,10 @@ pub(super) fn funnel(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx) { if chart.data_labels { let t = f.format(s.values.get(j).copied().flatten().unwrap_or(0.0), None); let color = if ctx.tw(&t, LABEL_SIZE) + 4.0 <= bw { on(s.color) } else { TEXT }; - ctx.text(plot.cx(), yc, &t, LABEL_SIZE, color, HAlign::Center, VAlign::Middle); + ctx.text(plot.cx(), yc, &t, LABEL_SIZE, color, (HAlign::Center, VAlign::Middle)); } let l = ctx.fit(labels.get(j).map(String::as_str).unwrap_or(""), LABEL_SIZE, lw - 8.0); - ctx.text(plot.x - 6.0, yc, &l, LABEL_SIZE, TEXT, HAlign::Right, VAlign::Middle); + ctx.text(plot.x - 6.0, yc, &l, LABEL_SIZE, TEXT, (HAlign::Right, VAlign::Middle)); } } } @@ -194,7 +197,7 @@ pub(super) fn pie(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx, inne a += sweep; } for (x, y, t, c) in labels { - ctx.text(x, y, &t, LABEL_SIZE, c, HAlign::Center, VAlign::Middle); + ctx.text(x, y, &t, LABEL_SIZE, c, (HAlign::Center, VAlign::Middle)); } } @@ -281,11 +284,11 @@ pub(super) fn treemap(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx) ctx.rect(b.x, b.y, b.w, b.h, fill, Some((WHITE, 1.5))); if b.w > 24.0 && b.h > LABEL_SIZE * 1.6 { let name = ctx.fit(&cat_label(data, j), LABEL_SIZE, b.w - 8.0); - ctx.text(b.x + 4.0, b.y + 4.0, &name, LABEL_SIZE, on(fill), HAlign::Left, VAlign::Top); + ctx.text(b.x + 4.0, b.y + 4.0, &name, LABEL_SIZE, on(fill), (HAlign::Left, VAlign::Top)); if chart.data_labels && b.h > LABEL_SIZE * 3.2 { let v = items.iter().find(|x| x.0 == j).map(|x| x.1).unwrap_or(0.0); let t = ctx.fit(&f.format(v, None), LABEL_SIZE, b.w - 8.0); - ctx.text(b.x + 4.0, b.y + 4.0 + LABEL_SIZE * 1.4, &t, LABEL_SIZE, on(fill), HAlign::Left, VAlign::Top); + ctx.text(b.x + 4.0, b.y + 4.0 + LABEL_SIZE * 1.4, &t, LABEL_SIZE, on(fill), (HAlign::Left, VAlign::Top)); } } } @@ -325,7 +328,7 @@ pub(super) fn radar(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx) { ctx.line(ring, GRID, 0.75); } if ti % every == 0 { - ctx.text(cx - 3.0, cy - rr, &f.format(t, Some(sc.step)), LABEL_SIZE * 0.9, TEXT, HAlign::Right, VAlign::Middle); + ctx.text(cx - 3.0, cy - rr, &f.format(t, Some(sc.step)), LABEL_SIZE * 0.9, TEXT, (HAlign::Right, VAlign::Middle)); } } let step = ncat.div_ceil(72).max(1); @@ -349,7 +352,7 @@ pub(super) fn radar(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx) { VAlign::Middle }; let t = ctx.fit(labels.get(j).map(String::as_str).unwrap_or(""), LABEL_SIZE, lw.max(20.0)); - ctx.text(p[0], p[1], &t, LABEL_SIZE, TEXT, align, valign); + ctx.text(p[0], p[1], &t, LABEL_SIZE, TEXT, (align, valign)); } for s in &data.series { let mut pts: Vec<[f32; 2]> = (0..ncat).map(|j| pt(ang(j), sc.frac(s.values.get(j).copied().flatten().unwrap_or(sc.min)) * r)).collect(); diff --git a/crates/chart/src/render/legend.rs b/crates/chart/src/render/legend.rs index 95cb39d48..5ddb5a3cd 100644 --- a/crates/chart/src/render/legend.rs +++ b/crates/chart/src/render/legend.rs @@ -1,6 +1,10 @@ use super::frame::Bx; use super::trend::supports_trendline; -use super::*; +use super::{ + Chart, ChartData, ChartKind, Ctx, DOWN, HAlign, LABEL_SIZE, MAX_CATS, MAX_SERIES, MAX_TRENDLINES, Prim, Rgba, SeriesData, TEXT, Theme, UP, + VAlign, auto_name, series_color, +}; +use gridcraft_model::LegendPos; /// How a legend entry's swatch is drawn. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -81,7 +85,7 @@ fn swatch_w(sw: Swatch, s: f32) -> f32 { if sw == Swatch::Box { s } else { s * 2.0 } } -fn draw_swatch(ctx: &mut Ctx, sw: Swatch, x: f32, yc: f32, s: f32, color: Rgba) { +pub(super) fn draw_swatch(ctx: &mut Ctx, sw: Swatch, x: f32, yc: f32, s: f32, color: Rgba) { match sw { Swatch::Box => ctx.rect(x, yc - s / 2.0, s, s, color, None), Swatch::Line => ctx.hline(x, x + 2.0 * s, yc, color, 2.0), @@ -132,7 +136,7 @@ pub(super) fn draw_legend(ctx: &mut Ctx, entries: &[(String, Rgba, Swatch)], pos draw_swatch(ctx, e.2, x, yc, s, e.1); let sx = swatch_w(e.2, s) + gap; let t = ctx.fit(&e.0, size, (iw - sx).max(0.0) + 0.5); - ctx.text(x + sx, yc, &t, size, TEXT, HAlign::Left, VAlign::Middle); + ctx.text(x + sx, yc, &t, size, TEXT, (HAlign::Left, VAlign::Middle)); } x += iw + spacing; } @@ -152,7 +156,7 @@ pub(super) fn draw_legend(ctx: &mut Ctx, entries: &[(String, Rgba, Swatch)], pos draw_swatch(ctx, e.2, x0, yc, s, e.1); let sx = swatch_w(e.2, s) + gap; let t = ctx.fit(&e.0, size, (colw - sx).max(0.0) + 0.5); - ctx.text(x0 + sx, yc, &t, size, TEXT, HAlign::Left, VAlign::Middle); + ctx.text(x0 + sx, yc, &t, size, TEXT, (HAlign::Left, VAlign::Middle)); } let used = colw + 8.0; if pos == LegendPos::Right { Bx { w: area.w - used, ..area } } else { Bx { x: area.x + used, w: area.w - used, ..area } } diff --git a/crates/chart/src/render/lines.rs b/crates/chart/src/render/lines.rs index 9cef4608c..ae0229cfb 100644 --- a/crates/chart/src/render/lines.rs +++ b/crates/chart/src/render/lines.rs @@ -1,5 +1,6 @@ use super::frame::Bx; -use super::*; +use super::{Ctx, MAX_LINE_PTS, MAX_MARKERS, Prim, Rgba, WHITE}; +use std::collections::HashSet; // ---------------------------------------------------------------- helpers diff --git a/crates/chart/src/render/table.rs b/crates/chart/src/render/table.rs new file mode 100644 index 000000000..f0839dcfe --- /dev/null +++ b/crates/chart/src/render/table.rs @@ -0,0 +1,107 @@ +//! The chart data table uses the same primitives as the plot, including in print/PDF output. + +use super::frame::{Bx, CatAxis, level_groups}; +use super::legend::{Swatch, cat_label, draw_swatch, plot_kind}; +use super::{AXIS, Chart, ChartData, ChartKind, Ctx, Fmt, HAlign, LABEL_SIZE, TEXT, VAlign}; + +pub(super) struct Layout { + top: f32, + row_height: f32, + headers: usize, + stub: f32, + columns: usize, +} + +pub(super) fn category_axis<'a>(layout: &Option, labels: &'a [String], outer: &'a [Vec]) -> CatAxis<'a> { + match layout { + Some(t) => CatAxis::Table(labels, t.stub), + None => CatAxis::Labels(labels, outer), + } +} + +/// Reserve at most half the available height. A table that cannot fit is explicitly labelled +/// instead of dropping rows or allocating a series × point-sized matrix. +pub(super) fn reserve(ctx: &mut Ctx, chart: &Chart, data: &ChartData, mut area: Bx, columns: usize) -> (Bx, Option) { + if chart.data_table.is_none() || !chart.supports_data_table() { + return (area, None); + } + let headers = data.category_levels.len() + 1; + let rows = headers + data.series.len(); + let row_height = LABEL_SIZE * 1.4 + 4.0; + let height = row_height * rows as f32; + let key_width = if chart.data_table.as_ref().is_some_and(|t| t.show_keys) { 22.0 } else { 0.0 }; + let names = data.series.iter().take(32).map(|s| ctx.tw(&s.name, LABEL_SIZE)).fold(0.0f32, f32::max); + let stub = (names + key_width + 10.0).clamp(24.0, (area.w * 0.3).max(24.0)); + if columns == 0 || columns > 128 || data.series.len() > 32 || height > area.h * 0.5 || (area.w * 0.6) / (columns as f32) < 18.0 { + let message = ctx.fit("Data table needs more space", LABEL_SIZE, area.w); + ctx.text(area.cx(), area.bottom(), &message, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Bottom)); + area.h -= row_height; + return (area, None); + } + area.h -= height; + (area, Some(Layout { top: area.bottom(), row_height, headers, stub, columns })) +} + +pub(super) fn draw(ctx: &mut Ctx, chart: &Chart, data: &ChartData, plot: Bx, layout: Option<&Layout>) { + let Some(layout) = layout else { return }; + let Some(settings) = &chart.data_table else { return }; + let x0 = plot.x - layout.stub; + let top = layout.top; + let rows = layout.headers + data.series.len(); + let bottom = top + layout.row_height * rows as f32; + let cw = plot.w / layout.columns as f32; + // Outermost category level first, then the individual categories. + for (i, level) in data.category_levels.iter().rev().enumerate() { + for (first, end) in level_groups(level, layout.columns) { + let text = level.get(first).map(String::as_str).unwrap_or(""); + let text = ctx.fit(text, LABEL_SIZE, (end - first) as f32 * cw - 4.0); + let x = plot.x + (first + end) as f32 * cw / 2.0; + ctx.text(x, top + (i as f32 + 0.5) * layout.row_height, &text, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Middle)); + } + } + let header_y = top + (layout.headers as f32 - 0.5) * layout.row_height; + let n = data.categories.len().max(data.series.iter().map(|s| s.values.len()).max().unwrap_or(0)); + for j in 0..layout.columns { + let text = if j < n { cat_label(data, j) } else { String::new() }; + let text = ctx.fit(&text, LABEL_SIZE, cw - 4.0); + ctx.text(plot.x + (j as f32 + 0.5) * cw, header_y, &text, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Middle)); + } + for (i, series) in data.series.iter().enumerate() { + let y = top + (layout.headers as f32 + i as f32 + 0.5) * layout.row_height; + let name_x = if settings.show_keys { x0 + 25.0 } else { x0 + 4.0 }; + if settings.show_keys { + let swatch = match plot_kind(chart, series) { + ChartKind::Line | ChartKind::LineStacked => Swatch::Line, + ChartKind::LineMarkers => Swatch::LineMarker, + _ => Swatch::Box, + }; + draw_swatch(ctx, swatch, x0 + 4.0, y, 7.0, series.color); + } + let name = ctx.fit(&series.name, LABEL_SIZE, plot.x - name_x - 4.0); + ctx.text(name_x, y, &name, LABEL_SIZE, TEXT, (HAlign::Left, VAlign::Middle)); + let fmt = Fmt::new(&series.number_format); + for j in 0..layout.columns { + let value = series.values.get(j).copied().flatten().map(|v| fmt.format(v, None)).unwrap_or_default(); + let text = ctx.fit(&value, LABEL_SIZE, cw - 4.0); + ctx.text(plot.x + (j as f32 + 0.5) * cw, y, &text, LABEL_SIZE, TEXT, (HAlign::Center, VAlign::Middle)); + } + } + if settings.horizontal_border { + for row in 1..rows { + ctx.hline(x0, plot.right(), top + row as f32 * layout.row_height, AXIS, 0.75); + } + } + if settings.vertical_border { + ctx.vline(plot.x, top, bottom, AXIS, 0.75); + // The outer headers span groups; category dividers begin at the innermost header. + for j in 1..layout.columns { + ctx.vline(plot.x + j as f32 * cw, top + (layout.headers - 1) as f32 * layout.row_height, bottom, AXIS, 0.75); + } + } + if settings.outline { + ctx.hline(x0, plot.right(), top, AXIS, 0.75); + ctx.hline(x0, plot.right(), bottom, AXIS, 0.75); + ctx.vline(x0, top, bottom, AXIS, 0.75); + ctx.vline(plot.right(), top, bottom, AXIS, 0.75); + } +} diff --git a/crates/chart/src/render/trend.rs b/crates/chart/src/render/trend.rs index 38e435a47..b642df262 100644 --- a/crates/chart/src/render/trend.rs +++ b/crates/chart/src/render/trend.rs @@ -1,6 +1,9 @@ use super::frame::{Bx, value_range}; use super::lines::decimate; -use super::*; +use super::{ + ChartKind, Ctx, HAlign, LABEL_SIZE, LineDash, MAX_CATS, MAX_LINE_PTS, MAX_TRENDLINES, Prim, SeriesData, TEXT, TrendlineKind, VAlign, + fit_trendline, moving_average, +}; // ---------------------------------------------------------------- trendlines @@ -36,10 +39,11 @@ pub(super) fn max_backward(s: &SeriesData) -> f64 { s.trendlines.iter().filter(|t| !matches!(t.spec.kind, TrendlineKind::MovingAverage(_))).map(|t| forecast(t.spec.backward)).fold(0.0, f64::max) } +type ClippedSegment = ((f64, f64), (f64, f64), bool, bool); + /// Liang–Barsky: the part of segment `a`–`b` inside the unit square, with whether the start and /// end were cut. -#[allow(clippy::type_complexity)] -fn clip_segment(a: (f64, f64), b: (f64, f64)) -> Option<((f64, f64), (f64, f64), bool, bool)> { +fn clip_segment(a: (f64, f64), b: (f64, f64)) -> Option { let (dx, dy) = (b.0 - a.0, b.1 - a.1); let (mut t0, mut t1) = (0.0f64, 1.0f64); for (p, q) in [(-dx, a.0), (dx, 1.0 - a.0), (-dy, a.1), (dy, 1.0 - a.1)] { @@ -161,6 +165,6 @@ pub(super) fn draw_trend_labels(ctx: &mut Ctx, plot: Bx, labels: &[String]) { break; } let t = ctx.fit(t, LABEL_SIZE, plot.w - 8.0); - ctx.text(plot.right() - 4.0, y, &t, LABEL_SIZE, TEXT, HAlign::Right, VAlign::Top); + ctx.text(plot.right() - 4.0, y, &t, LABEL_SIZE, TEXT, (HAlign::Right, VAlign::Top)); } } diff --git a/crates/chart/src/tests.rs b/crates/chart/src/tests.rs index 3f1a068a2..46d047afc 100644 --- a/crates/chart/src/tests.rs +++ b/crates/chart/src/tests.rs @@ -5,6 +5,8 @@ use gridcraft_model::{Anchor, Chart, ChartKind, Color, LegendPos, Series, Sparkl use crate::*; +mod data_table; + const ALL_KINDS: [ChartKind; 26] = [ ChartKind::ColumnClustered, ChartKind::ColumnStacked, @@ -43,6 +45,7 @@ fn chart(kind: ChartKind) -> Chart { series: vec![], legend: LegendPos::Bottom, data_labels: true, + data_table: None, gridlines: true, style: 0, x_title: Some("Quarter".into()), diff --git a/crates/chart/src/tests/data_table.rs b/crates/chart/src/tests/data_table.rs new file mode 100644 index 000000000..13e2baf6c --- /dev/null +++ b/crates/chart/src/tests/data_table.rs @@ -0,0 +1,121 @@ +use super::*; +use gridcraft_model::ChartDataTable; + +#[test] +fn tables_draw_source_values_for_every_supported_chart_kind() { + for kind in ALL_KINDS { + let mut c = chart(kind); + c.legend = LegendPos::None; + c.data_labels = false; + c.data_table = Some(ChartDataTable::default()); + let supported = c.supports_data_table(); + let d = sample(kind); + let prims = render(&c, &d, 900.0, 500.0, &ApproxMeasure); + assert_sane(&prims, 900.0, 500.0); + let labels = texts(&prims); + // Source amounts, including on 100% stacked charts, not shares or running totals. + if supported { + for value in ["$12,000", "-$4,000", "$38,000", "$61,870"] { + assert!(labels.contains(&value), "{kind:?}: missing {value}"); + } + } else { + c.data_table = None; + assert_eq!(prims, render(&c, &d, 900.0, 500.0, &ApproxMeasure)); + } + } +} + +#[test] +fn table_columns_align_with_plot_and_keep_missing_values_blank() { + let mut c = chart(ChartKind::ColumnClustered); + c.title = Some(String::new()); + c.x_title = None; + c.y_title = None; + c.legend = LegendPos::None; + c.data_labels = false; + c.data_table = Some(ChartDataTable::default()); + let color = [1, 2, 3, 255]; + let mut s = series("Rate", &[0.25, 0.0, 0.75], color, c.kind); + s.values[1] = None; + s.number_format = "0.0%".into(); + let d = ChartData { categories: ["Jan", "Feb", "Mar"].map(String::from).to_vec(), series: vec![s], ..Default::default() }; + let p = render(&c, &d, 600.0, 400.0, &ApproxMeasure); + let at = |text: &str| -> (f32, f32) { + p.iter() + .find_map(|p| match p { + Prim::Text { text: t, x, y, .. } if t == text => Some((*x, *y)), + _ => None, + }) + .unwrap() + }; + let (x1, y1) = at("25.0%"); + let (x3, y3) = at("75.0%"); + let header_y = at("Jan").1; + assert!(y1 > header_y && (y1 - y3).abs() < 0.01); + assert_eq!(at("Jan").0, x1); + assert_eq!(at("Mar").0, x3); + assert_eq!(texts(&p).iter().filter(|t| **t == "Jan").count(), 1, "the table replaces category-axis labels"); + let centers: Vec = p + .iter() + .filter_map(|p| match p { + Prim::Rect { x, y, w, fill: Some(f), .. } if *f == color && *y < header_y => Some(x + w / 2.0), + _ => None, + }) + .collect(); + assert_eq!(centers.len(), 2); + assert!((centers[0] - x1).abs() < 0.01 && (centers[1] - x3).abs() < 0.01); + let cells: Vec<&str> = p + .iter() + .filter_map(|p| match p { + Prim::Text { y, text, .. } if (*y - y1).abs() < 0.01 => Some(text.as_str()), + _ => None, + }) + .collect(); + assert_eq!(cells, ["Rate", "25.0%", "75.0%"], "blank cells are not zeroes"); + assert!( + p.iter().any(|p| matches!(p, Prim::Rect { y, fill: Some(f), .. } if *f == color && *y > header_y)), + "legend key independent of hidden chart legend" + ); + c.data_table = Some(ChartDataTable { show_keys: false, ..Default::default() }); + let off = render(&c, &d, 600.0, 400.0, &ApproxMeasure); + assert!(!off.iter().any(|p| matches!(p, Prim::Rect { y, fill: Some(f), .. } if *f == color && *y > header_y))); +} + +#[test] +fn table_borders_are_independent_and_multilevel_headers_are_kept() { + let mut c = chart(ChartKind::Line); + c.legend = LegendPos::None; + c.data_labels = false; + let mut d = sample(c.kind); + d.category_levels = vec![vec!["2025".into(), "".into(), "".into(), "2026".into(), "".into(), "".into()]]; + for (h, v, outline, want) in [(false, false, false, 0), (true, false, false, 4), (false, true, false, 6), (false, false, true, 4)] { + c.data_table = Some(ChartDataTable { horizontal_border: h, vertical_border: v, outline, show_keys: false }); + let p = render(&c, &d, 900.0, 600.0, &ApproxMeasure); + let year_y = p + .iter() + .find_map(|p| match p { + Prim::Text { text, y, .. } if text == "2025" => Some(*y), + _ => None, + }) + .unwrap(); + assert!(texts(&p).contains(&"2026")); + let lines = p.iter().filter(|p| matches!(p, Prim::Line { pts, .. } if pts.iter().all(|p| p[1] > year_y - 9.0))).count(); + assert_eq!(lines, want, "{h} {v} {outline}"); + assert_sane(&p, 900.0, 600.0); + } +} + +#[test] +fn dense_and_tiny_tables_are_bounded_and_explicit() { + let mut c = chart(ChartKind::ColumnClustered); + c.data_table = Some(ChartDataTable::default()); + let mut d = sample(c.kind); + d.categories = (0..20_000).map(|i| i.to_string()).collect(); + let p = render(&c, &d, 600.0, 400.0, &ApproxMeasure); + assert!(texts(&p).contains(&"Data table needs more space")); + assert_sane(&p, 600.0, 400.0); + d.categories.clear(); + for (w, h) in [(1.0, 1.0), (10.0, 10.0), (40.0, 35.0), (200.0, 100.0)] { + assert_sane(&render(&c, &d, w, h, &ApproxMeasure), w, h); + } +} diff --git a/crates/core/src/addr.rs b/crates/core/src/addr.rs index fd8ffec11..9d10ea87c 100644 --- a/crates/core/src/addr.rs +++ b/crates/core/src/addr.rs @@ -295,15 +295,16 @@ mod tests { use super::*; #[test] - fn range_parse_rejects_signed_rows() { + fn range_parse_rejects_signed_rows() -> Result<(), std::io::Error> { assert!(RangeRef::parse("+1:+3").is_none()); assert!(RangeRef::parse("-1:3").is_none()); assert!(RangeRef::parse("$$1:3").is_none()); assert!(RangeRef::parse(":3").is_none()); assert!(RangeRef::parse("$$A:$$C").is_none()); assert!(RangeRef::parse("$A:$C").is_some()); - let r = RangeRef::parse("$1:$3").unwrap(); + let r = RangeRef::parse("$1:$3").ok_or_else(|| std::io::Error::other("valid row range"))?; assert_eq!((r.start.row, r.end.row), (0, 2)); + Ok(()) } #[test] @@ -389,18 +390,20 @@ mod tests { } #[test] - fn ranges() { - let r = RangeRef::parse("C3:A1").unwrap(); + fn ranges() -> Result<(), std::io::Error> { + let parse = |s| RangeRef::parse(s).ok_or_else(|| std::io::Error::other("valid range")); + let r = parse("C3:A1")?; assert_eq!(r.a1(), "A1:C3"); assert_eq!(r.count(), 9); - assert_eq!(RangeRef::parse("A:C").unwrap().a1(), "A:C"); - assert_eq!(RangeRef::parse("3:5").unwrap().a1(), "3:5"); + assert_eq!(parse("A:C")?.a1(), "A:C"); + assert_eq!(parse("3:5")?.a1(), "3:5"); assert!(RangeRef::parse("0:5").is_none()); - let a = RangeRef::parse("A1:C3").unwrap(); - let b = RangeRef::parse("B2:D4").unwrap(); - assert_eq!(a.intersection(&b).unwrap().a1(), "B2:C3"); + let a = parse("A1:C3")?; + let b = parse("B2:D4")?; + assert_eq!(a.intersection(&b).ok_or_else(|| std::io::Error::other("overlap expected"))?.a1(), "B2:C3"); assert_eq!(a.union(&b).a1(), "A1:D4"); assert_eq!(a.iter().count(), 9); + Ok(()) } #[test] @@ -458,18 +461,19 @@ mod category_level_tests { use super::*; #[test] - fn a_wide_range_beside_a_column_of_values_is_a_multi_level_axis() { - let r = |s: &str| RangeRef::parse(s).unwrap(); - assert_eq!(category_levels(&r("A4:B39"), Some(&r("C4:C39"))), Some(CategoryLevels { by_columns: true, outer: 1 })); - assert_eq!(category_levels(&r("A4:D39"), Some(&r("C4:C39"))), Some(CategoryLevels { by_columns: true, outer: 3 })); - assert_eq!(category_levels(&r("A4:F39"), Some(&r("C4:C39"))).map(|l| l.outer), Some(3), "capped"); - assert_eq!(category_levels(&r("A4:A39"), Some(&r("C4:C39"))), None, "one column: a plain list"); + fn a_wide_range_beside_a_column_of_values_is_a_multi_level_axis() -> Result<(), std::io::Error> { + let r = |s| RangeRef::parse(s).ok_or_else(|| std::io::Error::other("valid test range")); + assert_eq!(category_levels(&r("A4:B39")?, Some(&r("C4:C39")?)), Some(CategoryLevels { by_columns: true, outer: 1 })); + assert_eq!(category_levels(&r("A4:D39")?, Some(&r("C4:C39")?)), Some(CategoryLevels { by_columns: true, outer: 3 })); + assert_eq!(category_levels(&r("A4:F39")?, Some(&r("C4:C39")?)).map(|l| l.outer), Some(3), "capped"); + assert_eq!(category_levels(&r("A4:A39")?, Some(&r("C4:C39")?)), None, "one column: a plain list"); // A row of values: levels are rows. - assert_eq!(category_levels(&r("B2:M3"), Some(&r("B4:M4"))), Some(CategoryLevels { by_columns: false, outer: 1 })); - assert_eq!(category_levels(&r("B3:M3"), Some(&r("B4:M4"))), None); + assert_eq!(category_levels(&r("B2:M3")?, Some(&r("B4:M4")?)), Some(CategoryLevels { by_columns: false, outer: 1 })); + assert_eq!(category_levels(&r("B3:M3")?, Some(&r("B4:M4")?)), None); // Literal values: the categories' own shape decides. - assert_eq!(category_levels(&r("A1:B6"), None), Some(CategoryLevels { by_columns: true, outer: 1 })); - assert_eq!(category_levels(&r("A1:F2"), None), Some(CategoryLevels { by_columns: false, outer: 1 })); - assert_eq!(category_levels(&r("A:B"), Some(&r("C:C"))), None, "whole columns"); + assert_eq!(category_levels(&r("A1:B6")?, None), Some(CategoryLevels { by_columns: true, outer: 1 })); + assert_eq!(category_levels(&r("A1:F2")?, None), Some(CategoryLevels { by_columns: false, outer: 1 })); + assert_eq!(category_levels(&r("A:B")?, Some(&r("C:C")?)), None, "whole columns"); + Ok(()) } } diff --git a/crates/core/src/date.rs b/crates/core/src/date.rs index 8e68695f1..606d3dacf 100644 --- a/crates/core/src/date.rs +++ b/crates/core/src/date.rs @@ -47,13 +47,11 @@ pub fn days_from_civil(y: i32, m: u32, d: u32) -> i64 { } /// Milliseconds since the Unix epoch (UTC) from the system clock; `None` when the clock is unavailable. -/// On wasm32 `std::time::SystemTime::now` traps, so the browser clock is read through `Date.now()`. -#[allow(clippy::disallowed_methods)] // the std clock is read only off wasm +/// Chrono uses the browser clock on wasm32, where the standard library clock methods trap. pub fn unix_now_ms() -> Option { #[cfg(not(target_arch = "wasm32"))] { - let d = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).ok()?; - Some(d.as_secs_f64() * 1000.0) + Some(chrono::Utc::now().timestamp_millis() as f64) } #[cfg(target_arch = "wasm32")] { @@ -323,7 +321,7 @@ mod tests { } #[test] - fn fictitious_1900_02_29() { + fn fictitious_1900_02_29() -> Result<(), std::io::Error> { // Excel's 1900 calendar has a 29 February 1900 (serial 60); day and month overflow count // through it. let s = DateSystem::D1900; @@ -345,7 +343,7 @@ mod tests { assert_eq!(serial_from_ymd(s, 1904, 2, 29), Some(1521.0)); // Every serial up to 1900-12-31 comes back from its own year, month and day. for serial in 0..=366 { - let d = datetime_from_serial(s, serial as f64).unwrap(); + let d = datetime_from_serial(s, serial as f64).ok_or_else(|| std::io::Error::other("serial should convert"))?; assert_eq!(serial_from_ymd(s, d.year as i64, d.month as i64, d.day as i64), Some(serial as f64), "{d:?}"); } assert_eq!(days_in_month_in(s, 1900, 2), 29); @@ -358,6 +356,7 @@ mod tests { assert_eq!(serial_from_ymd(s, 1904, 3, 1), Some(60.0)); assert_eq!(serial_from_ymd(s, 1904, 1, 0), None); assert_eq!(serial_from_ymd(s, 1900, 2, 29), None); + Ok(()) } #[test] @@ -371,17 +370,18 @@ mod tests { } #[test] - fn roundtrip() { + fn roundtrip() -> Result<(), std::io::Error> { let s = DateSystem::D1900; for serial in [1.0, 59.0, 60.0, 61.0, 36526.0, 46302.0, 2958465.0] { - let d = datetime_from_serial(s, serial).unwrap(); + let d = datetime_from_serial(s, serial).ok_or_else(|| std::io::Error::other("serial should convert"))?; assert_eq!(serial_from_ymd(s, d.year as i64, d.month as i64, d.day as i64), Some(serial)); } - let d = datetime_from_serial(s, 60.0).unwrap(); + let d = datetime_from_serial(s, 60.0).ok_or_else(|| std::io::Error::other("leap-day serial should convert"))?; assert_eq!((d.year, d.month, d.day), (1900, 2, 29)); - let d = datetime_from_serial(s, 46302.75).unwrap(); + let d = datetime_from_serial(s, 46302.75).ok_or_else(|| std::io::Error::other("time serial should convert"))?; assert_eq!((d.hour, d.minute, d.weekday), (18, 0, 3)); // 2026-10-07 is a Wednesday - let d = datetime_from_serial(s, 1.0).unwrap(); + let d = datetime_from_serial(s, 1.0).ok_or_else(|| std::io::Error::other("first serial should convert"))?; assert_eq!(d.weekday, 0); // 1900-01-01 was a Sunday in Excel's calendar + Ok(()) } } diff --git a/crates/core/src/value.rs b/crates/core/src/value.rs index c531de4dc..0000028e9 100644 --- a/crates/core/src/value.rs +++ b/crates/core/src/value.rs @@ -21,10 +21,18 @@ pub enum CellError { Calc, /// Circular reference that could not be resolved (shown as 0 by Excel; we keep it explicit). Circ, + /// A Python cell's code raised an error, or Python stopped. + Python, + /// A Python cell is waiting to run. + Busy, + /// A Python cell ran longer than the time limit. + Timeout, + /// Python can't run here: the workbook's content isn't enabled, or this build has no Python. + Blocked, } impl CellError { - pub const ALL: [CellError; 10] = [ + pub const ALL: [CellError; 14] = [ CellError::Null, CellError::Div0, CellError::Value, @@ -35,6 +43,10 @@ impl CellError { CellError::GettingData, CellError::Spill, CellError::Calc, + CellError::Python, + CellError::Busy, + CellError::Timeout, + CellError::Blocked, ]; pub fn as_str(&self) -> &'static str { match self { @@ -49,9 +61,16 @@ impl CellError { CellError::Spill => "#SPILL!", CellError::Calc => "#CALC!", CellError::Circ => "#CIRC!", + CellError::Python => "#PYTHON!", + CellError::Busy => "#BUSY!", + CellError::Timeout => "#TIMEOUT!", + CellError::Blocked => "#BLOCKED!", } } - /// `ERROR.TYPE` number. + /// `ERROR.TYPE` number. Microsoft documents 1 to 8; 9 (#SPILL!), 11 (#BLOCKED!) and 14 (#CALC!) + /// are the numbers Excel returns. Microsoft documents no number for #PYTHON!, #BUSY! or + /// #TIMEOUT!: 19, 20 and 21 are GridCraft's own choice, after the numbers Excel is known to use, + /// and may change if Excel's numbers become known. pub fn code(&self) -> u32 { match self { CellError::Null => 1, @@ -65,6 +84,10 @@ impl CellError { CellError::Spill => 9, CellError::Calc => 14, CellError::Circ => 15, + CellError::Blocked => 11, + CellError::Python => 19, + CellError::Busy => 20, + CellError::Timeout => 21, } } pub fn parse(s: &str) -> Option { @@ -454,11 +477,12 @@ mod tests { } #[test] - fn arrays() { - let a = Array::new(2, 2, vec![1.0.into(), 2.0.into(), 3.0.into(), 4.0.into()]).unwrap(); + fn arrays() -> Result<(), std::io::Error> { + let a = Array::new(2, 2, vec![1.0.into(), 2.0.into(), 3.0.into(), 4.0.into()]).ok_or_else(|| std::io::Error::other("valid array"))?; assert_eq!(a.transpose().get(0, 1), Some(&Value::Number(3.0))); assert!(Array::new(2, 2, vec![]).is_none()); let row = Array::row(vec![1.0.into(), 2.0.into()]); assert_eq!(row.get_broadcast(5, 1), Value::Number(2.0)); + Ok(()) } } diff --git a/crates/engine/Cargo.toml b/crates/engine/Cargo.toml index e8ea91c26..341d1b35c 100644 --- a/crates/engine/Cargo.toml +++ b/crates/engine/Cargo.toml @@ -17,11 +17,13 @@ gridcraft-model = { workspace = true } gridcraft-calc = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } +web-time = { workspace = true } thiserror = { workspace = true } log = { workspace = true } gridcraft-xlsx = { workspace = true } gridcraft-chart = { workspace = true } gridcraft-pdf = { workspace = true } +gridcraft-math = { workspace = true } image = { workspace = true } [dev-dependencies] diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index 007cbf4bc..45455e00b 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -269,6 +269,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("view.unhideWindow", "View", "Window", "Unhide"), ("view.switchWindows", "View", "Window", "Switch Windows"), ("view.macros", "View", "Macros", "Macros"), + ("macro.record", "View", "Macros", "Record Macro"), // Automate ("automate.newScript", "Automate", "Scripting Tools", "New Script"), ("automate.allScripts", "Automate", "Scripting Tools", "All Scripts"), diff --git a/crates/engine/src/cf.rs b/crates/engine/src/cf.rs index 0c0f9683e..4c9325e36 100644 --- a/crates/engine/src/cf.rs +++ b/crates/engine/src/cf.rs @@ -136,6 +136,95 @@ fn lerp(a: [u8; 3], b: [u8; 3], t: f64) -> [u8; 3] { pub use gridcraft_model::icon_count; +struct RuleEval<'a> { + wb: &'a Workbook, + sheet: usize, + cell: CellRef, + first: CellRef, + value: &'a Value, + text: &'a str, + stats: &'a RuleStats, +} + +fn simple_rule_style<'a>(ctx: &RuleEval<'_>, rule: &'a CfRule) -> Option<&'a Style> { + let RuleEval { wb, sheet, cell, first, value, text, stats } = ctx; + match rule { + CfRule::CellIs { op, a, b, style } => { + let eval = |f: &str| gridcraft_calc::evaluate(wb, *sheet, *cell, &relative(f, *first, *cell)); + let av = eval(a); + let matches = match op { + CfOperator::Between | CfOperator::NotBetween => { + let bv = b.as_deref().map(eval).unwrap_or_default(); + let (lo, hi) = if compare(&av, &bv) == std::cmp::Ordering::Greater { (bv, av) } else { (av, bv) }; + let within = compare(value, &lo) != std::cmp::Ordering::Less && compare(value, &hi) != std::cmp::Ordering::Greater; + if *op == CfOperator::Between { within } else { !within } + } + CfOperator::Equal => compare(value, &av) == std::cmp::Ordering::Equal, + CfOperator::NotEqual => compare(value, &av) != std::cmp::Ordering::Equal, + CfOperator::Greater => compare(value, &av) == std::cmp::Ordering::Greater, + CfOperator::Less => compare(value, &av) == std::cmp::Ordering::Less, + CfOperator::GreaterOrEqual => compare(value, &av) != std::cmp::Ordering::Less, + CfOperator::LessOrEqual => compare(value, &av) != std::cmp::Ordering::Greater, + }; + (matches && !value.is_error()).then_some(style.as_ref()) + } + CfRule::Expression { formula, style } => { + gridcraft_calc::evaluate(wb, *sheet, *cell, &relative(formula, *first, *cell)).to_bool().unwrap_or(false).then_some(style.as_ref()) + } + CfRule::ContainsText { text: needle, style } => { + (!value.is_error() && text.to_lowercase().contains(&needle.to_lowercase())).then_some(style.as_ref()) + } + CfRule::NotContainsText { text: needle, style } => { + (value.is_error() || !text.to_lowercase().contains(&needle.to_lowercase())).then_some(style.as_ref()) + } + CfRule::BeginsWith { text: needle, style } => { + (!value.is_error() && text.to_lowercase().starts_with(&needle.to_lowercase())).then_some(style.as_ref()) + } + CfRule::EndsWith { text: needle, style } => { + (!value.is_error() && text.to_lowercase().ends_with(&needle.to_lowercase())).then_some(style.as_ref()) + } + CfRule::Blanks { style } => (value.is_empty() || text.trim().is_empty()).then_some(style.as_ref()), + CfRule::NoBlanks { style } => (!value.is_empty() && !text.trim().is_empty()).then_some(style.as_ref()), + CfRule::Errors { style } => value.is_error().then_some(style.as_ref()), + CfRule::NoErrors { style } => (!value.is_error()).then_some(style.as_ref()), + CfRule::Duplicate { style } => { + (!value.is_empty() && stats.counts.get(&text.to_lowercase()).copied().unwrap_or(0) > 1).then_some(style.as_ref()) + } + CfRule::Unique { style } => { + (!value.is_empty() && stats.counts.get(&text.to_lowercase()).copied().unwrap_or(0) == 1).then_some(style.as_ref()) + } + _ => None, + } +} + +fn time_period_matches(wb: &Workbook, period: &str, serial: f64) -> bool { + let today = gridcraft_core::date::serial_in_system(wb.date_system, gridcraft_calc::now_serial()).floor(); + let day = serial.floor(); + let date = |serial| gridcraft_core::date::datetime_from_serial(wb.date_system, serial); + let week_start = today - date(today).map(|value| value.weekday as f64).unwrap_or(0.0); + let (year, month) = date(today).map(|value| (value.year, value.month)).unwrap_or((2026, 1)); + let year_month = date(day).map(|value| (value.year, value.month)); + let month_matches = |offset: i32| { + let shifted_month = month as i32 + offset; + let shifted_year = year + (shifted_month - 1).div_euclid(12); + let shifted_month = (shifted_month - 1).rem_euclid(12) + 1; + year_month == Some((shifted_year, shifted_month as u32)) + }; + match period { + "yesterday" => day == today - 1.0, + "today" => day == today, + "tomorrow" => day == today + 1.0, + "last7Days" => day > today - 7.0 && day <= today, + "lastWeek" => day >= week_start - 7.0 && day < week_start, + "thisWeek" => day >= week_start && day < week_start + 7.0, + "nextWeek" => day >= week_start + 7.0 && day < week_start + 14.0, + "lastMonth" => month_matches(-1), + "thisMonth" => month_matches(0), + "nextMonth" => month_matches(1), + _ => false, + } +} + impl CfCache { pub fn new() -> Self { Self::default() @@ -161,52 +250,8 @@ impl CfCache { None => &no_stats, }; let text = v.display(); - let matched_style: Option<&Style> = match &cf.rule { - CfRule::CellIs { op, a, b, style } => { - let eval = |f: &str| gridcraft_calc::evaluate(wb, sheet, c, &relative(f, first, c)); - let av = eval(a); - let ok = match op { - CfOperator::Between | CfOperator::NotBetween => { - let bv = b.as_deref().map(eval).unwrap_or_default(); - let (lo, hi) = if compare(&av, &bv) == std::cmp::Ordering::Greater { (bv, av) } else { (av, bv) }; - let inside = compare(&v, &lo) != std::cmp::Ordering::Less && compare(&v, &hi) != std::cmp::Ordering::Greater; - if *op == CfOperator::Between { inside } else { !inside } - } - CfOperator::Equal => compare(&v, &av) == std::cmp::Ordering::Equal, - CfOperator::NotEqual => compare(&v, &av) != std::cmp::Ordering::Equal, - CfOperator::Greater => compare(&v, &av) == std::cmp::Ordering::Greater, - CfOperator::Less => compare(&v, &av) == std::cmp::Ordering::Less, - CfOperator::GreaterOrEqual => compare(&v, &av) != std::cmp::Ordering::Less, - CfOperator::LessOrEqual => compare(&v, &av) != std::cmp::Ordering::Greater, - }; - // As in Excel, a blank cell compares as 0 (or ""), and an error never matches. - (ok && !v.is_error()).then_some(style.as_ref()) - } - CfRule::Expression { formula, style } => { - gridcraft_calc::evaluate(wb, sheet, c, &relative(formula, first, c)).to_bool().unwrap_or(false).then_some(style.as_ref()) - } - // As in Excel, an error value never contains, begins or ends with anything (its rule - // formulas SEARCH/LEFT/RIGHT fail on it), so it always "does not contain" (#1162). - CfRule::ContainsText { text: t, style } => { - (!v.is_error() && text.to_lowercase().contains(&t.to_lowercase())).then_some(style.as_ref()) - } - CfRule::NotContainsText { text: t, style } => { - (v.is_error() || !text.to_lowercase().contains(&t.to_lowercase())).then_some(style.as_ref()) - } - CfRule::BeginsWith { text: t, style } => { - (!v.is_error() && text.to_lowercase().starts_with(&t.to_lowercase())).then_some(style.as_ref()) - } - CfRule::EndsWith { text: t, style } => (!v.is_error() && text.to_lowercase().ends_with(&t.to_lowercase())).then_some(style.as_ref()), - CfRule::Blanks { style } => (v.is_empty() || text.trim().is_empty()).then_some(style.as_ref()), - CfRule::NoBlanks { style } => (!v.is_empty() && !text.trim().is_empty()).then_some(style.as_ref()), - CfRule::Errors { style } => v.is_error().then_some(style.as_ref()), - CfRule::NoErrors { style } => (!v.is_error()).then_some(style.as_ref()), - CfRule::Duplicate { style } => { - (!v.is_empty() && st.counts.get(&text.to_lowercase()).copied().unwrap_or(0) > 1).then_some(style.as_ref()) - } - CfRule::Unique { style } => { - (!v.is_empty() && st.counts.get(&text.to_lowercase()).copied().unwrap_or(0) == 1).then_some(style.as_ref()) - } + let context = RuleEval { wb, sheet, cell: c, first, value: &v, text: &text, stats: st }; + let matched_style = simple_rule_style(&context, &cf.rule).or_else(|| match &cf.rule { CfRule::Top10 { bottom, percent, rank, style } => match v.as_f64() { Some(n) if !st.sorted.is_empty() => { let k = if *percent { ((st.sorted.len() as f64) * (*rank as f64) / 100.0).floor().max(1.0) as usize } else { *rank as usize }; @@ -229,36 +274,8 @@ impl CfCache { None => None, }, CfRule::TimePeriod { period, style } => match v.as_f64() { - Some(n) => { - let today = gridcraft_core::date::serial_in_system(wb.date_system, gridcraft_calc::now_serial()).floor(); - let d = n.floor(); - let wd = gridcraft_core::date::datetime_from_serial(wb.date_system, today).map(|x| x.weekday as f64).unwrap_or(0.0); - let week_start = today - wd; - let (ty, tm) = - gridcraft_core::date::datetime_from_serial(wb.date_system, today).map(|x| (x.year, x.month)).unwrap_or((2026, 1)); - let ym = gridcraft_core::date::datetime_from_serial(wb.date_system, d).map(|x| (x.year, x.month)); - let month_off = |k: i32| { - let m = tm as i32 + k; - let y = ty + (m - 1).div_euclid(12); - let m = (m - 1).rem_euclid(12) + 1; - ym == Some((y, m as u32)) - }; - let ok = match period.as_str() { - "yesterday" => d == today - 1.0, - "today" => d == today, - "tomorrow" => d == today + 1.0, - "last7Days" => d > today - 7.0 && d <= today, - "lastWeek" => d >= week_start - 7.0 && d < week_start, - "thisWeek" => d >= week_start && d < week_start + 7.0, - "nextWeek" => d >= week_start + 7.0 && d < week_start + 14.0, - "lastMonth" => month_off(-1), - "thisMonth" => month_off(0), - "nextMonth" => month_off(1), - _ => false, - }; - ok.then_some(style.as_ref()) - } - None => None, + Some(n) if time_period_matches(wb, period, n) => Some(style.as_ref()), + _ => None, }, CfRule::ColorScale { stops } => { if let Some(n) = v.as_f64() { @@ -329,7 +346,8 @@ impl CfCache { } None } - }; + _ => None, + }); if let Some(st) = matched_style { any = true; // Every matching rule applies; where two set the same part, the rule with the diff --git a/crates/engine/src/cmd.rs b/crates/engine/src/cmd.rs index ef6714eea..77fbb1d7a 100644 --- a/crates/engine/src/cmd.rs +++ b/crates/engine/src/cmd.rs @@ -5,6 +5,7 @@ pub mod analysis; pub mod data; pub mod draw; pub mod edit; +pub mod equation; pub mod extra; pub mod file; pub mod forecast; @@ -12,8 +13,10 @@ pub mod format; pub mod formulas; pub mod insert; pub mod inspect; +pub mod macros; pub mod pivot; pub mod print; +pub mod python; pub mod review; pub mod sheet; pub mod slicer; @@ -39,7 +42,11 @@ pub struct CommandSpec { pub label: &'static str, /// Ribbon/menu placement, e.g. `["Home", "Font"]`. Empty = not in menus. pub menu: &'static [&'static str], + /// Portable notation: `Cmd` is ⌘ on macOS and Ctrl elsewhere. pub shortcut: Option<&'static str>, + /// macOS override when its key differs. Equal to `shortcut` marks a Mac-only shortcut + /// (Align Left/Center): other platforms show none. See [`CommandInfo::shortcut_for_platform`]. + pub shortcut_mac: Option<&'static str>, pub params: &'static str, pub enabled: Enabled, pub run: Run, @@ -53,12 +60,32 @@ pub struct CommandInfo { pub label: &'static str, pub menu: Vec<&'static str>, pub shortcut: Option<&'static str>, + pub shortcut_mac: Option<&'static str>, pub params: &'static str, pub enabled: bool, #[serde(skip_serializing_if = "Option::is_none")] pub disabled_reason: Option, } +impl CommandInfo { + /// Format the portable shortcut notation for the runtime operating system. + pub fn shortcut_for_platform(&self, mac: bool) -> Option { + let keys = if mac { + self.shortcut_mac.or(self.shortcut) + } else if self.shortcut_mac.is_some() && self.shortcut_mac == self.shortcut { + // An identical Mac override marks a Mac-only display shortcut. + // The portable field remains unchanged for legacy CLI consumers. + None + } else { + self.shortcut + }?; + if mac { + return Some(keys.to_owned()); + } + Some(keys.split('+').map(|part| if part == "Cmd" { "Ctrl" } else { part }).collect::>().join("+")) + } +} + impl CommandSpec { pub fn info(&self, s: &Session) -> CommandInfo { let e = (self.enabled)(s); @@ -67,6 +94,7 @@ impl CommandSpec { label: self.label, menu: self.menu.to_vec(), shortcut: self.shortcut, + shortcut_mac: self.shortcut_mac, params: self.params, enabled: e.is_ok(), disabled_reason: e.err(), @@ -117,13 +145,13 @@ pub fn has_clipboard(s: &Session) -> std::result::Result<(), String> { macro_rules! cmd { ($id:literal, $label:literal, [$($m:literal),*], $sc:expr, $params:literal, $en:expr, $run:expr) => { - $crate::cmd::CommandSpec { id: $id, label: $label, menu: &[$($m),*], shortcut: $sc, params: $params, enabled: $en, run: $run, journal: true, undoable: true } + $crate::cmd::CommandSpec { id: $id, label: $label, menu: &[$($m),*], shortcut: $sc, shortcut_mac: None, params: $params, enabled: $en, run: $run, journal: true, undoable: true } }; (query $id:literal, $label:literal, [$($m:literal),*], $sc:expr, $params:literal, $en:expr, $run:expr) => { - $crate::cmd::CommandSpec { id: $id, label: $label, menu: &[$($m),*], shortcut: $sc, params: $params, enabled: $en, run: $run, journal: false, undoable: false } + $crate::cmd::CommandSpec { id: $id, label: $label, menu: &[$($m),*], shortcut: $sc, shortcut_mac: None, params: $params, enabled: $en, run: $run, journal: false, undoable: false } }; (noundo $id:literal, $label:literal, [$($m:literal),*], $sc:expr, $params:literal, $en:expr, $run:expr) => { - $crate::cmd::CommandSpec { id: $id, label: $label, menu: &[$($m),*], shortcut: $sc, params: $params, enabled: $en, run: $run, journal: true, undoable: false } + $crate::cmd::CommandSpec { id: $id, label: $label, menu: &[$($m),*], shortcut: $sc, shortcut_mac: None, params: $params, enabled: $en, run: $run, journal: true, undoable: false } }; } pub(crate) use cmd; @@ -137,16 +165,19 @@ pub fn command_specs() -> &'static [CommandSpec] { v.extend(format::specs()); v.extend(sheet::specs()); v.extend(insert::specs()); + v.extend(equation::specs()); v.extend(data::specs()); v.extend(formulas::specs()); v.extend(review::specs()); v.extend(view::specs()); v.extend(inspect::specs()); v.extend(extra::specs()); + v.extend(macros::specs()); v.extend(draw::specs()); v.extend(spelling::specs()); v.extend(solver::specs()); v.extend(pivot::specs()); + v.extend(python::specs()); v.extend(slicer::specs()); v.extend(whatif::specs()); v.extend(forecast::specs()); @@ -178,6 +209,7 @@ pub fn register_test_command(id: &'static str, run: Run) { label: id, menu: &[], shortcut: None, + shortcut_mac: None, params: "", enabled: |_| Ok(()), run, @@ -195,6 +227,7 @@ pub(crate) fn register_undoable_test_command(id: &'static str, run: Run) { label: id, menu: &[], shortcut: None, + shortcut_mac: None, params: "", enabled: |_| Ok(()), run, @@ -205,7 +238,6 @@ pub(crate) fn register_undoable_test_command(id: &'static str, run: Run) { } fn test_commands() -> std::sync::MutexGuard<'static, Vec<&'static CommandSpec>> { - #[allow(clippy::unwrap_used)] // a poisoned test registry has no meaningful recovery match TEST_COMMANDS.lock() { Ok(guard) => guard, Err(poisoned) => poisoned.into_inner(), @@ -338,6 +370,9 @@ pub struct Ctx<'a> { /// The command applied sheet protection itself (row/column inserts and deletes the /// protection allows), so cells it shifts aren't refused as edits of locked cells. pub protection_checked: bool, + /// The structural edits the command made, in order (the sheet's name, the edit), as + /// references were rewritten for them. + pub edits: Vec<(String, gridcraft_formula::adjust::Edit)>, } pub(crate) fn edit(s: &mut Session, f: impl FnOnce(&mut Ctx) -> Result) -> Result { @@ -349,26 +384,51 @@ pub(crate) fn edit(s: &mut Session, f: impl FnOnce(&mut Ctx) -> Result) -> }) } +fn wall_clock() -> std::time::Duration { + std::time::SystemTime::UNIX_EPOCH.elapsed().unwrap_or_default() +} + pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) -> Result { - // `GRIDCRAFT_PROFILE` timing. Read the clock only when profiling, and never on wasm: there - // `Instant::now` traps (`RuntimeError: unreachable`), which broke every edit in the web app. - #[allow(clippy::disallowed_methods)] - let t0 = if cfg!(target_arch = "wasm32") { None } else { std::env::var_os("GRIDCRAFT_PROFILE").map(|_| std::time::Instant::now()) }; + // `GRIDCRAFT_PROFILE` timing, never on wasm where the clock traps. `elapsed` on the epoch + // reads the wall clock without the (disallowed) `now`. + let t0 = if cfg!(target_arch = "wasm32") { None } else { std::env::var_os("GRIDCRAFT_PROFILE").map(|_| wall_clock()) }; + let since = |t0: std::time::Duration| wall_clock().saturating_sub(t0); let mut sel = d.selection.clone(); - let mut ctx = Ctx { wb: (*d.wb).clone(), changed: Vec::new(), structural: false, sel: &mut sel, fit_rows: Vec::new(), protection_checked: false }; + let mut ctx = Ctx { + wb: (*d.wb).clone(), + changed: Vec::new(), + structural: false, + sel: &mut sel, + fit_rows: Vec::new(), + protection_checked: false, + edits: Vec::new(), + }; let r = f(&mut ctx)?; - let Ctx { mut wb, changed, structural, mut fit_rows, protection_checked, .. } = ctx; + let Ctx { mut wb, changed, structural, mut fit_rows, protection_checked, edits, .. } = ctx; + // Equations whose source changed (or that arrived without a layout, as from a paste) are + // laid out here, so painting only ever reads the cache. + wb.refresh_equations(); if !protection_checked { check_protection(&d.wb, &wb, &d.unlocked_ranges)?; } if !structural { check_arrays(&d.wb, &wb)?; } + if let Some(log) = d.touched.as_mut() { + for (si, c) in changed.iter().take(TOUCH_LOG_LIMIT.saturating_sub(log.len())) { + if let Some(sh) = wb.sheet(*si) { + log.push((sh.name.clone(), *c)); + } + } + } + if let Some(log) = d.edits.as_mut() { + log.extend(edits); + } if changed.len() <= 200_000 { fit_rows.extend(changed.iter().map(|(s, c)| (*s, c.row))); } if let Some(t0) = t0 { - eprintln!("commit: edit {:?}, {} changed", t0.elapsed(), changed.len()); + eprintln!("commit: edit {:?}, {} changed", since(t0), changed.len()); } if structural { d.calc.recalc_all(&mut wb); @@ -376,7 +436,7 @@ pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) d.calc.cells_changed(&mut wb, &changed); } if let Some(t0) = t0 { - eprintln!("commit: + recalc {:?}", t0.elapsed()); + eprintln!("commit: + recalc {:?}", since(t0)); } fit_rows.sort_unstable(); fit_rows.dedup(); @@ -384,13 +444,17 @@ pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) auto_row_height(&mut wb, si, row); } if let Some(t0) = t0 { - eprintln!("commit: + rows {:?}", t0.elapsed()); + eprintln!("commit: + rows {:?}", since(t0)); } d.wb = std::sync::Arc::new(wb); d.selection = sel; Ok(r) } +/// The most cells a dry run's touch log keeps; a footprint whose log is full may be missing writes +/// that kept a cell's value, so a shift is refused then. +pub(crate) const TOUCH_LOG_LIMIT: usize = 1_000_000; + /// Excel's message for an edit of a locked cell on a protected sheet. pub(crate) const PROTECTED: &str = "The cell or chart you're trying to change is on a protected sheet."; @@ -460,7 +524,7 @@ fn check_arrays(old: &Workbook, new: &Workbook) -> Result<()> { continue; } for c in sh.cells.diff(&after.cells) { - let filled = after.cell(c).is_some_and(|x| x.formula.is_some() || !x.value.is_empty()); + let filled = after.cell(c).is_some_and(gridcraft_model::Cell::has_content); if !filled || !sh.spill.contains_key(&c) || same_content(sh.cell(c), after.cell(c)) { continue; } @@ -476,13 +540,18 @@ fn check_arrays(old: &Workbook, new: &Workbook) -> Result<()> { Ok(()) } -/// Two cells hold the same value or formula (formats aside). +/// Two cells hold the same value, formula or Python code (formats aside; a Python cell's result +/// is not something the person edits). fn same_content(a: Option<&gridcraft_model::Cell>, b: Option<&gridcraft_model::Cell>) -> bool { - let empty = |c: Option<&gridcraft_model::Cell>| c.is_none_or(|c| c.formula.is_none() && c.value.is_empty()); + let empty = |c: Option<&gridcraft_model::Cell>| c.is_none_or(|c| !c.has_content()); match (a, b) { (Some(a), Some(b)) => match (&a.formula, &b.formula) { (Some(x), Some(y)) => x.text == y.text && x.array == y.array, - (None, None) => a.value == b.value, + (None, None) => match (&a.py, &b.py) { + (Some(x), Some(y)) => x.code == y.code && x.object == y.object, + (None, None) => a.value == b.value, + _ => false, + }, _ => false, }, _ => empty(a) && empty(b), diff --git a/crates/engine/src/cmd/analysis.rs b/crates/engine/src/cmd/analysis.rs index 6225b4934..9cd10e50f 100644 --- a/crates/engine/src/cmd/analysis.rs +++ b/crates/engine/src/cmd/analysis.rs @@ -14,7 +14,7 @@ use gridcraft_model::{Cell, NumFmt, Sheet}; use serde_json::{Value as Json, json}; use super::whatif::parse_range; -use super::*; +use super::{CommandSpec, EngineError, Result, Session, bad, cmd, edit, has_doc, str_param}; const CMD: &str = "data.analysisToolPak"; /// Most input cells a tool reads. @@ -27,6 +27,14 @@ const MAX_MATRIX_VARS: usize = 256; const MAX_REGRESSORS: usize = 16; /// Most histogram bins. const MAX_BINS: usize = 100_000; +/// Excel's limit on Fourier Analysis input values. +const MAX_FOURIER: usize = 4096; +/// Most numbers Random Number Generation or Sampling writes in one run. +const MAX_RANDOM: usize = 1_000_000; +/// Most trials of a binomial draw, and the farthest Poisson outcome considered. +const MAX_OUTCOMES: f64 = 1_000_000.0; +/// Largest seed: seeds are whole numbers that a JSON number holds exactly. +const MAX_SEED: u64 = (1 << 53) - 1; /// Part of the error a programmatic call gets when the output range already holds data; the /// dialog recognises it and asks before running again with `overwrite: true`. @@ -35,18 +43,24 @@ pub const OVERWRITE_ERROR: &str = "already holds data"; /// (id, label) of every tool, in the order of Excel's Analysis Tools list. pub const TOOLS: &[(&str, &str)] = &[ ("anovaSingle", "Anova: Single Factor"), + ("anovaTwoFactorWith", "Anova: Two-Factor With Replication"), + ("anovaTwoFactorWithout", "Anova: Two-Factor Without Replication"), ("correlation", "Correlation"), ("covariance", "Covariance"), ("descriptive", "Descriptive Statistics"), ("exponentialSmoothing", "Exponential Smoothing"), ("fTest", "F-Test Two-Sample for Variances"), + ("fourierAnalysis", "Fourier Analysis"), ("histogram", "Histogram"), ("movingAverage", "Moving Average"), + ("randomNumberGeneration", "Random Number Generation"), ("rankPercentile", "Rank and Percentile"), ("regression", "Regression"), + ("sampling", "Sampling"), ("tTestPaired", "t-Test: Paired Two Sample for Means"), ("tTestEqual", "t-Test: Two-Sample Assuming Equal Variances"), ("tTestUnequal", "t-Test: Two-Sample Assuming Unequal Variances"), + ("zTest", "z-Test: Two Sample for Means"), ]; pub fn specs() -> Vec { @@ -55,7 +69,7 @@ pub fn specs() -> Vec { "Data Analysis…", ["Data", "Analysis"], None, - "{tool: anovaSingle|correlation|covariance|descriptive|exponentialSmoothing|fTest|histogram|movingAverage|rankPercentile|regression|tTestPaired|tTestEqual|tTestUnequal (or the tool's label), output?: \"E1\" (top-left of the report; default a new sheet), newSheet?: name, outputTo?: range|newSheet (picks between the two), overwrite?: bool (else non-empty output cells are refused), labels?: bool (first row/column holds names), groupedBy?: columns|rows, …tool params} — range tools take `range`; descriptive: summary? (default true), confidence?: 95, kthLargest?, kthSmallest?; histogram: bins?: range (default evenly spaced), pareto?, cumulative?; regression: inputY, inputX, constantZero?, confidence?: 95, residuals?, standardizedResiduals?, normalProbability?; movingAverage: interval? (3), standardErrors?; exponentialSmoothing: damping? (0.3), standardErrors?; t-tests and fTest: range1, range2, alpha? (0.05), hypothesizedDifference? (0); anovaSingle: alpha?. Writes static values. No `tool` → {tools} (the menu opens the dialog).", + "{tool: anovaSingle|anovaTwoFactorWith|anovaTwoFactorWithout|correlation|covariance|descriptive|exponentialSmoothing|fTest|fourierAnalysis|histogram|movingAverage|randomNumberGeneration|rankPercentile|regression|sampling|tTestPaired|tTestEqual|tTestUnequal|zTest (or the tool's label), output?: \"E1\" (top-left of the report; default a new sheet), newSheet?: name, outputTo?: range|newSheet (picks between the two), overwrite?: bool (else non-empty output cells are refused), labels?: bool (first row/column holds names), groupedBy?: columns|rows, …tool params} — range tools take `range`; descriptive: summary? (default true), confidence?: 95, kthLargest?, kthSmallest?; histogram: bins?: range (default evenly spaced), pareto?, cumulative?; regression: inputY, inputX, constantZero?, confidence?: 95, residuals?, standardizedResiduals?, normalProbability?; movingAverage: interval? (3), standardErrors?; exponentialSmoothing: damping? (0.3), standardErrors?; t-tests and fTest: range1, range2, alpha? (0.05), hypothesizedDifference? (0); anovaSingle: alpha?; anovaTwoFactorWithout: range, labels? (the first row holds column names and the first column row names), alpha?; anovaTwoFactorWith: range (first row and column hold labels), rowsPerSample, alpha?; zTest: range1, range2, variance1, variance2 (known variances), alpha?, hypothesizedDifference?; fourierAnalysis: range (one row or column of 2^k numbers or complex text, at most 4096), inverse?; randomNumberGeneration: variables? (1), count (per variable; not for patterned), distribution: uniform{from, to} | normal{mean, standardDeviation} | bernoulli{p} | binomial{p, trials} | poisson{lambda} | patterned{from, to, step, repeatEach, repeatSequence} | discrete{valueRange: two columns of value and probability}, seed? (whole number, not for patterned; the same seed repeats the numbers; the result reports the seed used), at most 1,000,000 numbers; sampling: range, method?: periodic|random, period (periodic), samples (random, with replacement), seed?. Writes static values. No `tool` → {tools} (the menu opens the dialog).", has_doc, run )] @@ -107,7 +121,7 @@ fn confidence(p: &Json, key: &str) -> Result> { } fn alpha(p: &Json) -> Result { - match num(p, "alpha") { + match opt_num(p, "alpha")? { None => Ok(0.05), Some(a) if a > 0.0 && a < 1.0 => Ok(a), Some(_) => Err(bad(CMD, "alpha must be between 0 and 1")), @@ -1050,6 +1064,684 @@ fn anova_single(s: &Session, p: &Json) -> Result { Ok(g) } +// ---------------------------------------------------------------- two-factor anova and z-test + +/// Count, Sum, Average and Variance of a group, as the Anova summaries list them. +fn group_summary(x: &[f64]) -> [Value; 4] { + [Value::number(x.len() as f64), Value::number(x.iter().sum()), n_or_div0(mean(x)), n_or_div0(var_s(x))] +} + +const ANOVA_HEAD: [&str; 7] = ["Source of Variation", "SS", "df", "MS", "F", "P-value", "F crit"]; + +/// One line of an Anova table. A factor line, tested against the error term `err` (its MS and +/// df), adds F, the P-value and the critical F. +fn anova_line(g: &mut Grid, row: u32, label: &str, ss: f64, df: f64, err: Option<(f64, f64)>, alpha: f64) { + g.txt(row, 0, label); + g.num(row, 1, ss); + g.num(row, 2, df); + g.put(row, 3, ratio(ss, df)); + if let Some((mse, dfe)) = err { + let f = ss / df / mse; + g.put(row, 4, n_or_div0(f)); + g.put(row, 5, if f.is_finite() && dfe > 0.0 { callf("F.DIST.RT", &[f, df, dfe]) } else { div0() }); + g.put(row, 6, if df > 0.0 && dfe > 0.0 { callf("F.INV.RT", &[alpha, df, dfe]) } else { div0() }); + } +} + +/// The cells of a two-factor input range, row by row. +fn input_rows(s: &Session, p: &Json) -> Result>> { + let (si, r) = input_range(s, p, &["range", "input"])?; + Ok(variables(s, si, r, true, false, &|_| String::new())?.into_iter().map(|v| v.vals).collect()) +} + +/// The values of column `j` of a table of numbers. +fn column(data: &[Vec], j: usize) -> Vec { + data.iter().filter_map(|r| r.get(j).copied()).collect() +} + +/// Anova: Two-Factor Without Replication: one observation for each pair of levels (rows × +/// columns), the error term being the residual after both main effects. +fn anova_two_factor_without(s: &Session, p: &Json) -> Result { + let alpha = alpha(p)?; + let labels = flag(p, "labels"); + let mut rows = input_rows(s, p)?; + let head = if labels && !rows.is_empty() { Some(rows.remove(0)) } else { None }; + let mut row_names = Vec::with_capacity(rows.len()); + let mut data: Vec> = Vec::with_capacity(rows.len()); + for (i, mut row) in rows.into_iter().enumerate() { + let name = if labels && !row.is_empty() { row.remove(0).display() } else { format!("Row {}", i + 1) }; + row_names.push(name); + data.push(numbers(&row, false)?); + } + let (a, b) = (data.len(), data.first().map_or(0, Vec::len)); + if a < 2 || b < 2 { + return Err(EngineError::Other("Anova needs at least two rows and two columns of numbers.".into())); + } + let col_names: Vec = match head { + Some(h) => h.iter().skip(1).map(Value::display).collect(), + None => (1..=b).map(|j| format!("Column {j}")).collect(), + }; + let cols: Vec> = (0..b).map(|j| column(&data, j)).collect(); + let all: Vec = data.iter().flatten().copied().collect(); + let grand = mean(&all); + let row_means: Vec = data.iter().map(|r| mean(r)).collect(); + let col_means: Vec = cols.iter().map(|c| mean(c)).collect(); + let ss_rows: f64 = row_means.iter().map(|m| b as f64 * (m - grand).powi(2)).sum(); + let ss_cols: f64 = col_means.iter().map(|m| a as f64 * (m - grand).powi(2)).sum(); + // The residual of each observation after removing both main effects. + let ss_err: f64 = + data.iter().zip(&row_means).flat_map(|(r, rm)| r.iter().zip(&col_means).map(move |(x, cm)| (x - rm - cm + grand).powi(2))).sum(); + let (df_rows, df_cols) = ((a - 1) as f64, (b - 1) as f64); + let df_err = df_rows * df_cols; + let mut g = Grid::default(); + g.txt(0, 0, "Anova: Two-Factor Without Replication"); + g.labels(2, 0, &["SUMMARY", "Count", "Sum", "Average", "Variance"]); + let summary = |g: &mut Grid, row: u32, name: &str, x: &[f64]| { + g.txt(row, 0, name); + for (k, v) in group_summary(x).into_iter().enumerate() { + g.put(row, 1 + idx(k), v); + } + }; + for (i, (name, x)) in row_names.iter().zip(&data).enumerate() { + summary(&mut g, 3 + idx(i), name, x); + } + let first_col = 3 + idx(a) + 1; + for (j, (name, x)) in col_names.iter().zip(&cols).enumerate() { + summary(&mut g, first_col + idx(j), name, x); + } + let row = first_col + idx(b) + 2; + g.txt(row, 0, "ANOVA"); + g.labels(row + 1, 0, &ANOVA_HEAD); + let err = Some((ss_err / df_err, df_err)); + anova_line(&mut g, row + 2, "Rows", ss_rows, df_rows, err, alpha); + anova_line(&mut g, row + 3, "Columns", ss_cols, df_cols, err, alpha); + anova_line(&mut g, row + 4, "Error", ss_err, df_err, None, alpha); + g.txt(row + 6, 0, "Total"); + g.num(row + 6, 1, ss(&all)); + g.num(row + 6, 2, (all.len() - 1) as f64); + Ok(g) +} + +/// The number of rows in each sample of a two-factor Anova with replication. +fn rows_per_sample(p: &Json, data_rows: usize) -> Result { + let per = num(p, "rowsPerSample").ok_or_else(|| bad(CMD, "missing `rowsPerSample` (the number of rows in each sample)"))?; + if per < 2.0 || per.fract() != 0.0 || per > data_rows as f64 || !data_rows.is_multiple_of(per as usize) { + return Err(EngineError::Other("Rows per sample must be a whole number of at least 2 that divides the number of data rows.".into())); + } + Ok(per as usize) +} + +/// Anova: Two-Factor With Replication. The input range holds column labels in its first row and +/// sample labels in its first column; each sample (a level of the row factor) spans +/// `rowsPerSample` consecutive rows, so every cell of the design holds that many observations. +fn anova_two_factor_with(s: &Session, p: &Json) -> Result { + let alpha = alpha(p)?; + let mut rows = input_rows(s, p)?; + if rows.len() < 3 { + return Err(EngineError::Other("The input range must hold a row of labels and at least two samples.".into())); + } + let head = rows.remove(0); + let col_names: Vec = head.iter().skip(1).map(Value::display).collect(); + let per = rows_per_sample(p, rows.len())?; + let mut sample_names = Vec::new(); + let mut data: Vec> = Vec::with_capacity(rows.len()); + for (i, mut row) in rows.into_iter().enumerate() { + let name = if row.is_empty() { Value::Empty } else { row.remove(0) }; + if i.is_multiple_of(per) { + sample_names.push(name.display()); + } + data.push(numbers(&row, false)?); + } + let (a, b) = (sample_names.len(), col_names.len()); + if a < 2 || b < 2 { + return Err(EngineError::Other("Anova needs at least two samples and two columns of numbers.".into())); + } + // cells[i][j]: the observations of sample i in column j. + let cells: Vec>> = data.chunks(per).map(|block| (0..b).map(|j| column(block, j)).collect()).collect(); + let all: Vec = data.iter().flatten().copied().collect(); + let grand = mean(&all); + let sample_all: Vec> = data.chunks(per).map(|block| block.iter().flatten().copied().collect()).collect(); + let cols: Vec> = (0..b).map(|j| column(&data, j)).collect(); + let (sm, cm): (Vec, Vec) = (sample_all.iter().map(|x| mean(x)).collect(), cols.iter().map(|x| mean(x)).collect()); + let n = per as f64; + let ss_sample: f64 = sm.iter().map(|m| n * b as f64 * (m - grand).powi(2)).sum(); + let ss_cols: f64 = cm.iter().map(|m| n * a as f64 * (m - grand).powi(2)).sum(); + let mut ss_inter = 0.0; + let mut ss_within = 0.0; + for (i, row) in cells.iter().enumerate() { + for (j, cell) in row.iter().enumerate() { + let (si, cj) = (sm.get(i).copied().unwrap_or(f64::NAN), cm.get(j).copied().unwrap_or(f64::NAN)); + ss_inter += n * (mean(cell) - si - cj + grand).powi(2); + ss_within += ss(cell); + } + } + let (df_sample, df_cols) = ((a - 1) as f64, (b - 1) as f64); + let df_within = (a * b * (per - 1)) as f64; + let mut g = Grid::default(); + g.txt(0, 0, "Anova: Two-Factor With Replication"); + g.txt(2, 0, "SUMMARY"); + for (j, name) in col_names.iter().enumerate() { + g.txt(2, 1 + idx(j), name); + } + g.txt(2, 1 + idx(b), "Total"); + let stats = ["Count", "Sum", "Average", "Variance"]; + let mut row = 3u32; + let block = |g: &mut Grid, row: &mut u32, name: &str, groups: Vec<&[f64]>| { + g.txt(*row, 0, name); + let sums: Vec<[Value; 4]> = groups.iter().map(|x| group_summary(x)).collect(); + for (k, label) in stats.iter().enumerate() { + let r = *row + 1 + idx(k); + g.txt(r, 0, label); + for (j, sm) in sums.iter().enumerate() { + g.put(r, 1 + idx(j), sm.get(k).cloned().unwrap_or(Value::Empty)); + } + } + *row += 6; + }; + for ((name, row_cells), whole) in sample_names.iter().zip(&cells).zip(&sample_all) { + let mut groups: Vec<&[f64]> = row_cells.iter().map(Vec::as_slice).collect(); + groups.push(whole); + block(&mut g, &mut row, name, groups); + } + block(&mut g, &mut row, "Total", cols.iter().map(Vec::as_slice).collect()); + row += 1; + g.txt(row, 0, "ANOVA"); + g.labels(row + 1, 0, &ANOVA_HEAD); + let err = Some((ss_within / df_within, df_within)); + anova_line(&mut g, row + 2, "Sample", ss_sample, df_sample, err, alpha); + anova_line(&mut g, row + 3, "Columns", ss_cols, df_cols, err, alpha); + anova_line(&mut g, row + 4, "Interaction", ss_inter, df_sample * df_cols, err, alpha); + anova_line(&mut g, row + 5, "Within", ss_within, df_within, None, alpha); + g.txt(row + 7, 0, "Total"); + g.num(row + 7, 1, ss(&all)); + g.num(row + 7, 2, (all.len() - 1) as f64); + Ok(g) +} + +/// A known variance parameter: a positive number. +fn known_variance(p: &Json, key: &str) -> Result { + match num(p, key) { + Some(v) if v > 0.0 => Ok(v), + _ => Err(EngineError::Other(format!("Enter the known variance `{key}` (a positive number)."))), + } +} + +/// z-Test: Two Sample for Means, for two samples whose population variances are known. +fn z_test(s: &Session, p: &Json) -> Result { + let alpha = alpha(p)?; + let hmd = match opt_num(p, "hypothesizedDifference")? { + Some(d) => d, + None => num_or(p, "difference", 0.0)?, + }; + let (v1, v2) = (known_variance(p, "variance1")?, known_variance(p, "variance2")?); + let a = single(s, p, "range1", "Variable 1")?; + let b = single(s, p, "range2", "Variable 2")?; + let (x, y) = (numbers(&a.vals, true)?, numbers(&b.vals, true)?); + if x.is_empty() || y.is_empty() { + return Err(EngineError::Other("Each input range needs at least one number.".into())); + } + let (n1, n2) = (x.len() as f64, y.len() as f64); + let (m1, m2) = (mean(&x), mean(&y)); + let z = (m1 - m2 - hmd) / (v1 / n1 + v2 / n2).sqrt(); + let mut g = Grid::default(); + let row = test_head(&mut g, "z-Test: Two Sample for Means", &a.name, &b.name); + pair_row(&mut g, row, "Mean", Value::number(m1), Value::number(m2)); + pair_row(&mut g, row + 1, "Known Variance", Value::number(v1), Value::number(v2)); + pair_row(&mut g, row + 2, "Observations", Value::number(n1), Value::number(n2)); + pair_row(&mut g, row + 3, "Hypothesized Mean Difference", Value::number(hmd), Value::Empty); + pair_row(&mut g, row + 4, "z", n_or_div0(z), Value::Empty); + // The tail beyond |z|: 1 − Φ(|z|). + let tail = call("NORM.S.DIST", &[Value::Number(z.abs()), Value::Bool(true)]).as_f64().map(|c| 1.0 - c); + let lines: [(&str, Value); 4] = [ + ("P(Z<=z) one-tail", tail.map_or_else(div0, Value::number)), + ("z Critical one-tail", callf("NORM.S.INV", &[1.0 - alpha])), + ("P(Z<=z) two-tail", tail.map_or_else(div0, |t| Value::number(2.0 * t))), + ("z Critical two-tail", callf("NORM.S.INV", &[1.0 - alpha / 2.0])), + ]; + for (k, (label, v)) in lines.into_iter().enumerate() { + pair_row(&mut g, row + 5 + idx(k), label, v, Value::Empty); + } + Ok(g) +} + +// ---------------------------------------------------------------- fourier analysis + +/// Radix-2 fast Fourier transform (iterative Cooley–Tukey) in place, for a power-of-two number of +/// complex values (re, im). The forward transform is unscaled, X[k] = Σ x[n]·e^(−2πikn/N); the +/// inverse uses e^(+2πikn/N) and divides by N, so it returns the values the forward one took. +fn fft(a: &mut [(f64, f64)], inverse: bool) { + let n = a.len(); + if n < 2 || !n.is_power_of_two() { + return; + } + let bits = n.trailing_zeros(); + for i in 0..n { + let j = i.reverse_bits() >> (usize::BITS - bits); + if i < j { + a.swap(i, j); + } + } + let sign = if inverse { 1.0 } else { -1.0 }; + let mut len = 2; + while len <= n { + let ang = sign * 2.0 * std::f64::consts::PI / len as f64; + for start in (0..n).step_by(len) { + for k in 0..len / 2 { + // Each twiddle factor is computed directly, not by repeated multiplication. + let (wr, wi) = ((ang * k as f64).cos(), (ang * k as f64).sin()); + let (i, j) = (start + k, start + k + len / 2); + let (Some(&(ur, ui)), Some(&(xr, xi))) = (a.get(i), a.get(j)) else { continue }; + let (vr, vi) = (xr * wr - xi * wi, xr * wi + xi * wr); + if let Some(u) = a.get_mut(i) { + *u = (ur + vr, ui + vi); + } + if let Some(x) = a.get_mut(j) { + *x = (ur - vr, ui - vi); + } + } + } + len <<= 1; + } + if inverse { + let nf = n as f64; + a.iter_mut().for_each(|v| *v = (v.0 / nf, v.1 / nf)); + } +} + +/// Fourier Analysis: the discrete Fourier transform (or with `inverse` its inverse) of one row or +/// column of 2^k values, read and written as complex numbers the way IMREAL, IMAGINARY and +/// COMPLEX read and write them. +fn fourier_analysis(s: &Session, p: &Json) -> Result { + let v = single(s, p, "range", "")?; + let n = v.vals.len(); + if n == 0 || n > MAX_FOURIER || !n.is_power_of_two() { + return Err(EngineError::Other(format!("Fourier Analysis needs 1, 2, 4, 8 … {MAX_FOURIER} input values (a power of two)."))); + } + let mut a = Vec::with_capacity(n); + for x in &v.vals { + let part = |f: &str| call(f, std::slice::from_ref(x)).as_f64(); + match (x, part("IMREAL"), part("IMAGINARY")) { + (Value::Empty, ..) => return Err(non_numeric()), + (_, Some(re), Some(im)) => a.push((re, im)), + _ => return Err(EngineError::Other("Input range must hold numbers or complex numbers such as 3+4i.".into())), + } + } + let total: f64 = a.iter().map(|(re, im)| re.abs() + im.abs()).sum(); + let inverse = flag(p, "inverse"); + fft(&mut a, inverse); + // Parts that can only be rounding noise of the transform are written as 0. Each of the + // log2(N) butterfly stages adds at most a few units of f64::EPSILON times the magnitude of + // the values it combines, so the computed transform differs from the exact one by at most + // about log2(N) · ε · Σ|x_n| times a small constant (Higham, Accuracy and Stability of + // Numerical Algorithms, 2nd ed., chapter 24, The Fast Fourier Transform). The bound comes + // from the input and N, never from the largest output, so a small part beside a large one + // (an imaginary part of 1 beside a real part of 1e14, a harmonic of 4e-7 beside a DC term of + // 8e6) is kept. With N = 1 the transform is the identity and log2(N) = 0: nothing is zeroed. + // The inverse divides its outputs by N, so its noise is divided by N as well. An input sum + // that overflows gives no finite bound; then nothing is cleaned up. + let mut noise = 8.0 * f64::EPSILON * f64::from(n.trailing_zeros()) * total; + if inverse { + noise /= n as f64; + } + let noise = if noise.is_finite() { noise } else { 0.0 }; + let clean = |x: f64| if x.abs() <= noise { 0.0 } else { x }; + let mut g = Grid::default(); + let first = u32::from(flag(p, "labels")); + if first == 1 { + g.txt(0, 0, &v.name); + } + for (i, (re, im)) in a.into_iter().enumerate() { + g.put(first + idx(i), 0, callf("COMPLEX", &[clean(re), clean(im)])); + } + Ok(g) +} + +// ---------------------------------------------------------------- random numbers and sampling + +/// A seeded generator: xoshiro256** (Blackman and Vigna, a public-domain algorithm) with its +/// state filled by SplitMix64. The same seed gives the same numbers in GridCraft; they are not +/// Excel's numbers, whose generator is not documented. +struct Rng([u64; 4]); + +impl Rng { + fn new(seed: u64) -> Rng { + let mut x = seed; + let mut next = || { + x = x.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = x; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + }; + Rng([next(), next(), next(), next()]) + } + + fn next_u64(&mut self) -> u64 { + let [s0, s1, s2, s3] = self.0; + let out = s1.wrapping_mul(5).rotate_left(7).wrapping_mul(9); + let (s2, s3) = (s2 ^ s0, s3 ^ s1); + self.0 = [s0 ^ s3, s1 ^ s2, s2 ^ (s1 << 17), s3.rotate_left(45)]; + out + } + + /// Uniform in [0, 1), with 53 random bits. + fn uniform(&mut self) -> f64 { + (self.next_u64() >> 11) as f64 / (1u64 << 53) as f64 + } + + /// Uniform whole number in [0, n), without modulo bias. + fn below(&mut self, n: u64) -> u64 { + if n <= 1 { + return 0; + } + let zone = u64::MAX - u64::MAX % n; + loop { + let v = self.next_u64(); + if v < zone { + return v % n; + } + } + } + + /// Standard normal by the Box–Muller transform. + fn normal(&mut self) -> f64 { + let (u1, u2) = (1.0 - self.uniform(), self.uniform()); + (-2.0 * u1.ln()).sqrt() * (2.0 * std::f64::consts::PI * u2).cos() + } +} + +/// The `seed` parameter, or a seed taken from the clock when there is none. +fn seed_of(p: &Json) -> Result { + let given = p.get("seed").is_some_and(|v| match v { + Json::Null => false, + Json::String(t) => !t.trim().is_empty(), + _ => true, + }); + if !given { + let ms = gridcraft_core::date::unix_now_ms().unwrap_or(0.0); + return Ok((ms * 1000.0) as u64 & MAX_SEED); + } + match num(p, "seed") { + Some(x) if x >= 0.0 && x.fract() == 0.0 && x <= MAX_SEED as f64 => Ok(x as u64), + _ => Err(EngineError::Other("The random seed must be a whole number from 0.".into())), + } +} + +/// A number parameter: `None` when absent, null or blank; an error when present but not a +/// finite number (a typed "0,3" or "abc" must not quietly become the default). +fn opt_num(p: &Json, key: &str) -> Result> { + match p.get(key) { + None | Some(Json::Null) => Ok(None), + Some(Json::String(t)) if t.trim().is_empty() => Ok(None), + Some(_) => num(p, key).map(Some).ok_or_else(|| EngineError::Other(format!("`{key}` must be a number."))), + } +} + +/// A number parameter, `default` when absent or blank. +fn num_or(p: &Json, key: &str, default: f64) -> Result { + Ok(opt_num(p, key)?.unwrap_or(default)) +} + +/// A whole-number parameter of at least `min`, `default` when absent. +fn whole(p: &Json, key: &str, default: Option, min: f64) -> Result { + match opt_num(p, key)?.or(default) { + Some(x) if x >= min && x.fract() == 0.0 => Ok(x), + Some(_) => Err(EngineError::Other(format!("`{key}` must be a whole number of at least {min}."))), + None => Err(bad(CMD, format!("missing `{key}`"))), + } +} + +/// A probability parameter in [0, 1]. +fn probability(p: &Json) -> Result { + match num_or(p, "p", 0.5)? { + x if (0.0..=1.0).contains(&x) => Ok(x), + _ => Err(EngineError::Other("The p value must be between 0 and 1.".into())), + } +} + +/// The probabilities of a distribution on 0, 1, 2 … `last`, as (first outcome, probabilities of +/// first, first + 1, …), from its probability `at_mode` at `mode`, its most likely outcome, and +/// `ratio`(k) = pmf(k + 1) / pmf(k). Each term is one multiplication of its neighbour towards the +/// mode, so no worksheet function runs per outcome, and the terms only shrink away from the mode, +/// so nothing underflows before the terms that matter. The left tail ends where its terms vanish, +/// the right tail once the cumulative probability reaches 1 within f64 resolution (every later +/// draw would map to the last outcome). The table is as wide as the distribution's effective +/// support, not as `last`: Binomial(1,000,000, 0.5) takes about 23,000 entries, not 1,000,001. +fn pmf_table(last: f64, mode: f64, at_mode: f64, ratio: impl Fn(f64) -> f64) -> Result<(f64, Vec)> { + if !at_mode.is_finite() || at_mode <= 0.0 || !(0.0..=last).contains(&mode) || mode.fract() != 0.0 { + return Err(EngineError::Other("The distribution could not be tabulated.".into())); + } + let mut left = Vec::new(); + let (mut k, mut v) = (mode, at_mode); + while k > 0.0 { + v /= ratio(k - 1.0); + if v.is_nan() || v < f64::MIN_POSITIVE { + break; + } + left.push(v); + k -= 1.0; + } + let first = k; + let mut pmf: Vec = left.into_iter().rev().collect(); + let mut acc = pmf.iter().sum::() + at_mode; + pmf.push(at_mode); + let (mut k, mut v) = (mode, at_mode); + while k < last && acc < 1.0 - 1e-15 { + v *= ratio(k); + if v.is_nan() || v < f64::MIN_POSITIVE { + break; + } + pmf.push(v); + acc += v; + k += 1.0; + } + Ok((first, pmf)) +} + +/// A distribution on whole numbers tabulated for inversion sampling: the cumulative +/// probabilities of `first`, `first + 1`, … up to the last outcome that matters. +struct Table { + first: f64, + cdf: Vec, +} + +impl Table { + /// The distribution on 0, 1, 2 … `last` that `pmf_table` describes. + fn from_mode(last: f64, mode: f64, at_mode: f64, ratio: impl Fn(f64) -> f64) -> Result { + let (first, pmf) = pmf_table(last, mode, at_mode, ratio)?; + let mut acc = 0.0; + let cdf = pmf + .into_iter() + .map(|p| { + acc += p; + acc + }) + .collect(); + Ok(Table { first, cdf }) + } + + /// All the probability on one outcome. + fn point(value: f64) -> Table { + Table { first: value, cdf: vec![1.0] } + } + + /// A draw: the first outcome whose cumulative probability exceeds a uniform number (inversion). + fn draw(&self, r: &mut Rng) -> f64 { + let u = r.uniform() * self.cdf.last().copied().unwrap_or(1.0); + self.first + self.cdf.partition_point(|c| *c <= u).min(self.cdf.len().saturating_sub(1)) as f64 + } +} + +/// The numbers of a patterned sequence: `from` to `to` in steps, each number repeated, the whole +/// sequence repeated. +fn pattern(p: &Json) -> Result> { + let (from, to, step) = (num_or(p, "from", 1.0)?, num_or(p, "to", 1.0)?, num_or(p, "step", 1.0)?); + let (each, times) = (whole(p, "repeatEach", Some(1.0), 1.0)?, whole(p, "repeatSequence", Some(1.0), 1.0)?); + if step <= 0.0 || to < from { + return Err(EngineError::Other("The pattern must run from a smaller number to a larger one in positive steps.".into())); + } + let steps = ((to - from) / step + 1e-9).floor() + 1.0; + if steps * each * times > MAX_RANDOM as f64 { + return Err(EngineError::Other(format!("Random Number Generation writes at most {MAX_RANDOM} numbers."))); + } + let base: Vec = (0..steps as usize).map(|k| from + step * k as f64).collect(); + let one: Vec = base.iter().flat_map(|v| std::iter::repeat_n(*v, each as usize)).collect(); + Ok(one.repeat(times as usize)) +} + +/// Random Number Generation: `count` numbers from a distribution in each of `variables` +/// columns, drawn row by row. +fn random_numbers(s: &Session, p: &Json, seed: u64) -> Result { + let vars = whole(p, "variables", Some(1.0), 1.0)?; + let dist = text(p, &["distribution"]).map(norm).unwrap_or_else(|| "uniform".into()); + let cells = |rows: f64| { + if rows * vars > MAX_RANDOM as f64 { + Err(EngineError::Other(format!("Random Number Generation writes at most {MAX_RANDOM} numbers."))) + } else { + Ok(rows as usize) + } + }; + let mut g = Grid::default(); + let cols = vars as usize; + if dist == "patterned" { + let seq = pattern(p)?; + cells(seq.len() as f64)?; + for (i, x) in seq.iter().enumerate() { + for c in 0..cols { + g.num(idx(i), idx(c), *x); + } + } + return Ok(g); + } + let count = cells(whole(p, "count", None, 1.0)?)?; + let mut draw: Box f64> = match dist.as_str() { + "uniform" => { + let (a, b) = (num_or(p, "from", 0.0)?, num_or(p, "to", 1.0)?); + Box::new(move |r| a + (b - a) * r.uniform()) + } + "normal" => { + let (m, sd) = (num_or(p, "mean", 0.0)?, num_or(p, "standardDeviation", 1.0)?); + if sd < 0.0 { + return Err(EngineError::Other("The standard deviation must not be negative.".into())); + } + Box::new(move |r| m + sd * r.normal()) + } + "bernoulli" => { + let pr = probability(p)?; + Box::new(move |r| if r.uniform() < pr { 1.0 } else { 0.0 }) + } + "binomial" => { + let (pr, n) = (probability(p)?, whole(p, "trials", Some(1.0), 1.0)?); + if n > MAX_OUTCOMES { + return Err(EngineError::Other(format!("The number of trials must be at most {MAX_OUTCOMES}."))); + } + // p = 0 and p = 1 are exact: every draw is 0 or n. Otherwise one BINOM.DIST value at + // the mode ⌊(n + 1)p⌋ anchors pmf(k + 1) = pmf(k) · (n − k)/(k + 1) · p/(1 − p). + let table = if pr == 0.0 { + Table::point(0.0) + } else if pr == 1.0 { + Table::point(n) + } else { + let mode = ((n + 1.0) * pr).floor().min(n); + let at_mode = f64_of(&call("BINOM.DIST", &[Value::Number(mode), Value::Number(n), Value::Number(pr), Value::Bool(false)])); + let odds = pr / (1.0 - pr); + Table::from_mode(n, mode, at_mode, |k| (n - k) / (k + 1.0) * odds)? + }; + Box::new(move |r| table.draw(r)) + } + "poisson" => { + let lambda = num_or(p, "lambda", 1.0)?; + if lambda <= 0.0 || lambda > 100_000.0 { + return Err(EngineError::Other("Lambda must be greater than 0 and at most 100000.".into())); + } + // One POISSON.DIST value at the mode ⌊λ⌋ anchors pmf(k + 1) = pmf(k) · λ/(k + 1). + let last = (lambda + 40.0 * lambda.sqrt() + 100.0).min(MAX_OUTCOMES); + let mode = lambda.floor(); + let at_mode = f64_of(&call("POISSON.DIST", &[Value::Number(mode), Value::Number(lambda), Value::Bool(false)])); + let table = Table::from_mode(last, mode, at_mode, |k| lambda / (k + 1.0))?; + Box::new(move |r| table.draw(r)) + } + "discrete" => { + let (si, r) = input_range(s, p, &["valueRange", "values"])?; + if r.width() != 2 { + return Err(EngineError::Other("The value and probability input range must have two columns.".into())); + } + let v = variables(s, si, r, false, false, &|_| String::new())?; + let (values, probs) = match v.as_slice() { + [a, b] => (numbers(&a.vals, false)?, numbers(&b.vals, false)?), + _ => return Err(non_numeric()), + }; + if probs.iter().any(|x| !(0.0..=1.0).contains(x)) || (probs.iter().sum::() - 1.0).abs() > 1e-9 { + return Err(EngineError::Other("The probabilities must be between 0 and 1 and add up to 1.".into())); + } + let cdf: Vec = probs + .iter() + .scan(0.0, |acc, x| { + *acc += x; + Some(*acc) + }) + .collect(); + let table = Table { first: 0.0, cdf }; + Box::new(move |r| values.get(table.draw(r) as usize).copied().unwrap_or(f64::NAN)) + } + other => return Err(bad(CMD, format!("unknown distribution `{other}`"))), + }; + let mut rng = Rng::new(seed); + for i in 0..count { + for c in 0..cols { + g.num(idx(i), idx(c), draw(&mut rng)); + } + } + Ok(g) +} + +/// Whether Random Number Generation writes a patterned sequence (which uses no seed). +fn patterned(p: &Json) -> bool { + text(p, &["distribution"]).map(norm).is_some_and(|d| d == "patterned") +} + +fn random_sampling(p: &Json) -> bool { + text(p, &["method"]).is_some_and(|m| m.eq_ignore_ascii_case("random")) +} + +/// Sampling: every `period`-th value of the input, or `samples` values drawn at random with +/// replacement, down one column. The input is read column by column (row by row for a single +/// row); blanks are skipped. +fn sampling(s: &Session, p: &Json, seed: Option) -> Result { + let (si, r) = input_range(s, p, &["range", "input"])?; + let by_rows = r.height() == 1 && r.width() > 1; + let vars = variables(s, si, r, by_rows, flag(p, "labels"), &|_| String::new())?; + let mut x = Vec::new(); + for v in &vars { + x.extend(numbers(&v.vals, true)?); + } + if x.is_empty() { + return Err(EngineError::Other("The input range holds no numbers.".into())); + } + let picked: Vec = match seed { + Some(seed) => { + let k = whole(p, "samples", None, 1.0)?; + if k > MAX_RANDOM as f64 { + return Err(EngineError::Other(format!("Sampling takes at most {MAX_RANDOM} samples."))); + } + let mut rng = Rng::new(seed); + (0..k as usize).filter_map(|_| x.get(rng.below(x.len() as u64) as usize).copied()).collect() + } + None => { + let period = whole(p, "period", None, 1.0)?; + if period > x.len() as f64 { + return Err(EngineError::Other("The period must be at most the number of input values.".into())); + } + x.iter().skip(period as usize - 1).step_by(period as usize).copied().collect() + } + }; + let mut g = Grid::default(); + for (i, v) in picked.iter().enumerate() { + g.num(idx(i), 0, *v); + } + Ok(g) +} + // ---------------------------------------------------------------- output /// Writes the report at `output` (refusing to overwrite data unless `overwrite`) or on a new @@ -1096,7 +1788,7 @@ fn write_report(s: &mut Session, p: &Json, tool: &str, g: Grid) -> Result { for row in rect.start.row..=rect.end.row { for (c, cell) in sh.cells.row(row, rect.start.col, rect.end.col) { - if !cell.value.is_empty() || cell.formula.is_some() { + if cell.has_content() { occupied.push(CellRef::new(row, c)); } } @@ -1124,13 +1816,13 @@ fn write_report(s: &mut Session, p: &Json, tool: &str, g: Grid) -> Result let sh = cx.sheet_mut(si)?; for c in &occupied { let style = sh.style_id(*c); - sh.set_cell(*c, Cell { value: Value::Empty, formula: None, style }); + sh.set_cell(*c, Cell { value: Value::Empty, formula: None, style, py: None }); } let mut touched = occupied.clone(); for ((r, c), v) in &g.cells { let cell = CellRef::new(at.row + r, at.col + c); let style = pct_styles.get(&cell).copied().unwrap_or_else(|| sh.style_id(cell)); - sh.set_cell(cell, Cell { value: v.clone(), formula: None, style }); + sh.set_cell(cell, Cell { value: v.clone(), formula: None, style, py: None }); touched.push(cell); } for c in touched { @@ -1150,6 +1842,7 @@ fn run(s: &mut Session, p: &Json) -> Result { return Ok(json!({"tools": TOOLS.iter().map(|(id, label)| json!({"id": id, "label": label})).collect::>()})); }; let tool = tool_id(t).ok_or_else(|| bad(CMD, format!("unknown tool `{t}`")))?; + let mut seed = None; let g = match tool { "descriptive" => descriptive(s, p)?, "histogram" => histogram(s, p)?, @@ -1162,16 +1855,38 @@ fn run(s: &mut Session, p: &Json) -> Result { "tTestPaired" | "tTestEqual" | "tTestUnequal" => t_test(s, p, tool)?, "fTest" => f_test(s, p)?, "anovaSingle" => anova_single(s, p)?, + "anovaTwoFactorWithout" => anova_two_factor_without(s, p)?, + "anovaTwoFactorWith" => anova_two_factor_with(s, p)?, + "zTest" => z_test(s, p)?, + "fourierAnalysis" => fourier_analysis(s, p)?, + "randomNumberGeneration" => { + // A patterned sequence draws nothing: its seed is neither read nor reported. + if !patterned(p) { + seed = Some(seed_of(p)?); + } + random_numbers(s, p, seed.unwrap_or(0))? + } + "sampling" => { + if random_sampling(p) { + seed = Some(seed_of(p)?); + } + sampling(s, p, seed)? + } _ => return Err(bad(CMD, format!("unknown tool `{t}`"))), }; - write_report(s, p, tool, g) + let mut out = write_report(s, p, tool, g)?; + if let Some(k) = seed { + out["seed"] = json!(k); + } + Ok(out) } #[cfg(test)] mod tests { use gridcraft_core::{CellRef, Value}; - use serde_json::json; + use serde_json::{Value as Json, json}; + use super::{MAX_OUTCOMES, call, f64_of, pmf_table}; use crate::Session; fn s() -> Session { @@ -1375,6 +2090,462 @@ mod tests { assert_eq!(n(&s, "X4"), 0.25, "4 has one value below it among five"); } + #[test] + fn two_factor_anova_without_replication_matches_a_textbook_example() { + // Hardness testing, a randomized complete block design (Montgomery, Design and Analysis + // of Experiments, Example 4.1): four tips (rows) by four coupons (columns). Published: + // SS tips 0.385, SS coupons 0.825, SS error 0.080, SS total 1.290, F tips 14.44. + let mut s = s(); + s.execute( + "range.setValues", + json!({"range": "A1", "values": [ + ["", "Coupon 1", "Coupon 2", "Coupon 3", "Coupon 4"], + ["Tip 1", 9.3, 9.4, 9.6, 10.0], + ["Tip 2", 9.4, 9.3, 9.8, 9.9], + ["Tip 3", 9.2, 9.4, 9.5, 9.7], + ["Tip 4", 9.7, 9.6, 10.0, 10.2], + ]}), + ) + .unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "Anova: Two-Factor Without Replication", "range": "A1:E5", "labels": true, "output": "H1"})) + .unwrap(); + assert_eq!(v(&s, "H1"), Value::text("Anova: Two-Factor Without Replication")); + assert_eq!((v(&s, "H4"), n(&s, "I4")), (Value::text("Tip 1"), 4.0)); + assert!(close(n(&s, "J4"), 38.3, 1e-12), "sum of tip 1"); + assert_eq!(v(&s, "H9"), Value::text("Coupon 1")); + assert!(close(n(&s, "K12"), 9.95, 1e-12), "average of coupon 4"); + assert_eq!(v(&s, "H17"), Value::text("Rows")); + assert!(close(n(&s, "I17"), 0.385, 1e-12)); + assert!(close(n(&s, "I18"), 0.825, 1e-12)); + assert!(close(n(&s, "I19"), 0.080, 1e-12)); + assert!(close(n(&s, "I21"), 1.290, 1e-12)); + assert_eq!((n(&s, "J17"), n(&s, "J19"), n(&s, "J21")), (3.0, 9.0, 15.0)); + assert!(close(n(&s, "L17"), 14.4375, 1e-9), "F for tips"); + assert!(close(n(&s, "M17"), eval(&mut s, "=F.DIST.RT(14.4375,3,9)"), 1e-9)); + assert!(close(n(&s, "N17"), eval(&mut s, "=F.INV.RT(0.05,3,9)"), 1e-9)); + assert!(v(&s, "L19").is_empty(), "the error line has no F"); + } + + #[test] + fn two_factor_anova_with_replication_matches_a_textbook_example() { + let mut s = s(); + // Battery life (Montgomery, Example 5.1): three materials (samples of four rows) by three + // temperatures (columns). Published: SS material 10683.72, SS temperature 39118.72, + // SS interaction 9613.78, SS error 18230.75 on 27 df, SS total 77646.97. + s.execute( + "range.setValues", + json!({"range": "A20", "values": [ + ["Material", "15 F", "70 F", "125 F"], + ["Type 1", 130, 34, 20], ["", 155, 40, 70], ["", 74, 80, 82], ["", 180, 75, 58], + ["Type 2", 150, 136, 25], ["", 188, 122, 70], ["", 159, 106, 58], ["", 126, 115, 45], + ["Type 3", 138, 174, 96], ["", 110, 120, 104], ["", 168, 150, 82], ["", 160, 139, 60], + ]}), + ) + .unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "anovaTwoFactorWith", "range": "A20:D32", "rowsPerSample": 4, "newSheet": "Battery"})) + .unwrap(); + assert_eq!(v(&s, "Battery!A3"), Value::text("SUMMARY")); + assert_eq!((v(&s, "Battery!B3"), v(&s, "Battery!E3")), (Value::text("15 F"), Value::text("Total"))); + assert_eq!(v(&s, "Battery!A4"), Value::text("Type 1")); + assert_eq!((n(&s, "Battery!B5"), n(&s, "Battery!B6"), n(&s, "Battery!E5")), (4.0, 539.0, 12.0)); + assert_eq!(v(&s, "Battery!A22"), Value::text("Total")); + assert_eq!(n(&s, "Battery!B24"), 1738.0, "sum of the 15 F column"); + assert_eq!(v(&s, "Battery!A29"), Value::text("ANOVA")); + assert_eq!( + (v(&s, "Battery!A31"), v(&s, "Battery!A34"), v(&s, "Battery!A36")), + (Value::text("Sample"), Value::text("Within"), Value::text("Total")) + ); + assert!(close(n(&s, "Battery!B31"), 10683.72, 0.01)); + assert!(close(n(&s, "Battery!B32"), 39118.72, 0.01)); + assert!(close(n(&s, "Battery!B33"), 9613.78, 0.01)); + assert!(close(n(&s, "Battery!B34"), 18230.75, 0.01)); + assert!(close(n(&s, "Battery!B36"), 77646.97, 0.01)); + assert_eq!((n(&s, "Battery!C33"), n(&s, "Battery!C34"), n(&s, "Battery!C36")), (4.0, 27.0, 35.0)); + assert!(close(n(&s, "Battery!E31"), 7.91, 0.005) && close(n(&s, "Battery!E32"), 28.97, 0.005) && close(n(&s, "Battery!E33"), 3.56, 0.005)); + assert!(close(n(&s, "Battery!F33"), 0.0186, 0.00005), "interaction P-value"); + // Rows per sample must divide the data rows. + let bad = json!({"tool": "anovaTwoFactorWith", "range": "Sheet1!A20:D32", "rowsPerSample": 5, "output": "F60"}); + assert!(s.execute("data.analysisToolPak", bad).is_err()); + } + + #[test] + fn z_test_with_known_variances() { + // x̄ = 3 (n = 5, σ² = 2), ȳ = 4 (n = 3, σ² = 6): z = −1/√2.4. Critical values from the + // standard normal table: 1.644854 (one tail) and 1.959964 (two tails) at α = 0.05. + let mut s = s(); + s.execute("range.setValues", json!({"range": "A1", "values": [["x", "y"], [1, 2], [2, 4], [3, 6], [4, null], [5, null]]})).unwrap(); + s.execute( + "data.analysisToolPak", + json!({"tool": "zTest", "range1": "A1:A6", "range2": "B1:B6", "labels": true, "variance1": 2, "variance2": 6, "output": "D1"}), + ) + .unwrap(); + assert_eq!(v(&s, "D1"), Value::text("z-Test: Two Sample for Means")); + assert_eq!((v(&s, "E3"), v(&s, "F3")), (Value::text("x"), Value::text("y"))); + assert_eq!((n(&s, "E4"), n(&s, "F4"), n(&s, "E5"), n(&s, "F6")), (3.0, 4.0, 2.0, 3.0)); + let z = -1.0 / 2.4f64.sqrt(); + assert!(close(n(&s, "E8"), z, 1e-12)); + let tail = 1.0 - eval(&mut s, &format!("=NORM.S.DIST({},TRUE)", -z)); + assert!(close(n(&s, "E9"), tail, 1e-12) && close(n(&s, "E9"), 0.2593, 1e-4)); + assert!(close(n(&s, "E10"), 1.644_853_6, 1e-6)); + assert!(close(n(&s, "E11"), 2.0 * tail, 1e-12)); + assert!(close(n(&s, "E12"), 1.959_964_0, 1e-6)); + // Unknown or non-positive variances are refused. + assert!(s.execute("data.analysisToolPak", json!({"tool": "zTest", "range1": "A2:A6", "range2": "B2:B4", "output": "H1"})).is_err()); + let zero = json!({"tool": "zTest", "range1": "A2:A6", "range2": "B2:B4", "variance1": 0, "variance2": 1, "output": "H1"}); + assert!(s.execute("data.analysisToolPak", zero).is_err()); + } + + #[test] + fn fourier_analysis_matches_known_transforms() { + let mut s = s(); + // DFT of 1, 2, 3, 4: 10, −2+2i, −2, −2−2i. + s.execute("range.setValues", json!({"range": "A1", "values": [["Signal"], [1], [2], [3], [4]]})).unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "Fourier Analysis", "range": "A1:A5", "labels": true, "output": "C1"})).unwrap(); + let col = |s: &Session, c: &str, rows: std::ops::RangeInclusive| rows.map(|r| v(s, &format!("{c}{r}"))).collect::>(); + assert_eq!(col(&s, "C", 1..=5), ["Signal", "10", "-2+2i", "-2", "-2-2i"].map(Value::text).to_vec()); + // The inverse transform of that output returns the signal. + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "C2:C5", "inverse": true, "output": "E1"})).unwrap(); + assert_eq!(col(&s, "E", 1..=4), ["1", "2", "3", "4"].map(Value::text).to_vec()); + // cos(2πn/8) over eight points: N/2 = 4 at the first and last frequencies, 0 elsewhere. + let wave: Vec = (0..8).map(|k| json!([(std::f64::consts::PI * k as f64 / 4.0).cos()])).collect(); + s.execute("range.setValues", json!({"range": "G1", "values": wave})).unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "G1:G8", "output": "H1"})).unwrap(); + assert_eq!(col(&s, "H", 1..=8), ["0", "4", "0", "0", "0", "0", "0", "4"].map(Value::text).to_vec()); + // Complex text input; a single value is its own transform. + s.execute("range.setValues", json!({"range": "J1", "values": [["3+4i"], ["1-i"]]})).unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "J1:J2", "output": "K1"})).unwrap(); + assert_eq!(col(&s, "K", 1..=2), ["4+3i", "2+5i"].map(Value::text).to_vec()); + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "J1", "output": "L1"})).unwrap(); + assert_eq!(v(&s, "L1"), Value::text("3+4i")); + // Three values are not a power of two; a word is not a number. + assert!(s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "A2:A4", "output": "N1"})).is_err()); + assert!(s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "A1:A4", "output": "N1"})).is_err()); + assert!(s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "A1:A8192", "output": "N1"})).is_err()); + } + + #[test] + fn fourier_analysis_keeps_a_small_imaginary_part_beside_a_large_real_one() { + let mut s = s(); + // With one value the transform is the identity and performs no arithmetic, so no part of + // the output is rounding noise: 1e14 + 1i comes back as 1e14 + 1i, forward and inverse. + s.execute("range.setValues", json!({"range": "A1", "values": [["100000000000000+1i"]]})).unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "A1", "output": "C1"})).unwrap(); + assert_eq!(eval(&mut s, "=IMREAL(C1)"), 1e14); + assert_eq!(eval(&mut s, "=IMAGINARY(C1)"), 1.0); + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "C1", "inverse": true, "output": "E1"})).unwrap(); + assert_eq!(eval(&mut s, "=IMREAL(E1)"), 1e14); + assert_eq!(eval(&mut s, "=IMAGINARY(E1)"), 1.0); + } + + #[test] + fn fourier_analysis_round_trip_recovers_small_components_beside_a_large_dc() { + let mut s = s(); + // x_n = 1e6 + 1e-7·cos(2πn/8) for n = 0 … 7. The transform of a constant c over N points is + // N·c at k = 0 and 0 elsewhere; the transform of a·cos(2πn/N) is a·N/2 at k = 1 and + // k = N − 1. So X_0 = 8·1e6 = 8e6, X_1 = X_7 = 1e-7·8/2 = 4e-7 and X_2 … X_6 = 0. The inputs + // are stored to within ulp(1e6)/2 ≈ 6e-11 and the first butterfly stage subtracts nearby + // numbers exactly, so the harmonics carry at most about 5e-10 of noise. The cleanup bound + // 8·ε·log2(8)·Σ|x_n| ≈ 8 · 2.2e-16 · 3 · 8e6 ≈ 4.3e-8 zeroes noise of that size and keeps + // 4e-7; a cutoff relative to the largest output (8e6 · 1e-13 = 8e-7) would zero the + // harmonics. + let x = |k: usize| 1e6 + 1e-7 * (std::f64::consts::PI * k as f64 / 4.0).cos(); + let wave: Vec = (0..8).map(|k| json!([x(k)])).collect(); + s.execute("range.setValues", json!({"range": "A1", "values": wave})).unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "A1:A8", "output": "C1"})).unwrap(); + assert!(close(eval(&mut s, "=IMREAL(C1)"), 8e6, 1e-6)); + assert_eq!(eval(&mut s, "=IMAGINARY(C1)"), 0.0); + for cell in ["C2", "C8"] { + assert!(close(eval(&mut s, &format!("=IMREAL({cell})")), 4e-7, 5e-9), "{cell}"); + assert_eq!(eval(&mut s, &format!("=IMAGINARY({cell})")), 0.0, "{cell}"); + } + for r in 3..=7 { + assert_eq!(v(&s, &format!("C{r}")), Value::text("0"), "C{r}"); + } + // The inverse of that output returns each x_n to within 2e-9 (2e-15 relative); without the + // harmonics every value would come back as 1e6, off by up to 1e-7. + s.execute("data.analysisToolPak", json!({"tool": "fourierAnalysis", "range": "C1:C8", "inverse": true, "output": "E1"})).unwrap(); + for k in 0..8 { + let cell = format!("E{}", k + 1); + assert!(close(eval(&mut s, &format!("=IMREAL({cell})")), x(k), 2e-9), "{cell}"); + assert_eq!(eval(&mut s, &format!("=IMAGINARY({cell})")), 0.0, "{cell}"); + } + } + + /// Column `col` of sheet `sheet`, rows 1 to `rows`, as numbers. + fn numbers_of(s: &Session, sheet: &str, col: &str, rows: u32) -> Vec { + (1..=rows).map(|r| n(s, &format!("{sheet}!{col}{r}"))).collect() + } + fn mean_var(x: &[f64]) -> (f64, f64) { + let m = x.iter().sum::() / x.len() as f64; + (m, x.iter().map(|v| (v - m).powi(2)).sum::() / (x.len() - 1) as f64) + } + + #[test] + fn random_numbers_are_seeded_and_follow_their_distributions() { + let mut s = s(); + let rng = |s: &mut Session, sheet: &str, p: Json| { + let mut p = p; + p["tool"] = json!("randomNumberGeneration"); + p["newSheet"] = json!(sheet); + s.execute("data.analysisToolPak", p).unwrap() + }; + // The same seed repeats the numbers; the result reports the seed. + let r = rng(&mut s, "U1", json!({"variables": 2, "count": 2000, "distribution": "uniform", "from": 2, "to": 4, "seed": 42})); + assert_eq!((r["seed"].as_u64(), r["range"].as_str()), (Some(42), Some("A1:B2000"))); + rng(&mut s, "U2", json!({"variables": 2, "count": 2000, "distribution": "uniform", "from": 2, "to": 4, "seed": 42})); + let (a, b) = (numbers_of(&s, "U1", "B", 2000), numbers_of(&s, "U2", "B", 2000)); + assert_eq!(a, b); + assert!(a.iter().all(|x| (2.0..4.0).contains(x))); + // Uniform on [2, 4]: mean 3, variance 1/3. + let (m, var) = mean_var(&a); + assert!((m - 3.0).abs() < 0.05 && (var - 1.0 / 3.0).abs() < 0.03, "{m} {var}"); + let unseeded = rng(&mut s, "U3", json!({"count": 1})); + assert!(unseeded["seed"].as_u64().is_some(), "a seed is chosen and reported"); + // Normal(10, 2): mean 10, variance 4. + rng(&mut s, "N", json!({"count": 20000, "distribution": "normal", "mean": 10, "standardDeviation": 2, "seed": 7})); + let (m, var) = mean_var(&numbers_of(&s, "N", "A", 20000)); + assert!((m - 10.0).abs() < 0.05 && (var - 4.0).abs() < 0.15, "{m} {var}"); + // Binomial(20, 0.3): mean np = 6, variance np(1 − p) = 4.2. + rng(&mut s, "B", json!({"count": 20000, "distribution": "binomial", "p": 0.3, "trials": 20, "seed": 8})); + let x = numbers_of(&s, "B", "A", 20000); + assert!(x.iter().all(|v| v.fract() == 0.0 && (0.0..=20.0).contains(v))); + let (m, var) = mean_var(&x); + assert!((m - 6.0).abs() < 0.08 && (var - 4.2).abs() < 0.2, "{m} {var}"); + // Poisson(3.5): mean and variance 3.5. + rng(&mut s, "P", json!({"count": 20000, "distribution": "poisson", "lambda": 3.5, "seed": 9})); + let (m, var) = mean_var(&numbers_of(&s, "P", "A", 20000)); + assert!((m - 3.5).abs() < 0.08 && (var - 3.5).abs() < 0.2, "{m} {var}"); + // Bernoulli(0.25) and a discrete distribution on {1, 2} with P(2) = 0.75. + rng(&mut s, "Ber", json!({"count": 4000, "distribution": "bernoulli", "p": 0.25, "seed": 10})); + let ones = numbers_of(&s, "Ber", "A", 4000).iter().filter(|v| **v == 1.0).count(); + assert!((900..1100).contains(&ones), "{ones}"); + s.execute("range.setValues", json!({"range": "Sheet1!A1", "values": [[1, 0.25], [2, 0.75]]})).unwrap(); + rng(&mut s, "D", json!({"count": 4000, "distribution": "discrete", "valueRange": "Sheet1!A1:B2", "seed": 11})); + let x = numbers_of(&s, "D", "A", 4000); + assert!(x.iter().all(|v| *v == 1.0 || *v == 2.0)); + let twos = x.iter().filter(|v| **v == 2.0).count(); + assert!((2900..3100).contains(&twos), "{twos}"); + // Patterned: 1 to 3, each number twice, the sequence twice. + rng(&mut s, "Pat", json!({"distribution": "patterned", "from": 1, "to": 3, "step": 1, "repeatEach": 2, "repeatSequence": 2})); + assert_eq!(numbers_of(&s, "Pat", "A", 12), [1.0, 1.0, 2.0, 2.0, 3.0, 3.0, 1.0, 1.0, 2.0, 2.0, 3.0, 3.0]); + assert!(v(&s, "Pat!A13").is_empty()); + // Output is bounded; parameters are checked. + for p in [ + json!({"count": 2_000_000}), + json!({"count": 1000, "variables": 2000}), + json!({"count": 0}), + json!({"count": 1.5}), + json!({"distribution": "uniform"}), + json!({"count": 1, "distribution": "bernoulli", "p": 2}), + json!({"count": 1, "distribution": "binomial", "trials": 1e9}), + json!({"count": 1, "distribution": "poisson", "lambda": -1}), + json!({"count": 1, "distribution": "normal", "standardDeviation": -1}), + json!({"count": 1, "distribution": "discrete", "valueRange": "Sheet1!A1:A2"}), + json!({"count": 1, "distribution": "patterned", "from": 1, "to": 1e9, "step": 1}), + json!({"count": 1, "distribution": "patterned", "from": 3, "to": 1}), + json!({"count": 1, "distribution": "weird"}), + json!({"count": 1, "seed": -1}), + json!({"count": 1, "seed": "abc"}), + ] { + let mut q = p.clone(); + q["tool"] = json!("randomNumberGeneration"); + q["output"] = json!("Sheet1!H1"); + assert!(s.execute("data.analysisToolPak", q).is_err(), "{p}"); + } + s.execute("range.setValues", json!({"range": "Sheet1!A1", "values": [[1, 0.5], [2, 0.6]]})).unwrap(); + let p = + json!({"tool": "randomNumberGeneration", "count": 1, "distribution": "discrete", "valueRange": "Sheet1!A1:B2", "output": "Sheet1!H1"}); + assert!(s.execute("data.analysisToolPak", p).is_err(), "probabilities must add up to 1"); + } + + #[test] + fn binomial_and_poisson_tables_agree_with_the_worksheet_functions() { + // The recurrence outward from the mode reproduces BINOM.DIST and POISSON.DIST at every + // tabulated outcome, the table carries the whole distribution, and it is only as wide as + // the outcomes that matter. The worksheet functions themselves are accurate to about + // n · ε for n trials (their logarithms are of that size), hence the moderate n and λ. + let binom = |k: f64, n: f64, p: f64| f64_of(&call("BINOM.DIST", &[Value::Number(k), Value::Number(n), Value::Number(p), Value::Bool(false)])); + for (n, p) in [(20.0f64, 0.3f64), (100.0, 0.5), (50.0, 0.9), (30.0, 0.01), (7.0, 0.999), (1.0, 0.5)] { + let mode = ((n + 1.0) * p).floor().min(n); + let odds = p / (1.0 - p); + let (first, pmf) = pmf_table(n, mode, binom(mode, n, p), |k| (n - k) / (k + 1.0) * odds).unwrap(); + assert!(first >= 0.0 && first + pmf.len() as f64 - 1.0 <= n, "n={n} p={p}: {first} + {}", pmf.len()); + for (i, got) in pmf.iter().enumerate() { + let k = first + i as f64; + let want = binom(k, n, p); + assert!(((got - want) / want).abs() < 1e-12, "n={n} p={p} k={k}: {got} vs {want}"); + } + assert!((pmf.iter().sum::() - 1.0).abs() < 1e-12, "n={n} p={p}"); + } + let poisson = |k: f64, lambda: f64| f64_of(&call("POISSON.DIST", &[Value::Number(k), Value::Number(lambda), Value::Bool(false)])); + for lambda in [0.2f64, 3.5, 50.0] { + let mode = lambda.floor(); + let (first, pmf) = pmf_table(MAX_OUTCOMES, mode, poisson(mode, lambda), |k| lambda / (k + 1.0)).unwrap(); + assert_eq!(first, 0.0); + for (i, got) in pmf.iter().enumerate() { + let want = poisson(i as f64, lambda); + assert!(((got - want) / want).abs() < 1e-12, "lambda={lambda} k={i}: {got} vs {want}"); + } + assert!((pmf.iter().sum::() - 1.0).abs() < 1e-12, "lambda={lambda}"); + } + // Binomial(1e6, 0.5): standard deviation 500, so the terms vanish within about 38 + // standard deviations below the mode and the sum reaches 1 within about 8 above it: + // roughly 23,000 entries starting near 481,000, not 1,000,001 starting at 0. + let (n, p) = (1_000_000.0, 0.5); + let (first, pmf) = pmf_table(n, 500_000.0, binom(500_000.0, n, p), |k| (n - k) / (k + 1.0)).unwrap(); + assert!((470_000.0..=490_000.0).contains(&first) && pmf.len() < 40_000, "{first} + {}", pmf.len()); + // The anchor BINOM.DIST(500000, 1e6, 0.5) carries that function's error of about n · ε + // (its logarithms are near 7e5), which scales the whole table; draws divide by the total. + assert!((pmf.iter().sum::() - 1.0).abs() < 1e-8); + // A bad anchor or mode is refused rather than tabulated. + assert!(pmf_table(10.0, 5.0, f64::NAN, |_| 1.0).is_err()); + assert!(pmf_table(10.0, 5.0, 0.0, |_| 1.0).is_err()); + assert!(pmf_table(10.0, 11.0, 0.5, |_| 1.0).is_err()); + } + + #[test] + #[allow(clippy::disallowed_methods)] // a native-only timing test: the clock is read off wasm + fn binomial_and_poisson_draws_at_the_limits_are_tabulated_quickly() { + let mut s = s(); + // The tables cover the outcomes around the mode, not every one of the 1,000,001 for a + // million trials, so the largest inputs cost milliseconds. Binomial(1e6, 0.5) has mean + // 500000 and standard deviation 500: 1000 draws put the sample mean within 500/√1000 ≈ 16 + // of it, so 100 is six standard errors. Poisson(1e5) has mean 1e5 and standard deviation + // 316: 1000 draws put the mean within 10, so 60 is six standard errors. With p = 1 every + // draw is exactly 1e6, with p = 0 exactly 0. + let mut draw = |sheet: &str, dist: Json| { + let mut dist = dist; + dist["tool"] = json!("randomNumberGeneration"); + dist["count"] = json!(1000); + dist["seed"] = json!(3); + dist["newSheet"] = json!(sheet); + s.execute("data.analysisToolPak", dist).unwrap(); + }; + let start = std::time::Instant::now(); + draw("Half", json!({"distribution": "binomial", "p": 0.5, "trials": 1_000_000})); + draw("Sure", json!({"distribution": "binomial", "p": 1, "trials": 1_000_000})); + draw("Never", json!({"distribution": "binomial", "p": 0, "trials": 1_000_000})); + draw("Pois", json!({"distribution": "poisson", "lambda": 100_000})); + let elapsed = start.elapsed(); + let (m, _) = mean_var(&numbers_of(&s, "Half", "A", 1000)); + assert!((m - 500_000.0).abs() < 100.0, "{m}"); + assert!(numbers_of(&s, "Sure", "A", 1000).iter().all(|v| *v == 1_000_000.0)); + assert!(numbers_of(&s, "Never", "A", 1000).iter().all(|v| *v == 0.0)); + let (m, _) = mean_var(&numbers_of(&s, "Pois", "A", 1000)); + assert!((m - 100_000.0).abs() < 60.0, "{m}"); + assert!(elapsed < std::time::Duration::from_secs(2), "{elapsed:?}"); + } + + #[test] + fn sampling_periodic_and_random() { + let mut s = s(); + let values: Vec = (1..=10).map(|k| json!([k])).collect(); + s.execute("range.setValues", json!({"range": "A1", "values": values})).unwrap(); + s.execute("data.analysisToolPak", json!({"tool": "Sampling", "range": "A1:A10", "period": 3, "output": "C1"})).unwrap(); + assert_eq!(numbers_of(&s, "Sheet1", "C", 3), [3.0, 6.0, 9.0]); + assert!(v(&s, "C4").is_empty()); + let r = s + .execute( + "data.analysisToolPak", + json!({"tool": "sampling", "range": "A1:A10", "method": "random", "samples": 20, "seed": 5, "output": "D1"}), + ) + .unwrap(); + assert_eq!(r["seed"], 5); + s.execute( + "data.analysisToolPak", + json!({"tool": "sampling", "range": "A1:A10", "method": "random", "samples": 20, "seed": 5, "output": "E1"}), + ) + .unwrap(); + let (a, b) = (numbers_of(&s, "Sheet1", "D", 20), numbers_of(&s, "Sheet1", "E", 20)); + assert_eq!(a, b, "the same seed repeats the sample"); + assert!(a.iter().all(|x| x.fract() == 0.0 && (1.0..=10.0).contains(x))); + for p in [ + json!({"range": "A1:A10", "period": 11}), + json!({"range": "A1:A10", "period": 0}), + json!({"range": "A1:A10"}), + json!({"range": "A1:A10", "method": "random", "samples": 2_000_000}), + json!({"range": "B1:B10", "period": 1}), + ] { + let mut q = p.clone(); + q["tool"] = json!("sampling"); + q["output"] = json!("G1"); + assert!(s.execute("data.analysisToolPak", q).is_err(), "{p}"); + } + } + + #[test] + fn options_typed_as_text_that_is_not_a_number_are_refused() { + // A form sends its options as text; one that doesn't read as a number (a decimal comma, + // a typo) is an error, not the default, while a blank option takes the default. + let mut s = s(); + s.execute("range.setValues", json!({"range": "A1", "values": [[1, 2], [2, 4], [3, 6]]})).unwrap(); + let rng = |dist: &str| json!({"tool": "randomNumberGeneration", "count": 3, "distribution": dist, "seed": 1, "outputTo": "newSheet"}); + let z = json!({"tool": "zTest", "range1": "Sheet1!A1:A3", "range2": "Sheet1!B1:B3", "variance1": 1, "variance2": 4, "outputTo": "newSheet"}); + let with = |base: &Json, key: &str, val: &str| { + let mut p = base.clone(); + p[key] = json!(val); + p + }; + let cases = [ + (rng("uniform"), "from"), + (rng("uniform"), "to"), + (rng("normal"), "mean"), + (rng("normal"), "standardDeviation"), + (rng("bernoulli"), "p"), + (rng("binomial"), "p"), + (rng("poisson"), "lambda"), + (rng("patterned"), "step"), + (rng("uniform"), "variables"), + (z.clone(), "hypothesizedDifference"), + ]; + for (base, key) in cases { + let e = s.execute("data.analysisToolPak", with(&base, key, "0,3")).unwrap_err().to_string(); + assert!(e.contains(key), "{key}: {e}"); + s.execute("data.analysisToolPak", with(&base, key, " ")).unwrap_or_else(|e| panic!("{key} blank: {e}")); + } + } + + #[test] + fn an_alpha_that_is_not_a_number_is_refused() { + // A decimal comma or a typo in Alpha must not run the test at 0.05 without a word. + let mut s = s(); + s.execute("range.setValues", json!({"range": "A1", "values": [["", "c1", "c2"], ["r1", 1, 2], ["r1", 2, 4], ["r2", 3, 6], ["r2", 5, 9]]})) + .unwrap(); + for base in [ + json!({"tool": "zTest", "range1": "Sheet1!B2:B5", "range2": "Sheet1!C2:C5", "variance1": 1, "variance2": 4}), + json!({"tool": "anovaTwoFactorWithout", "range": "Sheet1!B2:C5"}), + json!({"tool": "anovaTwoFactorWith", "range": "Sheet1!A1:C5", "rowsPerSample": 2}), + json!({"tool": "tTestEqual", "range1": "Sheet1!B2:B5", "range2": "Sheet1!C2:C5"}), + ] { + let mut p = base.clone(); + p["alpha"] = json!("0,01"); + let e = s.execute("data.analysisToolPak", p).unwrap_err().to_string(); + assert!(e.contains("alpha"), "{base}: {e}"); + let mut p = base.clone(); + p["alpha"] = json!(""); + s.execute("data.analysisToolPak", p).unwrap_or_else(|e| panic!("{base} blank alpha: {e}")); + } + } + + #[test] + fn a_patterned_sequence_ignores_the_seed() { + // Patterned generation draws nothing, so a seed left in the form (even one that isn't a + // number) neither blocks the run nor shows up in the result. + let mut s = s(); + for seed in [json!("abc"), json!(7)] { + let r = s + .execute( + "data.analysisToolPak", + json!({"tool": "randomNumberGeneration", "distribution": "patterned", "from": 1, "to": 3, "seed": seed, "outputTo": "newSheet"}), + ) + .unwrap_or_else(|e| panic!("seed {seed}: {e}")); + assert!(r.get("seed").is_none(), "seed {seed}: {r}"); + } + // A random distribution still refuses the bad seed. + let bad = json!({"tool": "randomNumberGeneration", "distribution": "uniform", "count": 2, "seed": "abc", "outputTo": "newSheet"}); + assert!(s.execute("data.analysisToolPak", bad).is_err()); + } + #[test] fn hostile_inputs_are_refused_without_panicking() { let mut s = s(); @@ -1399,6 +2570,11 @@ mod tests { json!({"tool": "anovaSingle", "range": "A1:A2"}), json!({"tool": "correlation", "range": "A1:B2", "alpha": "x", "groupedBy": 5}), json!({"tool": "descriptive", "range": "A1:A2", "confidence": 250}), + json!({"tool": "anovaTwoFactorWithout", "range": "A1:B2"}), + json!({"tool": "anovaTwoFactorWithout", "range": "A1:A2"}), + json!({"tool": "anovaTwoFactorWith", "range": "A1:B2", "rowsPerSample": 2}), + json!({"tool": "anovaTwoFactorWith", "range": "A1:B2", "rowsPerSample": 1e300}), + json!({"tool": "zTest", "range1": "A1", "range2": "A2", "variance1": "x", "variance2": -1}), ] { assert!(s.execute("data.analysisToolPak", p.clone()).is_err(), "{p}"); } diff --git a/crates/engine/src/cmd/data.rs b/crates/engine/src/cmd/data.rs index 5fa076e64..0906f4893 100644 --- a/crates/engine/src/cmd/data.rs +++ b/crates/engine/src/cmd/data.rs @@ -4,10 +4,16 @@ use std::cmp::Ordering; use std::sync::Arc; use gridcraft_core::{CellRef, RangeRef, Value, sort_compare}; -use gridcraft_model::*; +use gridcraft_model::{ + Anchor, AnchorMode, AutoFilter, Cell, CfOperator, Comment, ErrorStyle, FilterCriterion, Formula, Hyperlink, LineInfo, Sheet, Validation, + ValidationKind, Workbook, +}; use serde_json::{Value as Json, json}; -use super::*; +use super::{ + CommandSpec, EngineError, Result, Session, bad, bool_param, cell_param, cell_param_on, cmd, edit, has_doc, ok, parse_lines, split_sheet, + str_param, target_range, target_ranges, target_sheet, +}; use crate::outline; use crate::selection::current_region; @@ -93,18 +99,30 @@ pub fn specs() -> Vec { ), cmd!(query "data.validate", "Check Value Against Validation", [], None, "{cell?, input} → {ok, message?}", has_doc, validate_cmd), cmd!(noundo "data.circleInvalid", "Circle Invalid Data", ["Data", "Data Tools", "Data Validation"], None, "{} → invalid cells", has_doc, circle_invalid), - cmd!( - "data.group", - "Group", - ["Data", "Outline"], - Some("Alt+Shift+Right"), - "{rows?: \"2:5\" | cols?: \"B:C\" | by?: rows|cols (the selection's rows or columns; a block of cells groups rows by default), sheet?}", - has_doc, - |s, p| group(s, p, true) - ), - cmd!("data.ungroup", "Ungroup", ["Data", "Outline"], Some("Alt+Shift+Left"), "{rows? | cols? | by?: rows|cols, sheet?}", has_doc, |s, p| { - group(s, p, false) - }), + CommandSpec { + shortcut_mac: Some("Cmd+Shift+K"), + ..cmd!( + "data.group", + "Group", + ["Data", "Outline"], + Some("Alt+Shift+Right"), + "{rows?: \"2:5\" | cols?: \"B:C\" | by?: rows|cols (the selection's rows or columns; a block of cells groups rows by default), sheet?}", + has_doc, + |s, p| group(s, p, true) + ) + }, + CommandSpec { + shortcut_mac: Some("Cmd+Shift+J"), + ..cmd!( + "data.ungroup", + "Ungroup", + ["Data", "Outline"], + Some("Alt+Shift+Left"), + "{rows? | cols? | by?: rows|cols, sheet?}", + has_doc, + |s, p| { group(s, p, false) } + ) + }, cmd!( "data.hideDetail", "Hide Detail", diff --git a/crates/engine/src/cmd/data/filter.rs b/crates/engine/src/cmd/data/filter.rs index 71aa698f4..1b52e4537 100644 --- a/crates/engine/src/cmd/data/filter.rs +++ b/crates/engine/src/cmd/data/filter.rs @@ -1,5 +1,8 @@ use super::sort::sort_key; -use super::*; +use super::{ + AutoFilter, CellRef, EngineError, FilterCriterion, Json, Ordering, RangeRef, Result, Session, Sheet, Value, Workbook, bad, bool_param, col_param, + current_region, data_target, edit, json, +}; // ---------------------------------------------------------------- filter diff --git a/crates/engine/src/cmd/data/grouping.rs b/crates/engine/src/cmd/data/grouping.rs index 7b5a2254c..d2e183ae1 100644 --- a/crates/engine/src/cmd/data/grouping.rs +++ b/crates/engine/src/cmd/data/grouping.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + CellRef, EngineError, Json, LineInfo, RangeRef, Result, Session, bad, bool_param, col_param, data_target, edit, json, outline, parse_lines, + str_param, target_sheet, +}; // ---------------------------------------------------------------- outline diff --git a/crates/engine/src/cmd/data/sort.rs b/crates/engine/src/cmd/data/sort.rs index 919de1734..491e3d51e 100644 --- a/crates/engine/src/cmd/data/sort.rs +++ b/crates/engine/src/cmd/data/sort.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + Anchor, AnchorMode, Arc, Cell, CellRef, Comment, EngineError, Formula, Hyperlink, Json, LineInfo, Ordering, RangeRef, Result, Session, Sheet, + Value, bad, bool_param, col_param, data_range, edit, guess_header, json, ok, sort_compare, split_sheet, str_param, target_sheet, +}; #[derive(Clone)] struct SortKey { @@ -192,8 +195,7 @@ fn do_sort(s: &mut Session, sheet: usize, r: RangeRef, header: bool, keys: Vec = - sh.images.iter().map(|o| o.anchor).chain(sh.charts.iter().map(|o| o.anchor)).chain(sh.shapes.iter().map(|o| o.anchor)).collect(); + let objs: Vec = sh.object_anchors().copied().collect(); let descs: Vec<(u32, u32, f32, f32)> = objs .iter() .map(|a| { @@ -239,33 +241,22 @@ fn do_sort(s: &mut Session, sheet: usize, r: RangeRef, header: bool, keys: Vec o1 { - continue; - } - let na = sorted_anchor(shm, a, desc, by_cols, b0, b1, &dst_of); - if na != a { - if i < ni { - if let Some(o) = shm.images.get_mut(i) { - o.anchor = na; - } - } else if i < ni + nch { - if let Some(o) = shm.charts.get_mut(i - ni) { - o.anchor = na; - } - } else if let Some(o) = shm.shapes.get_mut(i - ni - nch) { - o.anchor = na; - } - } + let new_anchors: Vec = objs + .into_iter() + .zip(descs) + .map(|(a, desc)| { + let other = if by_cols { a.cell.row } else { a.cell.col }; + if other < o0 || other > o1 { a } else { sorted_anchor(shm, a, desc, by_cols, b0, b1, &dst_of) } + }) + .collect(); + for (a, n) in shm.object_anchors_mut().zip(new_anchors) { + *a = n; } for (c, _) in moved { cx.changed.push((sheet, c)); @@ -281,7 +272,9 @@ fn do_sort(s: &mut Session, sheet: usize, r: RangeRef, header: bool, keys: Vec Anchor { if a.mode == AnchorMode::Absolute { return a; @@ -299,7 +292,10 @@ fn sorted_anchor(sh: &Sheet, a: Anchor, bottom: (u32, u32, f32, f32), by_cols: b let y1 = sh.row_top(nr) + a.dy as f64; let x2 = sh.col_left(nbc) + tdx as f64; let y2 = sh.row_top(nbr) + tdy as f64; - Anchor { cell: CellRef::new(nr, nc), dx: a.dx, dy: a.dy, width: (x2 - x1).max(0.0) as f32, height: (y2 - y1).max(0.0) as f32, mode: a.mode } + // An extent that is not positive means the sort put the bottom-right line before the + // top-left one on that axis: the object keeps its old size there. + let fit = |extent: f64, old: f32| if extent > 0.0 { extent as f32 } else { old.max(0.0) }; + Anchor { cell: CellRef::new(nr, nc), dx: a.dx, dy: a.dy, width: fit(x2 - x1, a.width), height: fit(y2 - y1, a.height), mode: a.mode } } else { Anchor { cell: CellRef::new(nr, nc), ..a } } diff --git a/crates/engine/src/cmd/data/tools.rs b/crates/engine/src/cmd/data/tools.rs index 1603b2cb4..c4ae43ef4 100644 --- a/crates/engine/src/cmd/data/tools.rs +++ b/crates/engine/src/cmd/data/tools.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + Cell, CellRef, EngineError, Json, RangeRef, Result, Session, bad, bool_param, cell_param_on, col_param, data_target, edit, guess_header, json, + ok, str_param, target_range, target_sheet, +}; // ---------------------------------------------------------------- tools @@ -143,7 +146,16 @@ pub(super) fn text_to_columns(s: &mut Session, p: &Json) -> Result { } } for (c, t) in out { - let cell = crate::cmd::edit::input_to_cell(&t, sheet, c, &mut cx.wb).unwrap_or(None); + let cell = match crate::cmd::edit::input_to_cell(&t, sheet, c, &mut cx.wb) { + Ok(cell) => cell, + // `=PY(…)` doesn't make a Python cell (that takes python.set): the piece is kept + // as text rather than lost. + Err(_) if gridcraft_model::python::is_py_formula(&t) => { + let style = cx.wb.sheet(sheet).map(|sh| sh.style_id(c)).unwrap_or_default(); + Some(Cell { value: gridcraft_core::Value::text(t.as_str()), formula: None, style, py: None }) + } + Err(_) => None, + }; let shm = cx.sheet_mut(sheet)?; match cell { Some(x) => shm.set_cell(c, x), diff --git a/crates/engine/src/cmd/data/validation.rs b/crates/engine/src/cmd/data/validation.rs index 6d4ae73cf..ce0d7e3b8 100644 --- a/crates/engine/src/cmd/data/validation.rs +++ b/crates/engine/src/cmd/data/validation.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + Cell, CellRef, CfOperator, EngineError, ErrorStyle, Json, RangeRef, Result, Session, Validation, ValidationKind, Value, Workbook, bad, + bool_param, cell_param, edit, json, ok, str_param, target_ranges, target_sheet, +}; // ---------------------------------------------------------------- validation diff --git a/crates/engine/src/cmd/draw.rs b/crates/engine/src/cmd/draw.rs index a86be3099..c8921718c 100644 --- a/crates/engine/src/cmd/draw.rs +++ b/crates/engine/src/cmd/draw.rs @@ -1,10 +1,10 @@ //! Draw tab: pen and eraser tools, ink strokes, Ink to Shape. use gridcraft_core::CellRef; -use gridcraft_model::*; +use gridcraft_model::{Anchor, Color, Shape, ShapeKind}; use serde_json::{Value as Json, json}; -use super::*; +use super::{CommandSpec, EngineError, Result, Session, bad, bool_param, cmd, edit, f64_param, has_doc, ok, str_param, u32_param}; pub fn specs() -> Vec { vec![ @@ -80,6 +80,7 @@ fn stroke(s: &mut Session, p: &Json) -> Result { let text = pts.iter().map(|q| format!("{:.1},{:.1}", q[0] - minx, q[1] - miny)).collect::>().join(";"); let id = d.wb.next_object_id(); let shape = Shape { + math: Default::default(), id, kind: ShapeKind::Ink, anchor: Anchor { diff --git a/crates/engine/src/cmd/edit.rs b/crates/engine/src/cmd/edit.rs index e48fe8838..2aa9d8fe3 100644 --- a/crates/engine/src/cmd/edit.rs +++ b/crates/engine/src/cmd/edit.rs @@ -4,7 +4,10 @@ use gridcraft_core::{CellRef, MAX_ROWS, RangeRef, Value}; use gridcraft_model::{Cell, Formula, StyleId}; use serde_json::{Value as Json, json}; -use super::*; +use super::{ + CommandSpec, EngineError, Result, bad, bool_param, can_redo, can_undo, cell_param, cell_param_on, cmd, edit, f64_param, has_clipboard, has_doc, + locked_cell_error, ok, split_sheet, str_param, target_range, target_ranges, target_sheet, u32_param, +}; use crate::selection::{current_region, jump}; use crate::{Clipboard, Session}; @@ -17,7 +20,7 @@ mod paste_special; mod selection; use cell::{cell_set, range_fill, range_set_values}; -pub(crate) use cell::{check_arg_counts, input_to_cell, json_to_input, parse_typed_formula}; +pub(crate) use cell::{check_arg_counts, input_to_cell, is_formula_input, json_to_input, parse_typed_formula}; #[cfg(test)] pub(crate) use clipboard::parse_tsv; use clipboard::{clear_clipboard, copy, cut, paste}; @@ -32,7 +35,7 @@ use go_to::go_to_special; #[cfg(test)] use go_to::{GO_TO_MAX_AREAS, GO_TO_RECENT, reference_parts}; use paste_special::paste_special; -pub(crate) use paste_special::rewrite_all_formulas; +pub(crate) use paste_special::{rewrite_all_formulas, rewrite_cell_links, rewrite_python_refs}; use selection::{select_all, select_col, select_region, select_row, selection_move, selection_next, selection_set}; pub fn specs() -> Vec { @@ -43,7 +46,7 @@ pub fn specs() -> Vec { "Set Values", [], None, - "{range?: \"A1\", values: [[...]] (rows of numbers/strings/bools/null; strings starting with = are formulas), parse?: bool (default true: a string goes through typed input, so \"05.04.2026\" becomes a date and \"2.500\" a number, by en-US rules; false stores every other string as the text it is, so a dd.mm.yyyy date or a thousands separator survives)}", + "{range?: \"A1\", values: [[...]] (rows of numbers/strings/bools/null; strings starting with = are formulas), parse?: bool (default true: a string goes through typed input, so \"05.04.2026\" becomes a date and \"2.500\" a number, by en-US rules; false stores every other string as the text it is, so a dd.mm.yyyy date or a thousands separator survives, a leading ' is kept and an empty string clears the cell)} → {cells, convertedCount, converted: [{cell, input, stored}] (up to 20 of the strings typed input read as a number, date, time, boolean or error, so a script can notice)}", has_doc, range_set_values ), @@ -78,15 +81,18 @@ pub fn specs() -> Vec { has_doc, paste ), - cmd!( - "edit.pasteSpecial", - "Paste Special…", - ["Home", "Clipboard"], - Some("Ctrl+Cmd+V"), - "{what: all|formulas|values|formats|comments|validation|allExceptBorders|columnWidths|formulasAndNumberFormats|valuesAndNumberFormats, operation?: none|add|subtract|multiply|divide, skipBlanks?, transpose?, link?}", - has_clipboard, - paste_special - ), + CommandSpec { + shortcut_mac: Some("Ctrl+Cmd+V"), + ..cmd!( + "edit.pasteSpecial", + "Paste Special…", + ["Home", "Clipboard"], + Some("Ctrl+Alt+V"), + "{what: all|formulas|values|formats|comments|validation|allExceptBorders|columnWidths|formulasAndNumberFormats|valuesAndNumberFormats, operation?: none|add|subtract|multiply|divide, skipBlanks?, transpose?, link?}", + has_clipboard, + paste_special + ) + }, cmd!(noundo "edit.clearClipboard", "Cancel Copy", [], Some("Escape"), "{}", has_doc, clear_clipboard), cmd!("edit.clearAll", "Clear All", ["Home", "Editing", "Clear"], None, "{range?}", has_doc, |s, p| clear(s, p, "all")), cmd!("edit.clearFormats", "Clear Formats", ["Home", "Editing", "Clear"], None, "{range?}", has_doc, |s, p| clear(s, p, "formats")), @@ -126,7 +132,7 @@ pub fn specs() -> Vec { "Replace…", ["Home", "Editing", "Find & Select"], Some("Ctrl+H"), - "{what, with, matchCase?, wholeCell?, all?: true, within?: sheet|workbook}", + "{what, with, matchCase?, wholeCell?, all?: true, within?: sheet|workbook} → {replaced, skipped?} (in a Python cell the code is edited; skipped counts Python cells whose code would no longer fit)", has_doc, replace ), @@ -180,6 +186,8 @@ fn undo(s: &mut Session, p: &Json) -> Result { label = e.label.clone(); d.wb = e.wb; d.selection = e.selection; + // Step by step: a rename and a new sheet undone together match up only one step apart. + super::extra::keep_view_modes(d, &cur_wb); crate::push_history(&mut d.redo, label.clone(), cur_wb, &d.wb, cur_selection); } super::extra::keep_watch_list(&now, &mut d.wb); @@ -200,6 +208,7 @@ fn redo(s: &mut Session, p: &Json) -> Result { label = e.label.clone(); d.wb = e.wb; d.selection = e.selection; + super::extra::keep_view_modes(d, &cur_wb); crate::push_history(&mut d.undo, label.clone(), cur_wb, &d.wb, cur_selection); } super::extra::keep_watch_list(&now, &mut d.wb); @@ -210,6 +219,13 @@ fn redo(s: &mut Session, p: &Json) -> Result { // ---------------------------------------------------------------- clear +/// The rows of `range` an applied filter hides (rows hidden by hand are not among them). +fn filtered_rows(sh: &gridcraft_model::Sheet, range: RangeRef) -> std::collections::BTreeSet { + sh.rows.range(range.start.row..=range.end.row).filter(|(r, info)| info.hidden && sh.is_row_filtered(**r)).map(|(r, _)| *r).collect() +} + +/// The Clear commands (Delete is Clear Contents). As in Excel, the rows an applied filter hides +/// are left as they are; rows hidden by hand are cleared. fn clear(s: &mut Session, p: &Json, what: &str) -> Result { let sheet = target_sheet(s, p)?; let ranges = target_ranges(s, p)?; @@ -220,7 +236,10 @@ fn clear(s: &mut Session, p: &Json, what: &str) -> Result { d.wb.sheet(sheet).is_some_and(|sh| { ranges.iter().any(|r| { r.intersection(&sh.used_range().unwrap_or(RangeRef::cell(r.start))).is_some_and(|r| { - sh.cells.iter_range(r).any(|(c, x)| x.value == Value::Bool(true) && super::extra::is_checkbox_cell(&d.wb, sh, c)) + let skip = filtered_rows(sh, r); + sh.cells + .iter_range(r) + .any(|(c, x)| !skip.contains(&c.row) && x.value == Value::Bool(true) && super::extra::is_checkbox_cell(&d.wb, sh, c)) }) }) }) @@ -234,8 +253,12 @@ fn clear(s: &mut Session, p: &Json, what: &str) -> Result { }, None => continue, }; - let cells: Vec<(CellRef, Cell)> = - cx.wb.sheet(sheet).map(|sh| sh.cells.iter_range(r).map(|(c, x)| (c, x.clone())).collect()).unwrap_or_default(); + let skip = cx.wb.sheet(sheet).map(|sh| filtered_rows(sh, r)).unwrap_or_default(); + let cells: Vec<(CellRef, Cell)> = cx + .wb + .sheet(sheet) + .map(|sh| sh.cells.iter_range(r).filter(|(c, _)| !skip.contains(&c.row)).map(|(c, x)| (c, x.clone())).collect()) + .unwrap_or_default(); // Report every stored cell we touch, even beyond a million coordinates in a sparse // selection. Removed formula anchors must also reach calc so their spills disappear. cx.changed.extend(cells.iter().map(|(c, _)| (sheet, *c))); @@ -244,6 +267,7 @@ fn clear(s: &mut Session, p: &Json, what: &str) -> Result { for (_, cell) in cells.iter_mut() { let checkbox = cx.wb.styles.get(cell.style).num_fmt.as_str() == "checkbox"; cell.formula = None; + cell.py = None; if checkbox && uncheck_only { cell.value = Value::Bool(false); } else { @@ -270,16 +294,17 @@ fn clear(s: &mut Session, p: &Json, what: &str) -> Result { _ => {} } } + let cleared = |c: CellRef| r.contains(c) && !skip.contains(&c.row); if matches!(what, "all" | "comments") { - sh.comments.retain(|c, _| !r.contains(*c)); + sh.comments.retain(|c, _| !cleared(*c)); } if matches!(what, "all" | "hyperlinks") { - sh.hyperlinks.retain(|c, _| !r.contains(*c)); + sh.hyperlinks.retain(|c, _| !cleared(*c)); } if matches!(what, "all" | "formats") { let mut unmerged = false; sh.merges.retain(|m| { - let keep = !r.contains_range(m); + let keep = !r.contains_range(m) || skip.range(m.start.row..=m.end.row).next().is_some(); unmerged |= !keep; keep }); diff --git a/crates/engine/src/cmd/edit/cell.rs b/crates/engine/src/cmd/edit/cell.rs index 62a6b38bd..610135682 100644 --- a/crates/engine/src/cmd/edit/cell.rs +++ b/crates/engine/src/cmd/edit/cell.rs @@ -1,25 +1,40 @@ -use super::*; +use super::{ + Cell, CellRef, EngineError, Formula, Json, MAX_ROWS, RangeRef, Result, Session, StyleId, Value, bad, bool_param, cell_param_on, edit, json, + locked_cell_error, str_param, target_range, target_ranges, target_sheet, +}; // ---------------------------------------------------------------- entry +/// Typing `=PY(` doesn't make a Python cell; this says what does. +pub(crate) const PY_FORMULA: &str = "Python cells are made with the python.set command (the code as `code`), not by typing =PY(…)."; + +/// Whether typing `input` into a cell enters a formula: `=…`, or `+…`/`-…` followed by a name or +/// a parenthesis (not a number), as Excel takes `+SUM(1)`. +pub(crate) fn is_formula_input(input: &str) -> bool { + input.starts_with('=') + || (input.len() > 1 + && (input.starts_with('+') || input.starts_with('-')) + && gridcraft_core::parse::parse_number_text(input).is_none() + && input.chars().nth(1).is_some_and(|c| c.is_ascii_alphabetic() || c == '(')) +} + /// What typing `input` into a cell produces (constant or formula, plus an automatic number /// format). Inherits the destination's cell, row or column style, including its number /// format, before parsing. Errors when the formula can't be parsed. pub(crate) fn input_to_cell(input: &str, sheet: usize, at: CellRef, wb: &mut gridcraft_model::Workbook) -> Result> { let style = wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(); if input.is_empty() { - let c = Cell { value: Value::Empty, formula: None, style }; + let c = Cell { value: Value::Empty, formula: None, style, py: None }; return Ok(if c.is_blank() { None } else { Some(c) }); } let fmt_is_text = wb.styles.get(style).num_fmt.as_str() == "@"; if fmt_is_text { - return Ok(Some(Cell { value: Value::text(input), formula: None, style })); + return Ok(Some(Cell { value: Value::text(input), formula: None, style, py: None })); + } + let is_formula = is_formula_input(input); + if is_formula && gridcraft_model::python::is_py_formula(input) { + return Err(EngineError::Other(PY_FORMULA.into())); } - let is_formula = input.starts_with('=') - || (input.len() > 1 - && (input.starts_with('+') || input.starts_with('-')) - && gridcraft_core::parse::parse_number_text(input).is_none() - && input.chars().nth(1).is_some_and(|c| c.is_ascii_alphabetic() || c == '(')); if is_formula { let body = input.strip_prefix('=').unwrap_or(input); let expr = wb.name_call_case(parse_typed_formula(body)?); @@ -71,7 +86,7 @@ pub(crate) fn input_to_cell(input: &str, sheet: usize, at: CellRef, wb: &mut gri if input.contains('\n') && !wb.styles.get(style).align.wrap { style = wb.styles.derive(style, |s| s.align.wrap = true); } - Ok(Some(Cell { value: parsed.value, formula: None, style })) + Ok(Some(Cell { value: parsed.value, formula: None, style, py: None })) } /// Parses a formula as it is typed (without the `=`). Excel closes missing parentheses for you, @@ -139,7 +154,7 @@ pub(super) fn cell_set(s: &mut Session, p: &Json) -> Result { let sh = cx.sheet_mut(sheet)?; // The array takes the whole range: other cells in it lose their contents. for c in range.iter().skip(1).take(1_000_000) { - if let Some(old) = sh.cells.get(c).filter(|x| !x.value.is_empty() || x.formula.is_some()) { + if let Some(old) = sh.cells.get(c).filter(|x| x.has_content()) { let style = old.style; sh.set_cell(c, Cell { style, ..Cell::default() }); } @@ -180,6 +195,22 @@ pub(crate) fn json_to_input(v: &Json) -> String { } } +/// How many converted strings `range.setValues` reports back, and how much of each input and +/// stored text: the report is a diagnostic, so a big write must not put a second copy of its data +/// in the result. +const MAX_REPORTED_CONVERSIONS: usize = 20; +const MAX_REPORTED_CHARS: usize = 32; + +/// `text` cut to [`MAX_REPORTED_CHARS`] characters for the conversion report, with `…` when it was +/// cut. Counting characters keeps a multi-byte input whole. +fn clipped(text: &str) -> String { + let mut out: String = text.chars().take(MAX_REPORTED_CHARS).collect(); + if out.len() < text.len() { + out.push('…'); + } + out +} + pub(super) fn range_set_values(s: &mut Session, p: &Json) -> Result { let sheet = target_sheet(s, p)?; let start = target_range(s, p)?.start; @@ -197,6 +228,10 @@ pub(super) fn range_set_values(s: &mut Session, p: &Json) -> Result { return Err(bad("range.setValues", "too many rows")); } let mut count = 0; + // The strings typed input read as something other than the text given, for the result: a + // caller can notice a "05.04.2026" that became a date without reading the file back (#2255). + let mut converted: Vec<(CellRef, String)> = Vec::new(); + let mut converted_count = 0; edit(s, |cx| { for (ri, row) in rows.iter().enumerate() { let cols: Vec<&Json> = match row.as_array() { @@ -211,23 +246,47 @@ pub(super) fn range_set_values(s: &mut Session, p: &Json) -> Result { value: Value::number(n.as_f64().unwrap_or(0.0)), formula: None, style: cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(), + py: None, + }), + Json::Bool(b) => Some(Cell { + value: Value::Bool(*b), + formula: None, + style: cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(), + py: None, }), - Json::Bool(b) => { - Some(Cell { value: Value::Bool(*b), formula: None, style: cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default() }) - } // Text as given: no date or number reading, and a leading `'` is part of the // text rather than the marker that makes it text. A formula is still a // formula, which is what the ticket asks for and what `=` means in a cell. Json::String(text) if !parse && !text.starts_with('=') => { - let mut style = cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(); - // Line breaks still turn on Wrap Text, as typed input does. - if text.contains('\n') && !cx.wb.styles.get(style).align.wrap { - style = cx.wb.styles.derive(style, |s| s.align.wrap = true); + if text.is_empty() { + // An empty string clears the cell, as `null` and the default path do: + // an empty text cell is not blank, and a script clearing cells with + // `""` would leave cells `COUNTA` counts. The cell keeps its formatting. + input_to_cell("", sheet, at, &mut cx.wb)? + } else { + let mut style = cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(); + // Line breaks still turn on Wrap Text, as typed input does. + if text.contains('\n') && !cx.wb.styles.get(style).align.wrap { + style = cx.wb.styles.derive(style, |s| s.align.wrap = true); + } + Some(Cell { value: Value::Text(text.as_str().into()), formula: None, style, py: None }) } - Some(Cell { value: Value::Text(text.as_str().into()), formula: None, style }) } _ => input_to_cell(&input, sheet, at, &mut cx.wb)?, }; + // A string that became a formula was written as one (`=`), which is deliberate; + // only a number (dates and times included), boolean or error is a reading of the + // string and worth reporting. + if let Some(c) = &cell + && matches!(v, Json::String(_)) + && c.formula.is_none() + && matches!(c.value, Value::Number(_) | Value::Bool(_) | Value::Error(_)) + { + converted_count += 1; + if converted.len() < MAX_REPORTED_CONVERSIONS { + converted.push((at, input.clone())); + } + } let sh = cx.sheet_mut(sheet)?; match cell { Some(c) => sh.set_cell(at, c), @@ -242,7 +301,17 @@ pub(super) fn range_set_values(s: &mut Session, p: &Json) -> Result { } Ok(()) })?; - Ok(json!({"cells": count})) + let d = s.doc()?; + let reported: Vec = + d.wb.sheet(sheet) + .map(|sh| { + converted + .iter() + .map(|(at, input)| json!({"cell": at.a1(), "input": clipped(input), "stored": clipped(&crate::display::cell_text(&d.wb, sh, *at))})) + .collect() + }) + .unwrap_or_default(); + Ok(json!({"cells": count, "convertedCount": converted_count, "converted": reported})) } pub(super) fn range_fill(s: &mut Session, p: &Json) -> Result { diff --git a/crates/engine/src/cmd/edit/clipboard.rs b/crates/engine/src/cmd/edit/clipboard.rs index 60c05c687..3a1cd0739 100644 --- a/crates/engine/src/cmd/edit/clipboard.rs +++ b/crates/engine/src/cmd/edit/clipboard.rs @@ -1,6 +1,8 @@ use super::cell::input_to_cell; use super::paste_special::paste_special; -use super::*; +use super::{ + Cell, Clipboard, EngineError, Json, RangeRef, Result, Session, Value, cell_param_on, edit, json, ok, str_param, target_ranges, target_sheet, +}; // ---------------------------------------------------------------- clipboard @@ -90,7 +92,7 @@ fn slide_together(wb: &gridcraft_model::Workbook, sheet: usize, areas: &[RangeRe for c in cells { let Some(to) = c.offset(dr, dc) else { continue }; let style = src.cell(c).map(|x| x.style).unwrap_or_default(); - dst.set_cell(to, Cell { value: src.value(c), formula: None, style }); + dst.set_cell(to, Cell { value: src.value(c), formula: None, style, py: None }); if let Some(picture) = src.cell_pictures.get(&c) { dst.cell_pictures.insert(to, picture.clone()); } diff --git a/crates/engine/src/cmd/edit/fill.rs b/crates/engine/src/cmd/edit/fill.rs index 84f4cd6d3..8cd1e3d88 100644 --- a/crates/engine/src/cmd/edit/fill.rs +++ b/crates/engine/src/cmd/edit/fill.rs @@ -1,4 +1,4 @@ -use super::*; +use super::{CellRef, EngineError, Json, RangeRef, Result, Session, bad, edit, f64_param, json, str_param, target_range, target_sheet}; // ---------------------------------------------------------------- fill @@ -78,7 +78,7 @@ pub(super) fn fill_series(s: &mut Session, p: &Json) -> Result { let stop = f64_param(p, "stop"); let unit = str_param(p, "dateUnit").unwrap_or("day").to_string(); edit(s, |cx| { - crate::fill::series(cx, sheet, r, rows, &kind, step, stop, &unit)?; + crate::fill::series(cx, sheet, crate::fill::SeriesOptions { range: r, rows, kind: &kind, step, stop, unit: &unit })?; Ok(Json::Null) }) } diff --git a/crates/engine/src/cmd/edit/find.rs b/crates/engine/src/cmd/edit/find.rs index 2dcbfa886..0d4656320 100644 --- a/crates/engine/src/cmd/edit/find.rs +++ b/crates/engine/src/cmd/edit/find.rs @@ -1,5 +1,5 @@ use super::cell::input_to_cell; -use super::*; +use super::{Cell, CellRef, EngineError, Json, Result, Session, Value, bad, bool_param, edit, json, str_param}; // ---------------------------------------------------------------- find & replace @@ -115,6 +115,7 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result { let all = bool_param(p, "all").unwrap_or(true); let workbook = str_param(p, "within") == Some("workbook"); let active = s.doc()?.selection.active; + let mut skipped = 0usize; let n = edit(s, |cx| { let sheets: Vec = if workbook { (0..cx.wb.sheets.len()).collect() } else { vec![cx.wb.active_sheet()] }; let mut n = 0; @@ -135,6 +136,30 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result { } else { replace_ci(&text, &what, &with) }; + // A Python cell's text is its code: the replacement edits the code and the cell + // stays a Python cell. Code that would no longer fit in a PY formula is left as it was. + let old_py = cx.wb.sheet(si).and_then(|sh| sh.cell(c)).and_then(|x| x.py.clone()); + if let Some(old) = old_py { + let py = gridcraft_model::PyCode { object: old.object, ..gridcraft_model::PyCode::new(&new) }; + if py.code == old.code { + continue; + } + if !py.fits() { + skipped += 1; + continue; + } + // Its old spill is left to the recalculation, which removes it and + // recalculates what read it. + let sh = cx.sheet_mut(si)?; + let style = sh.style_id(c); + sh.set_cell( + c, + Cell { value: Value::Error(gridcraft_core::CellError::Blocked), formula: None, style, py: Some(std::sync::Arc::new(py)) }, + ); + cx.touch(si, c); + n += 1; + continue; + } let cell = input_to_cell(&new, si, c, &mut cx.wb).unwrap_or_else(|_| Some(Cell::value(Value::text(new.as_str())))); let sh = cx.sheet_mut(si)?; match cell { @@ -149,6 +174,9 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result { } Ok(n) })?; + if skipped > 0 { + return Ok(json!({"replaced": n, "skipped": skipped})); + } Ok(json!({"replaced": n})) } diff --git a/crates/engine/src/cmd/edit/go_to.rs b/crates/engine/src/cmd/edit/go_to.rs index b94bcd221..0cd7a3de9 100644 --- a/crates/engine/src/cmd/edit/go_to.rs +++ b/crates/engine/src/cmd/edit/go_to.rs @@ -1,5 +1,5 @@ use super::selection::{select_region, selection_set}; -use super::*; +use super::{CellRef, EngineError, Json, RangeRef, Result, Session, bad, bool_param, coalesce, json, split_sheet, str_param}; /// How many references the Go To dialog keeps for going back to. pub(super) const GO_TO_RECENT: usize = 10; @@ -174,8 +174,8 @@ pub(super) fn go_to_special(s: &mut Session, p: &Json) -> Result { let v = sheet.value(c); let ok = match kind { "blanks" => v.is_empty(), - "constants" => cell.is_some_and(|x| x.formula.is_none() && !x.value.is_empty()), - "formulas" => cell.is_some_and(|x| x.formula.is_some()), + "constants" => cell.is_some_and(|x| x.formula.is_none() && x.py.is_none() && !x.value.is_empty()), + "formulas" => cell.is_some_and(|x| x.formula.is_some() || x.py.is_some()), "comments" | "notes" => sheet.comments.contains_key(&c), "errors" => v.is_error(), "numbers" => v.is_number(), diff --git a/crates/engine/src/cmd/edit/paste_special.rs b/crates/engine/src/cmd/edit/paste_special.rs index eba1b429e..2477dc1a6 100644 --- a/crates/engine/src/cmd/edit/paste_special.rs +++ b/crates/engine/src/cmd/edit/paste_special.rs @@ -1,5 +1,76 @@ use super::clipboard::What; -use super::*; +use super::{ + Cell, CellRef, EngineError, Formula, Json, RangeRef, Result, Session, StyleId, Value, bad, bool_param, cell_param_on, edit, json, str_param, + target_sheet, +}; + +fn tiled_cells(rows: u32, cols: u32, tiles_r: u32, tiles_c: u32) -> impl Iterator + Clone { + (0..tiles_r).flat_map(move |tr| (0..tiles_c).flat_map(move |tc| (0..rows).flat_map(move |r| (0..cols).map(move |c| (tr, tc, r, c))))) +} + +fn apply_operation(op: &str, old: &Cell, new: &mut Cell) { + let operand = new.value.clone(); + if let (Ok(x), Ok(y)) = (old.value.to_number(), operand.to_number()) + && (old.value.is_number() || old.value.is_empty() || old.value.is_text()) + && (operand.is_number() || operand.is_empty()) + { + new.value = match op { + "add" => Value::number(x + y), + "subtract" => Value::number(x - y), + "multiply" => Value::number(x * y), + "divide" if y == 0.0 => Value::Error(gridcraft_core::CellError::Div0), + "divide" => Value::number(x / y), + _ => operand.clone(), + }; + } else if (old.value.is_text() || old.value.is_error()) && (operand.is_number() || operand.is_empty()) { + new.value = old.value.clone(); + } + if let Some(formula) = &old.formula + && let Ok(y) = operand.to_number() + && (operand.is_number() || operand.is_empty()) + && let Some(symbol) = operation_symbol(op) + { + let text = format!("=({}){symbol}{}", formula.text, gridcraft_core::number_to_text(y)); + new.formula = Some(std::sync::Arc::new(Formula::new(&text))); + new.value = Value::Empty; + } +} + +fn operation_symbol(op: &str) -> Option { + match op { + "add" => Some('+'), + "subtract" => Some('-'), + "multiply" => Some('*'), + "divide" => Some('/'), + _ => None, + } +} + +fn copy_merges( + cx: &mut crate::cmd::Ctx, + dest_sheet: usize, + src_sheet: &gridcraft_model::Sheet, + src: RangeRef, + at: CellRef, + what: What, + transpose: bool, +) -> Result<()> { + if !matches!(what, What::All | What::Formats) || transpose { + return Ok(()); + } + let merges: Vec = src_sheet.merges.iter().filter(|merge| src.contains_range(merge)).copied().collect(); + for merge in merges { + let row_offset = at.row as i64 - src.start.row as i64; + let col_offset = at.col as i64 - src.start.col as i64; + if let (Some(start), Some(end)) = (merge.start.offset(row_offset, col_offset), merge.end.offset(row_offset, col_offset)) { + let moved = RangeRef::new(start, end); + let sheet = cx.sheet_mut(dest_sheet)?; + sheet.merges.retain(|existing| !existing.intersects(&moved)); + sheet.merges.push(moved); + } + } + Ok(()) +} pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result { let Some(clip) = s.clipboard.clone() else { return Err(EngineError::Other("Nothing to paste.".into())) }; @@ -74,209 +145,181 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result { } moved_formulas = taken; } - for tr in 0..tiles_r { - for tc in 0..tiles_c { - for r in 0..src.height() { - for c in 0..src.width() { - let sc = CellRef::new(src.start.row + r, src.start.col + c); - let (dr, dc) = if transpose { (c, r) } else { (r, c) }; - let Some(dest) = at.offset((tr * h + dr) as i64, (tc * w + dc) as i64) else { continue }; - let src_cell = if cut_move { - moved_formulas.iter().find(|(k, _)| *k == sc).map(|(_, v)| v.clone()) + let dest_name = cx.wb.sheet(dest_sheet).map(|sh| sh.name.clone()).unwrap_or_default(); + for (tr, tc, r, c) in tiled_cells(src.height(), src.width(), tiles_r, tiles_c) { + let sc = CellRef::new(src.start.row + r, src.start.col + c); + let (dr, dc) = if transpose { (c, r) } else { (r, c) }; + let Some(dest) = at.offset((tr * h + dr) as i64, (tc * w + dc) as i64) else { continue }; + let src_cell = if cut_move { moved_formulas.iter().find(|(k, _)| *k == sc).map(|(_, v)| v.clone()) } else { src_sheet.cell(sc).cloned() }; + let src_val = src_sheet.value(sc); + let src_picture = if cut_move { moved_pictures.get(&sc) } else { src_sheet.cell_pictures.get(&sc) }.cloned(); + if skip_blanks && src_picture.is_none() && src_val.is_empty() && src_cell.as_ref().is_none_or(|x| x.formula.is_none() && x.py.is_none()) { + continue; + } + let old = cx.wb.sheet(dest_sheet).and_then(|sh| sh.cell(dest)).cloned().unwrap_or_default(); + let old_picture = cx.wb.sheet(dest_sheet).and_then(|sh| sh.cell_pictures.get(&dest)).cloned(); + let mut new_picture = old_picture.clone(); + let mut new = old.clone(); + let shift = |cell: &Cell| -> Option> { + let f = cell.formula.as_ref()?; + let e = f.expr()?; + if cut_move { + // A moved formula still reads cells on its original sheet. Qualify + // those references before the workbook-wide move adjustment below. + let e = if dest_sheet != clip.sheet { + e.map(&mut |x| match x { + gridcraft_formula::Expr::Ref(mut r) if r.sheet == gridcraft_formula::SheetSel::Current => { + r.sheet = gridcraft_formula::SheetSel::Named(src_sheet_name.clone()); + gridcraft_formula::Expr::Ref(r) + } + other => other, + }) + } else { + e + }; + let mut moved = Formula::from_expr(e); + // The array range moves with its formula. + let (dr, dc) = (dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64); + moved.array = f.array.and_then(|a| Some(RangeRef::new(a.start.offset(dr, dc)?, a.end.offset(dr, dc)?))); + moved.dynamic = f.dynamic; + return Some(std::sync::Arc::new(moved)); + } + let moved = gridcraft_formula::adjust::shift_relative(e, dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64); + Some(std::sync::Arc::new(Formula { dynamic: f.dynamic, ..Formula::from_expr(moved) })) + }; + // A Python cell's code travels with it. Cut to another sheet, it keeps reading the + // cells its `xl()` references named on its original sheet, as a moved formula does: + // they are qualified with that sheet before the workbook-wide move adjustment below. + let moved_py = |cell: &Cell| -> Result>> { + let Some(py) = cell.py.clone() else { return Ok(None) }; + if !cut_move || dest_sheet == clip.sheet { + return Ok(Some(py)); + } + let qualified = gridcraft_model::python::rewrite_xl(&py.code, |reference| { + let e = gridcraft_formula::parse(reference).ok()?.map(&mut |x| match x { + gridcraft_formula::Expr::Ref(mut r) if r.sheet == gridcraft_formula::SheetSel::Current => { + r.sheet = gridcraft_formula::SheetSel::Named(src_sheet_name.clone()); + gridcraft_formula::Expr::Ref(r) + } + other => other, + }); + Some(gridcraft_formula::print(&e)) + }); + Ok(Some(match qualified { + Some(code) => { + let moved = gridcraft_model::PyCode { code, ..(*py).clone() }; + check_python_fits(&moved, &dest_name, dest)?; + std::sync::Arc::new(moved) + } + None => py, + })) + }; + if link { + let sheet_prefix = if dest_sheet == clip.sheet && same_doc { + String::new() + } else { + format!("{}!", gridcraft_formula::quote_sheet(&src_sheet_name)) + }; + new.formula = Some(std::sync::Arc::new(Formula::new(&format!("={sheet_prefix}{}", sc.a1())))); + new.value = Value::Empty; + new.py = None; + new_picture = None; + } else { + let sc_cell = src_cell.clone().unwrap_or_default(); + match what { + What::All | What::AllExceptBorders => { + new_picture = src_picture.clone(); + new.formula = shift(&sc_cell); + new.value = if new.formula.is_some() { Value::Empty } else { sc_cell.value.clone() }; + new.py = moved_py(&sc_cell)?; + let st = map_style(sc_cell.style, &mut cx.wb); + new.style = if what == What::AllExceptBorders { + let borders = cx.wb.styles.get(old.style).borders; + cx.wb.styles.derive(st, |s| s.borders = borders) } else { - src_sheet.cell(sc).cloned() + st }; - let src_val = src_sheet.value(sc); - let src_picture = if cut_move { moved_pictures.get(&sc) } else { src_sheet.cell_pictures.get(&sc) }.cloned(); - if skip_blanks && src_picture.is_none() && src_val.is_empty() && src_cell.as_ref().is_none_or(|x| x.formula.is_none()) { - continue; + } + What::Formulas | What::FormulasAndNumberFormats => { + new_picture = src_picture.clone(); + new.formula = shift(&sc_cell); + new.value = if new.formula.is_some() { Value::Empty } else { sc_cell.value.clone() }; + new.py = moved_py(&sc_cell)?; + if what == What::FormulasAndNumberFormats { + let nf = src_wb.styles.get(sc_cell.style).num_fmt.clone(); + new.style = cx.wb.styles.derive(old.style, |s| s.num_fmt = nf); } - let old = cx.wb.sheet(dest_sheet).and_then(|sh| sh.cell(dest)).cloned().unwrap_or_default(); - let old_picture = cx.wb.sheet(dest_sheet).and_then(|sh| sh.cell_pictures.get(&dest)).cloned(); - let mut new_picture = old_picture.clone(); - let mut new = old.clone(); - let shift = |cell: &Cell| -> Option> { - let f = cell.formula.as_ref()?; - let e = f.expr()?; - if cut_move { - // A moved formula still reads cells on its original sheet. Qualify - // those references before the workbook-wide move adjustment below. - let e = if dest_sheet != clip.sheet { - e.map(&mut |x| match x { - gridcraft_formula::Expr::Ref(mut r) if r.sheet == gridcraft_formula::SheetSel::Current => { - r.sheet = gridcraft_formula::SheetSel::Named(src_sheet_name.clone()); - gridcraft_formula::Expr::Ref(r) - } - other => other, - }) - } else { - e - }; - let mut moved = Formula::from_expr(e); - // The array range moves with its formula. - let (dr, dc) = (dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64); - moved.array = f.array.and_then(|a| Some(RangeRef::new(a.start.offset(dr, dc)?, a.end.offset(dr, dc)?))); - moved.dynamic = f.dynamic; - return Some(std::sync::Arc::new(moved)); - } - let moved = - gridcraft_formula::adjust::shift_relative(e, dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64); - Some(std::sync::Arc::new(Formula { dynamic: f.dynamic, ..Formula::from_expr(moved) })) - }; - if link { - let sheet_prefix = if dest_sheet == clip.sheet && same_doc { - String::new() - } else { - format!("{}!", gridcraft_formula::quote_sheet(&src_sheet_name)) - }; - new.formula = Some(std::sync::Arc::new(Formula::new(&format!("={sheet_prefix}{}", sc.a1())))); - new.value = Value::Empty; - new_picture = None; - } else { - let sc_cell = src_cell.clone().unwrap_or_default(); - match what { - What::All | What::AllExceptBorders => { - new_picture = src_picture.clone(); - new.formula = shift(&sc_cell); - new.value = if new.formula.is_some() { Value::Empty } else { sc_cell.value.clone() }; - let st = map_style(sc_cell.style, &mut cx.wb); - new.style = if what == What::AllExceptBorders { - let borders = cx.wb.styles.get(old.style).borders; - cx.wb.styles.derive(st, |s| s.borders = borders) - } else { - st - }; - } - What::Formulas | What::FormulasAndNumberFormats => { - new_picture = src_picture.clone(); - new.formula = shift(&sc_cell); - new.value = if new.formula.is_some() { Value::Empty } else { sc_cell.value.clone() }; - if what == What::FormulasAndNumberFormats { - let nf = src_wb.styles.get(sc_cell.style).num_fmt.clone(); - new.style = cx.wb.styles.derive(old.style, |s| s.num_fmt = nf); - } - } - What::Values | What::ValuesAndNumberFormats => { - new_picture = src_picture.clone(); - new.formula = None; - new.value = src_val.clone(); - if what == What::ValuesAndNumberFormats { - let nf = src_wb.styles.get(sc_cell.style).num_fmt.clone(); - new.style = cx.wb.styles.derive(old.style, |s| s.num_fmt = nf); - } - } - What::Formats => new.style = map_style(sc_cell.style, &mut cx.wb), - What::Comments | What::Validation | What::ColumnWidths => {} - } - // Operations combine numbers with what is already there. - if op != "none" - && matches!(what, What::All | What::Values | What::ValuesAndNumberFormats | What::Formulas) - && new.formula.is_none() - { - if old_picture.is_some() || new_picture.is_some() { - new_picture = None; - new.value = Value::Error(gridcraft_core::CellError::Value); - } - let a = old.value.clone(); - let b = new.value.clone(); - // Numbers stored as text take part (multiplying by 1 turns them into - // numbers); other text and errors stay as they are. - if let (Ok(x), Ok(y)) = (a.to_number(), b.to_number()) - && (a.is_number() || a.is_empty() || a.is_text()) - && (b.is_number() || b.is_empty()) - { - new.value = match op.as_str() { - "add" => Value::number(x + y), - "subtract" => Value::number(x - y), - "multiply" => Value::number(x * y), - "divide" => { - if y == 0.0 { - Value::Error(gridcraft_core::CellError::Div0) - } else { - Value::number(x / y) - } - } - _ => b.clone(), - }; - } else if (a.is_text() || a.is_error()) && (b.is_number() || b.is_empty()) { - new.value = a; - } - // A formula already there is kept and combined with the pasted number. - if let Some(f) = &old.formula - && let Ok(y) = b.to_number() - && (b.is_number() || b.is_empty()) - && let Some(sym) = match op.as_str() { - "add" => Some('+'), - "subtract" => Some('-'), - "multiply" => Some('*'), - "divide" => Some('/'), - _ => None, - } - { - let text = format!("=({}){sym}{}", f.text, gridcraft_core::number_to_text(y)); - new.formula = Some(std::sync::Arc::new(Formula::new(&text))); - new.value = Value::Empty; - } - } + } + What::Values | What::ValuesAndNumberFormats => { + new_picture = src_picture.clone(); + new.formula = None; + new.py = None; + new.value = src_val.clone(); + if what == What::ValuesAndNumberFormats { + let nf = src_wb.styles.get(sc_cell.style).num_fmt.clone(); + new.style = cx.wb.styles.derive(old.style, |s| s.num_fmt = nf); } - // The copied cell's validation rule replaces whatever rule the destination had. - if !link && matches!(what, What::All | What::AllExceptBorders | What::Validation) { - let rule = src_sheet.validations.iter().find(|dv| dv.ranges.iter().any(|r| r.contains(sc))).cloned(); - let sh = cx.sheet_mut(dest_sheet)?; - crate::cmd::data::remove_validation_cells(&mut sh.validations, &[RangeRef::new(dest, dest)]); - if let Some(mut dv) = rule { - // Relative references move with the rule, as pasted formulas do. - let first = dv.ranges.first().map_or(sc, |r| r.start); - dv.f1 = crate::cmd::data::move_rule_formula(&dv.f1, first, dest); - dv.f2 = dv.f2.map(|f| crate::cmd::data::move_rule_formula(&f, first, dest)); - dv.ranges = vec![RangeRef::new(dest, dest)]; - sh.validations.push(dv); - } - } - match what { - What::Comments => { - if let Some(cm) = src_sheet.comments.get(&sc).cloned() { - cx.sheet_mut(dest_sheet)?.comments.insert(dest, cm); - } - } - What::ColumnWidths => { - if let Some(info) = src_sheet.cols.get(&sc.col).copied() { - cx.sheet_mut(dest_sheet)?.cols.insert(dest.col, info); - } - } - What::All => { - if let Some(cm) = src_sheet.comments.get(&sc).cloned() { - cx.sheet_mut(dest_sheet)?.comments.insert(dest, cm); - } - if let Some(h) = src_sheet.hyperlinks.get(&sc).cloned() { - cx.sheet_mut(dest_sheet)?.hyperlinks.insert(dest, h); - } - cx.sheet_mut(dest_sheet)?.set_cell(dest, new); - } - _ => cx.sheet_mut(dest_sheet)?.set_cell(dest, new), - } - if let Some(picture) = new_picture { - cx.sheet_mut(dest_sheet)?.cell_pictures.insert(dest, picture); - } - cx.touch(dest_sheet, dest); + } + What::Formats => new.style = map_style(sc_cell.style, &mut cx.wb), + What::Comments | What::Validation | What::ColumnWidths => {} + } + // Operations combine numbers with what is already there. + if op != "none" + && matches!(what, What::All | What::Values | What::ValuesAndNumberFormats | What::Formulas) + && new.formula.is_none() + && new.py.is_none() + { + if old_picture.is_some() || new_picture.is_some() { + new_picture = None; + new.value = Value::Error(gridcraft_core::CellError::Value); + } + // Numbers stored as text participate; other text and errors stay as they are. + apply_operation(&op, &old, &mut new); + } + } + // The copied cell's validation rule replaces whatever rule the destination had. + if !link && matches!(what, What::All | What::AllExceptBorders | What::Validation) { + let rule = src_sheet.validations.iter().find(|dv| dv.ranges.iter().any(|r| r.contains(sc))).cloned(); + let sh = cx.sheet_mut(dest_sheet)?; + crate::cmd::data::remove_validation_cells(&mut sh.validations, &[RangeRef::new(dest, dest)]); + if let Some(mut dv) = rule { + // Relative references move with the rule, as pasted formulas do. + let first = dv.ranges.first().map_or(sc, |r| r.start); + dv.f1 = crate::cmd::data::move_rule_formula(&dv.f1, first, dest); + dv.f2 = dv.f2.map(|f| crate::cmd::data::move_rule_formula(&f, first, dest)); + dv.ranges = vec![RangeRef::new(dest, dest)]; + sh.validations.push(dv); + } + } + match what { + What::Comments => { + if let Some(cm) = src_sheet.comments.get(&sc).cloned() { + cx.sheet_mut(dest_sheet)?.comments.insert(dest, cm); } } + What::ColumnWidths => { + if let Some(info) = src_sheet.cols.get(&sc.col).copied() { + cx.sheet_mut(dest_sheet)?.cols.insert(dest.col, info); + } + } + What::All => { + if let Some(cm) = src_sheet.comments.get(&sc).cloned() { + cx.sheet_mut(dest_sheet)?.comments.insert(dest, cm); + } + if let Some(h) = src_sheet.hyperlinks.get(&sc).cloned() { + cx.sheet_mut(dest_sheet)?.hyperlinks.insert(dest, h); + } + cx.sheet_mut(dest_sheet)?.set_cell(dest, new); + } + _ => cx.sheet_mut(dest_sheet)?.set_cell(dest, new), } + if let Some(picture) = new_picture { + cx.sheet_mut(dest_sheet)?.cell_pictures.insert(dest, picture); + } + cx.touch(dest_sheet, dest); } // Merged areas inside the copied block travel with it. - if matches!(what, What::All | What::Formats) && !transpose { - let merges: Vec = src_sheet.merges.iter().filter(|m| src.contains_range(m)).copied().collect(); - for m in merges { - let dr = at.row as i64 - src.start.row as i64; - let dc = at.col as i64 - src.start.col as i64; - if let (Some(a), Some(b)) = (m.start.offset(dr, dc), m.end.offset(dr, dc)) { - let nm = RangeRef::new(a, b); - let sh = cx.sheet_mut(dest_sheet)?; - sh.merges.retain(|x| !x.intersects(&nm)); - sh.merges.push(nm); - } - } - } + copy_merges(cx, dest_sheet, src_sheet, src, at, what, transpose)?; if cut_move { // References elsewhere to the moved block now point to its new place. let dest_name = cx.wb.sheet(dest_sheet).map(|s| s.name.clone()).unwrap_or_default(); @@ -285,7 +328,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result { } else { gridcraft_formula::adjust::Edit::MoveToSheet { from: src, to_sheet: dest_name, to_row: at.row, to_col: at.col } }; - rewrite_all_formulas(&mut cx.wb, &src_sheet_name, &e); + rewrite_all_formulas(&mut cx.wb, &src_sheet_name, &e)?; cx.structural = true; } let end = at.offset_clamped((tiles_r * h) as i64 - 1, (tiles_c * w) as i64 - 1); @@ -309,7 +352,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result { } /// Applies a reference adjustment to every cell formula and defined name in the workbook. -pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &str, e: &gridcraft_formula::adjust::Edit) { +pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &str, e: &gridcraft_formula::adjust::Edit) -> Result<()> { use gridcraft_formula::adjust::Edit; // Rows and columns inserted or deleted on the target sheet move its legacy array ranges too // (a cut moves its arrays itself). @@ -348,4 +391,109 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: & n.formula = gridcraft_formula::print(&new); } } + rewrite_cell_links(wb, |host, expr| gridcraft_formula::adjust::adjust(expr, host, target, e)); + rewrite_python_refs(wb, &|host, expr| gridcraft_formula::adjust::adjust(expr, host, target, e)) +} + +/// Runs `f` over every reference a floating object holds (a chart's series values, categories, +/// name and bubble sizes, and the source block it was made from). Every edit that moves +/// references calls it: inserting and deleting lines and cells and cut-moves (through +/// [`rewrite_all_formulas`]), and renaming and deleting sheets. `f` maps a reference held by an +/// object on the sheet named `host` to its new form; text that does not parse is kept. A source +/// block that no longer exists is dropped. +pub(crate) fn rewrite_cell_links(wb: &mut gridcraft_model::Workbook, f: impl Fn(&str, gridcraft_formula::Expr) -> gridcraft_formula::Expr) { + for si in 0..wb.sheets.len() { + let Some(host) = wb.sheets.get(si).filter(|s| !s.charts.is_empty()).map(|s| s.name.clone()) else { continue }; + let Some(sh) = wb.sheet_mut(si) else { continue }; + let fix = |text: &str| -> String { + match gridcraft_formula::parse(text) { + Ok(expr) => gridcraft_formula::print(&f(&host, expr)), + Err(_) => text.to_string(), + } + }; + for ch in sh.charts.iter_mut() { + ch.source = ch.source.as_deref().map(fix).filter(|s| !s.contains("#REF!")); + for se in ch.series.iter_mut() { + se.values = fix(&se.values); + se.categories = se.categories.as_deref().map(fix); + se.name = se.name.as_deref().map(fix); + se.bubble_sizes = se.bubble_sizes.as_deref().map(fix); + } + } + } +} + +/// Rewrites the `xl("…")` literals in every Python cell with `f`, which maps a reference held on +/// sheet `host` to its new form. Every reference-moving edit (insert and delete of lines or cells, +/// cut-move, sheet rename and delete) runs it, as it adjusts formulas. A literal is only replaced +/// when the reference it names changed, and text that doesn't parse is kept. A cell's cached +/// result is kept while its references still name the same data (a rename, a move); a reference +/// the edit deleted (`#REF!`), or one that grew or shrank (rows or columns inserted or deleted +/// inside it), leaves the result stale, so the cell is new code (`#BLOCKED!`) and the +/// recalculation removes its spill. A rewrite that would +/// make a cell's code longer than one cell holds (a longer sheet name, `Z1` becoming `AA1`) refuses +/// the whole edit, which then changes nothing: see [`check_python_fits`]. +pub(crate) fn rewrite_python_refs( + wb: &mut gridcraft_model::Workbook, + f: &impl Fn(&str, gridcraft_formula::Expr) -> gridcraft_formula::Expr, +) -> Result<()> { + for si in 0..wb.sheets.len() { + let Some(sh) = wb.sheets.get(si) else { continue }; + let host = sh.name.clone(); + let mut new_code: Vec<(CellRef, String, bool)> = Vec::new(); + for (c, cell) in sh.cells.iter() { + let Some(py) = &cell.py else { continue }; + let mut lost = false; + let changed = gridcraft_model::python::rewrite_xl(&py.code, |reference| { + let expr = gridcraft_formula::parse(reference).ok()?; + let before = gridcraft_formula::print(&expr); + let after = gridcraft_formula::print(&f(&host, expr)); + lost |= (after.contains("#REF!") && !before.contains("#REF!")) || area_shape(&before) != area_shape(&after); + (after != before).then_some(after) + }); + if let Some(code) = changed { + check_python_fits(&gridcraft_model::PyCode { code: code.clone(), object: py.object, ..Default::default() }, &host, c)?; + new_code.push((c, code, lost)); + } + } + if new_code.is_empty() { + continue; + } + let Some(sh) = wb.sheet_mut(si) else { continue }; + for (c, code, lost) in new_code { + if let Some(cell) = sh.cells.get_mut(c) + && let Some(py) = cell.py.as_mut() + { + let py = std::sync::Arc::make_mut(py); + py.code = code; + if lost { + py.output = None; + cell.value = Value::Error(gridcraft_core::CellError::Blocked); + } + } + } + } + Ok(()) +} + +/// The rows and columns a printed reference covers (`A1:B3`, `Sheet2!$A$1`, `C:E`), or `None` +/// for any other form (a name, an error). +fn area_shape(reference: &str) -> Option<(u32, u32)> { + let r = RangeRef::parse(reference.rsplit('!').next()?)?; + Some((r.height(), r.width())) +} + +/// Refuses Python code an edit has made longer than one cell holds: the limit `python.set` +/// enforces and opening a file applies (code over it is dropped to its value on open). The edit +/// is refused rather than made, so no code is lost and none keeps a reference the edit moved. +pub(crate) fn check_python_fits(py: &gridcraft_model::PyCode, sheet: &str, at: CellRef) -> Result<()> { + if py.fits() { + return Ok(()); + } + Err(EngineError::Other(format!( + "The edit would make the Python code in {}!{} longer than Excel's {}-character formula limit, so nothing was changed. Shorten the code first.", + gridcraft_formula::quote_sheet(sheet), + at.a1(), + gridcraft_model::python::MAX_FORMULA_CHARS + ))) } diff --git a/crates/engine/src/cmd/edit/selection.rs b/crates/engine/src/cmd/edit/selection.rs index 95793f338..ede66a34d 100644 --- a/crates/engine/src/cmd/edit/selection.rs +++ b/crates/engine/src/cmd/edit/selection.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + CellRef, EngineError, Json, RangeRef, Result, Session, bool_param, cell_param, current_region, json, jump, split_sheet, str_param, target_ranges, + target_sheet, +}; // ---------------------------------------------------------------- selection diff --git a/crates/engine/src/cmd/edit/tests.rs b/crates/engine/src/cmd/edit/tests.rs index c9e1e7a93..3f3a2fb6d 100644 --- a/crates/engine/src/cmd/edit/tests.rs +++ b/crates/engine/src/cmd/edit/tests.rs @@ -192,6 +192,74 @@ fn set_values_can_store_strings_as_the_text_they_are() { assert!(d.wb.styles.get(sh.style_id(CellRef::parse("A9").unwrap())).align.wrap); } +/// The result names the strings typed input read as something else, so a script notices without +/// reading the file back (#2255). A formula and a leading apostrophe are not conversions. +#[test] +fn set_values_reports_the_strings_typed_input_read() { + let mut s = Session::new(); + s.new_workbook(); + let row = json!(["05.04.2026", "02/11/2026", "2.500", "4.25 %", "=1+1", "'keep"]); + let r = s.execute("range.setValues", json!({"range": "A1", "values": [row.clone()]})).unwrap(); + assert_eq!(r["cells"], 6, "{r}"); + assert_eq!(r["convertedCount"], 4, "{r}"); + assert_eq!(r["converted"][0], json!({"cell": "A1", "input": "05.04.2026", "stored": "5/4/2026"}), "{r}"); + assert_eq!(r["converted"][1], json!({"cell": "B1", "input": "02/11/2026", "stored": "2/11/2026"}), "{r}"); + assert_eq!(r["converted"][2], json!({"cell": "C1", "input": "2.500", "stored": "2.5"}), "{r}"); + assert_eq!(r["converted"][3], json!({"cell": "D1", "input": "4.25 %", "stored": "4.25%"}), "{r}"); + // The report is a diagnostic, not a second copy of the data: a long input is cut. + let long = s.execute("range.setValues", json!({"range": "J1", "values": [["9".repeat(100)]]})).unwrap(); + assert_eq!(long["convertedCount"], 1, "{long}"); + let input = long["converted"][0]["input"].as_str().unwrap_or_default(); + assert_eq!(input.chars().count(), 33, "32 characters and the ellipsis: {input}"); + assert!(input.ends_with('…'), "{input}"); + // Nothing is read with `parse: false`, so nothing is reported. + let verbatim = s.execute("range.setValues", json!({"range": "A3", "parse": false, "values": [row]})).unwrap(); + assert_eq!(verbatim["convertedCount"], 0, "{verbatim}"); + assert_eq!(verbatim["converted"], json!([]), "{verbatim}"); +} + +/// An empty string clears the cell with `parse: false` too, as `null` and the default path do: a +/// script clearing cells with "" must not leave cells `COUNTA` counts. +#[test] +fn set_values_parse_false_clears_an_empty_string() { + let mut s = Session::new(); + s.new_workbook(); + s.execute("range.setValues", json!({"range": "A1:B1", "values": [["keep", "keep"]]})).unwrap(); + s.execute("home.bold", json!({"range": "A1"})).unwrap(); + s.execute("range.setValues", json!({"range": "A1", "parse": false, "values": [["", ""]]})).unwrap(); + let get = |s: &mut Session, a: &str| s.execute("cell.get", json!({"cell": a})).unwrap(); + assert_eq!(get(&mut s, "A1")["type"], "empty"); + assert_eq!(get(&mut s, "B1")["type"], "empty"); + let d = s.doc().unwrap(); + let sh = d.wb.sheet(0).unwrap(); + assert!(d.wb.styles.get(sh.style_id(CellRef::parse("A1").unwrap())).font.bold, "clearing with \"\" keeps the formatting"); + assert_eq!(s.execute("formulas.evaluate", json!({"formula": "=COUNTA(A1:B1)"})).unwrap(), json!(0.0)); +} + +/// The text `parse: false` stores is still text after the workbook is saved and reopened, and the +/// default's reading of a date survives as a date (#2255). +#[test] +fn set_values_text_survives_an_xlsx_roundtrip() { + let mut s = Session::new(); + s.new_workbook(); + let values = json!([["05.04.2026", "2.500", "4.25 %", "=1+1", 3, true]]); + s.execute("range.setValues", json!({"range": "A1", "parse": false, "values": values})).unwrap(); + s.execute("range.setValues", json!({"range": "A3", "values": [["05.04.2026"]]})).unwrap(); + let bytes = crate::io::save_bytes(&s.doc().unwrap().wb, "book.xlsx").unwrap(); + let (wb, _) = crate::io::open_bytes("book.xlsx", &bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + for (col, want) in [(0, "05.04.2026"), (1, "2.500"), (2, "4.25 %")] { + assert_eq!(sh.value(CellRef::new(0, col)), Value::text(want), "column {col} is the text it was given"); + } + assert_eq!(sh.value(CellRef::new(0, 3)), Value::number(2.0), "=1+1 is still a formula"); + assert_eq!(sh.value(CellRef::new(0, 4)), Value::number(3.0)); + assert_eq!(sh.value(CellRef::new(0, 5)), Value::Bool(true)); + // The default read the date, and the file keeps it as one. + let at = CellRef::new(2, 0); + assert_eq!(crate::display::cell_text(&wb, sh, at), "5/4/2026"); + assert_eq!(wb.styles.get(sh.style_id(at)).num_fmt.as_str(), "m/d/yyyy"); +} + #[test] fn legacy_array_entry() { let mut s = Session::new(); diff --git a/crates/engine/src/cmd/equation.rs b/crates/engine/src/cmd/equation.rs new file mode 100644 index 000000000..d485a3535 --- /dev/null +++ b/crates/engine/src/cmd/equation.rs @@ -0,0 +1,305 @@ +//! Equations (Insert › Symbols › Equation): floating shapes holding LaTeX source, laid out by +//! `gridcraft-math` and drawn from the cached layout. +//! +//! An equation is drawn to fit its frame, so resizing the frame scales it. Inserting sizes the +//! frame for the requested font size; editing keeps the current scale and resizes the frame to +//! the new content. + +use gridcraft_math::{MathError, MathList}; +use gridcraft_model::{Anchor, AnchorMode, Color, EquationCache, Shape, ShapeKind}; +use serde_json::{Value as Json, json}; + +use super::{CommandSpec, EngineError, PROTECTED, Result, Session, bad, cell_param, cmd, edit, f64_param, has_doc, ok, str_param, u32_param}; + +pub fn specs() -> Vec { + vec![ + cmd!( + "insert.equation", + "Equation", + ["Insert", "Symbols"], + None, + "{latex: \"x^2\", cell?|at?: \"B2\", size?: 11 (points), color?: hex|\"auto\"}", + has_doc, + insert_equation + ), + cmd!("equation.edit", "Edit Equation", [], None, "{id, latex}", has_doc, edit_equation), + ] +} + +/// Default equation font size, in points. +pub const DEFAULT_SIZE_PT: f64 = 11.0; +/// Space between an equation's frame and its content, in sheet pixels. +pub const PADDING: f32 = 4.0; +/// Sheet pixels per point. +const PX_PER_PT: f64 = 96.0 / 72.0; + +/// The scale (sheet pixels per em) at which `list` fits a frame of `w` by `h` pixels. +pub fn fit_scale(list: &MathList, w: f32, h: f32) -> f32 { + let (cw, ch) = (list.width.max(0.01), list.total_height().max(0.01)); + let s = ((w - 2.0 * PADDING) / cw).min((h - 2.0 * PADDING) / ch); + if s.is_finite() { s.max(0.0) } else { 0.0 } +} + +/// The frame size for `list` at `scale` pixels per em. +fn frame_for(list: &MathList, scale: f32) -> (f32, f32) { + let w = list.width.max(0.5) * scale + 2.0 * PADDING; + let h = list.total_height().max(1.0) * scale + 2.0 * PADDING; + (w.clamp(8.0, 20_000.0), h.clamp(8.0, 20_000.0)) +} + +/// `latex` laid out, or a readable error with the position of the problem. +fn lay_out(cmd: &str, latex: &str) -> Result { + gridcraft_math::layout(latex).map_err(|e| bad(cmd, describe(latex, &e))) +} + +/// "Error at character 5: Undefined control sequence: \foo". +pub fn describe(latex: &str, e: &MathError) -> String { + match e.char_pos(latex) { + Some(p) => format!("equation error at character {p}: {}", e.message), + None => format!("equation error: {}", e.message), + } +} + +fn insert_equation(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "insert.equation"; + let latex = str_param(p, "latex").ok_or_else(|| bad(CMD, "`latex` must be the equation's LaTeX source, such as \"x^2 + y^2 = r^2\""))?; + if latex.trim().is_empty() { + return Err(bad(CMD, "`latex` is empty")); + } + let size = f64_param(p, "size").unwrap_or(DEFAULT_SIZE_PT); + if !(1.0..=409.0).contains(&size) { + return Err(bad(CMD, "`size` must be between 1 and 409 points")); + } + let color = match p.get("color") { + None | Some(Json::Null) => Color::Auto, + Some(c) => super::format::color_param(Some(c)).ok_or_else(|| bad(CMD, "`color` must be a hex colour such as \"#1F4E79\", or \"auto\""))?, + }; + let list = lay_out(CMD, latex)?; + let at = match cell_param(s, p, "cell")? { + Some(c) => Some(c), + None => cell_param(s, p, "at")?, + }; + let d = s.doc()?; + if d.wb.active().is_some_and(|sh| sh.is_protected()) { + return Err(EngineError::Other(PROTECTED.into())); + } + let at = at.unwrap_or(d.selection.active); + let id = d.wb.next_object_id(); + let (width, height) = frame_for(&list, (size * PX_PER_PT) as f32); + let mut shape = Shape { + id, + kind: ShapeKind::Equation, + anchor: Anchor { cell: at, dx: 0.0, dy: 0.0, width, height, mode: AnchorMode::MoveOnly }, + fill: Color::Auto, + line: color, + text: latex.to_string(), + format: Default::default(), + math: Default::default(), + }; + shape.math.refresh(latex); + let sheet = d.wb.active_sheet(); + edit(s, |cx| { + cx.sheet_mut(sheet)?.shapes.push(shape); + Ok(json!({"equation": id})) + }) +} + +fn edit_equation(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "equation.edit"; + let id = u32_param(p, "id").ok_or_else(|| bad(CMD, "`id` must be an equation id"))?; + let latex = str_param(p, "latex").ok_or_else(|| bad(CMD, "`latex` must be the equation's LaTeX source"))?; + if latex.trim().is_empty() { + return Err(bad(CMD, "`latex` is empty; delete the equation instead")); + } + let d = s.doc()?; + let sheet = d.wb.active_sheet(); + let sh = d.wb.active().ok_or_else(|| bad(CMD, "no such equation"))?; + let shape = sh.shapes.iter().find(|x| x.id == id && x.kind == ShapeKind::Equation).ok_or_else(|| bad(CMD, "no such equation"))?; + if sh.is_protected() { + return Err(EngineError::Other(PROTECTED.into())); + } + // The same source is a no-op, unless it has no layout yet (a file with more equations than + // are laid out at once): then OK lays it out. + if shape.text == latex && matches!(shape.math.get(latex), Some(Ok(_))) { + return ok(); + } + let list = lay_out(CMD, latex)?; + // Keep the size the equation is drawn at now; without a layout, use the default size. + let scale = match shape.math.get(&shape.text) { + Some(Ok(old)) => fit_scale(old, shape.anchor.width, shape.anchor.height), + _ => 0.0, + }; + let scale = if scale > 0.1 { scale } else { (DEFAULT_SIZE_PT * PX_PER_PT) as f32 }; + let (width, height) = frame_for(&list, scale); + edit(s, |cx| { + let shape = cx.sheet_mut(sheet)?.shapes.iter_mut().find(|x| x.id == id).ok_or_else(|| bad(CMD, "no such equation"))?; + shape.text = latex.to_string(); + shape.math = EquationCache::default(); + shape.math.refresh(latex); + shape.anchor.width = width; + shape.anchor.height = height; + Ok(json!({"equation": id})) + }) +} + +// ---------------------------------------------------------------- PDF + +/// The first [`gridcraft_math::MAX_SOURCE_BYTES`] bytes of an equation's source, cut at a char +/// boundary. A longer source never lays out (the editor refuses it, and a file can carry one of +/// megabytes), so what shows it as text, the sheet painter and the PDF fallback, shows this much +/// and does no work on the rest. +pub fn source_excerpt(text: &str) -> &str { + let cap = gridcraft_math::MAX_SOURCE_BYTES; + if text.len() <= cap { + return text; + } + let mut end = cap; + while end > 0 && !text.is_char_boundary(end) { + end -= 1; + } + text.get(..end).unwrap_or("") +} + +/// The PDF face for a glyph: Symbol for Greek letters and operators, else the Times (or +/// Helvetica, Courier) face for its style. `None` when no standard font can show `c`. +fn pdf_font(c: char, face: gridcraft_math::Face) -> Option { + use gridcraft_math::Face; + use gridcraft_pdf::Font; + let text = match face { + Face::Italic => Font::TimesItalic, + Face::Bold => Font::TimesBold, + Face::BoldItalic => Font::TimesBoldItalic, + Face::Sans => Font::Helvetica, + Face::SansBold => Font::HelveticaBold, + Face::SansItalic => Font::HelveticaOblique, + Face::Mono => Font::Courier, + Face::Roman | Face::Large(_) => Font::TimesRoman, + }; + // Letters and digits stay in the text face; everything Symbol has goes to Symbol, whose + // operators (minus, equals, plus) match the maths better than the text faces'. + if c.is_ascii_alphanumeric() { + return Some(text); + } + if Font::Symbol.can_encode(c) { + return Some(Font::Symbol); + } + text.can_encode(c).then_some(text) +} + +/// Draws an equation's layout into the box `(x, y, w, h)` (page points) of a PDF page, with +/// `pad` points of padding. Glyphs no standard font has are drawn as `[?]` boxes. +pub(crate) fn draw_pdf(pg: &mut gridcraft_pdf::Page, list: &MathList, rect: (f32, f32, f32, f32), pad: f32, ink: gridcraft_pdf::Rgb) { + use gridcraft_math::{Face, Item, PathCmd}; + let (x, y, w, h) = rect; + let (cw, ch) = (list.width.max(0.01), list.total_height().max(0.01)); + let s = ((w - 2.0 * pad) / cw).min((h - 2.0 * pad) / ch); + if !(s.is_finite() && s > 0.01) { + return; + } + let s = s.min(4000.0); + let (ox, oy) = (x + (w - cw * s) / 2.0, y + (h - ch * s) / 2.0); + let col = |c: Option<[u8; 3]>| c.unwrap_or(ink); + for item in list.items.iter().take(gridcraft_math::MAX_ITEMS) { + match item { + Item::Glyph(g) => { + let color = col(g.color); + let (gx, base) = (ox + g.x * s, oy + g.y * s); + // The radical sign: Symbol's glyph (AFM bounding box -147 to 917 high, right edge + // at 558, in 1/1000 em) fitted to the TeX box, so its top meets the vinculum. + if g.ch == '\u{221A}' { + let size = (g.height + g.depth) * s / 1.064; + if size.is_finite() && size >= 0.5 { + let top = base - g.height * s; + pg.text(gx + g.width * s - 0.558 * size, top + 0.917 * size, size, gridcraft_pdf::Font::Symbol, color, "\u{221A}"); + } + continue; + } + let large = matches!(g.face, Face::Large(_)); + let size = if large { (g.height + g.depth) * s } else { g.size * s }; + if !(size.is_finite() && size >= 0.5) { + continue; + } + let pick = pdf_font(g.ch, g.face).map(|f| (g.ch, f)).or_else(|| g.alt.and_then(|a| pdf_font(a, g.face).map(|f| (a, f)))); + match pick { + Some((c, font)) => { + let text = c.to_string(); + if large { + // Centred on the glyph's box, like the screen. + let mid = oy + (g.y - (g.height - g.depth) / 2.0) * s; + let tw = gridcraft_pdf::text_width(font, size, &text); + pg.text(gx + (g.width * s - tw) / 2.0, mid + 0.25 * size, size, font, color, &text); + } else { + pg.text(gx, base, size, font, color, &text); + } + } + None => { + // [?]: the glyph's box with a question mark. + let (bw, bh) = ((g.width.max(0.3)) * s, (g.height + g.depth) * s); + pg.rect_stroke(gx, base - g.height * s, bw, bh, (0.05 * s).clamp(0.2, 1.0), color); + let qs = (bh * 0.6).min(bw * 1.2).max(1.0); + let qw = gridcraft_pdf::text_width(gridcraft_pdf::Font::Helvetica, qs, "?"); + pg.text(gx + (bw - qw) / 2.0, base - g.height * s + (bh + qs * 0.7) / 2.0, qs, gridcraft_pdf::Font::Helvetica, color, "?"); + } + } + } + Item::Rule(r) => { + if r.width > 0.0 && r.height > 0.0 { + pg.fill_rect(ox + r.x * s, oy + r.y * s, r.width * s, (r.height * s).max(0.25), col(r.color)); + } + } + Item::Path(p) => { + let (px, py) = (ox + p.x * s, oy + p.y * s); + let mut subs: Vec> = vec![]; + let mut cur: Vec<[f32; 2]> = vec![]; + let at = |a: f32, b: f32| [px + a * s, py + b * s]; + for c in &p.commands { + match *c { + PathCmd::MoveTo(a, b) => { + if cur.len() > 1 { + subs.push(std::mem::take(&mut cur)); + } + cur = vec![at(a, b)]; + } + PathCmd::LineTo(a, b) => cur.push(at(a, b)), + PathCmd::QuadTo(x1, y1, a, b) => { + let p0 = cur.last().copied().unwrap_or(at(x1, y1)); + let (c1, p1) = (at(x1, y1), at(a, b)); + for k in 1..=8 { + let t = k as f32 / 8.0; + let u = 1.0 - t; + cur.push([u * u * p0[0] + 2.0 * u * t * c1[0] + t * t * p1[0], u * u * p0[1] + 2.0 * u * t * c1[1] + t * t * p1[1]]); + } + } + PathCmd::CubicTo(x1, y1, x2, y2, a, b) => { + let p0 = cur.last().copied().unwrap_or(at(x1, y1)); + let (c1, c2, p1) = (at(x1, y1), at(x2, y2), at(a, b)); + for k in 1..=10 { + let t = k as f32 / 10.0; + let u = 1.0 - t; + let (a3, b3, c3, d3) = (u * u * u, 3.0 * u * u * t, 3.0 * u * t * t, t * t * t); + cur.push([a3 * p0[0] + b3 * c1[0] + c3 * c2[0] + d3 * p1[0], a3 * p0[1] + b3 * c1[1] + c3 * c2[1] + d3 * p1[1]]); + } + } + PathCmd::Close => { + if cur.len() > 1 { + subs.push(std::mem::take(&mut cur)); + } + } + } + } + if cur.len() > 1 { + subs.push(cur); + } + let color = col(p.color); + for sub in subs.iter().filter(|v| v.len() <= 4000) { + if p.fill && sub.len() >= 3 { + // The PDF fill rule handles concave outlines itself. + pg.polygon(sub, Some(color), None); + } else { + pg.polyline(sub, (0.05 * s).clamp(0.2, 1.0), color, &[]); + } + } + } + } + } +} diff --git a/crates/engine/src/cmd/extra.rs b/crates/engine/src/cmd/extra.rs index bf21433ab..a6f66b41b 100644 --- a/crates/engine/src/cmd/extra.rs +++ b/crates/engine/src/cmd/extra.rs @@ -2,10 +2,16 @@ //! tools, Get Data, scripts (Automate), cell checkboxes, theme colours and fonts. use gridcraft_core::{CellRef, RangeRef, Value}; -use gridcraft_model::*; +use gridcraft_model::{ + Anchor, Chart, Color, DefinedName, HAlign, LegendPos, NumFmt, Shape, ShapeKind, Sheet, SheetViewMode, TableColumn, TotalsFn, Trendline, + TrendlineKind, Workbook, +}; use serde_json::{Value as Json, json}; -use super::*; +use super::{ + CommandSpec, EngineError, Result, Session, always, anchor_mode_name, bad, bool_param, cell_param, cell_param_on, cmd, edit, f64_param, has_doc, + ok, split_sheet, str_param, target_range, target_ranges, target_sheet, u32_param, +}; mod cells; mod chart; @@ -22,7 +28,7 @@ use objects::{align, arrange, rotate, selection_pane, shape_fill, shape_outline} pub(crate) use scripts::has_scripts; use scripts::{all_scripts, delete_script, new_script, record, run_script, stop_action, trust_scripts}; use window::{arrange_all, custom_views, hide_window, new_window, ruler, set_view, split, switch_windows, unhide_window, watch_window}; -pub(crate) use window::{keep_watch_list, window_param}; +pub(crate) use window::{keep_view_modes, keep_watch_list, window_param}; pub fn specs() -> Vec { vec![ @@ -32,7 +38,7 @@ pub fn specs() -> Vec { "Add Chart Element", ["Chart Design", "Chart Layouts"], None, - "{chart?, element: title|legend|dataLabels|gridlines|axisTitles|smooth|trendline, on?: bool, position?: bottom|top|left|right; trendline: series?: index (default all), type?: linear|exponential|linearForecast|movingAverage|logarithmic|power|polynomial, order?: 2..6, period?: 2..255, forward?, backward?, displayEquation?, displayRSquared?}", + "{chart?, element: title|legend|dataLabels|dataTable|gridlines|axisTitles|smooth|trendline, on?: bool, position?: bottom|top|left|right; dataTable: dataTableKeys?: bool, dataTableHorizontalBorder?: bool, dataTableVerticalBorder?: bool, dataTableOutline?: bool; trendline: series?: index (default all), type?: linear|exponential|linearForecast|movingAverage|logarithmic|power|polynomial, order?: 2..6, period?: 2..255, forward?, backward?, displayEquation?, displayRSquared?}", has_doc, add_element ), @@ -91,7 +97,7 @@ pub fn specs() -> Vec { "Bring Forward", ["Page Layout", "Arrange"], None, - "{kind: chart|image|shape, id, toFront?: bool}", + "{kind: chart|image|shape|slicer|equation, id, toFront?: bool}", has_doc, |s, p| arrange(s, p, true) ), @@ -100,7 +106,7 @@ pub fn specs() -> Vec { "Send Backward", ["Page Layout", "Arrange"], None, - "{kind: chart|image|shape, id, toBack?: bool}", + "{kind: chart|image|shape|slicer|equation, id, toBack?: bool}", has_doc, |s, p| arrange(s, p, false) ), @@ -110,7 +116,7 @@ pub fn specs() -> Vec { "Align", ["Page Layout", "Arrange"], None, - "{kind: chart|image|shape, ids: [..], align: left|center|right|top|middle|bottom}", + "{kind: chart|image|shape|slicer|equation, ids: [..], align: left|center|right|top|middle|bottom}", has_doc, align ), @@ -304,7 +310,7 @@ pub fn specs() -> Vec { new_cell_style ), cmd!(query "home.addIns", "Add-ins", ["Home", "Add-ins"], None, "{} → GridCraft extends through MCP and scripts instead of add-ins", always, |_, _| Ok(json!({"addIns": [], "hint": "Use gridcraft-cli mcp or Automate › scripts"}))), - cmd!(query "formulas.recentlyUsed", "Recently Used", ["Formulas", "Function Library"], None, "{} → recently used functions in this workbook", has_doc, recently_used), + cmd!(query "formulas.recentlyUsed", "Recently Used", ["Formulas", "Function Library"], None, "{} → the functions most recently entered in formulas, newest first", has_doc, recently_used), cmd!(query "formulas.financial", "Financial", ["Formulas", "Function Library"], None, "{}", always, |s, _| s.execute("formulas.functions", json!({"category": "Financial"}))), cmd!(query "formulas.logical", "Logical", ["Formulas", "Function Library"], None, "{}", always, |s, _| s.execute("formulas.functions", json!({"category": "Logical"}))), cmd!(query "formulas.text", "Text", ["Formulas", "Function Library"], None, "{}", always, |s, _| s.execute("formulas.functions", json!({"category": "Text"}))), diff --git a/crates/engine/src/cmd/extra/cells.rs b/crates/engine/src/cmd/extra/cells.rs index 5cddf0982..838ce8342 100644 --- a/crates/engine/src/cmd/extra/cells.rs +++ b/crates/engine/src/cmd/extra/cells.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + CellRef, EngineError, HAlign, Json, NumFmt, Result, Session, Value, bad, bool_param, cell_param, cell_param_on, edit, json, object_params, ok, + str_param, target_ranges, target_sheet, +}; /// Whether `c` has checkbox formatting. pub(crate) fn is_checkbox_cell(wb: &gridcraft_model::Workbook, sh: &gridcraft_model::Sheet, c: CellRef) -> bool { @@ -48,6 +51,7 @@ pub(super) fn toggle_checkbox(s: &mut Session, p: &Json) -> Result { let sh = cx.sheet_mut(si)?; if let Some(cell) = sh.cells.get_mut(c) && cell.formula.is_none() + && cell.py.is_none() { // An empty checkbox cell counts as FALSE. let next = match cell.value { @@ -119,25 +123,10 @@ pub(super) fn new_cell_style(s: &mut Session, p: &Json) -> Result { }) } +/// Formulas › Recently Used: the functions most recently entered in a formula, newest first +/// ([`crate::Prefs::recent_functions`]). pub(super) fn recently_used(s: &mut Session, _: &Json) -> Result { - let mut seen: Vec = Vec::new(); - for (id, p) in s.journal.iter().rev() { - if id != "cell.set" { - continue; - } - let input = p.get("input").and_then(Json::as_str).unwrap_or(""); - if let Ok(e) = gridcraft_formula::parse(input.trim_start_matches('=')) { - for f in e.functions() { - if !seen.iter().any(|x| x == f) { - seen.push(f.to_string()); - } - } - } - if seen.len() >= 10 { - break; - } - } - Ok(json!(seen)) + Ok(json!(s.prefs.recent_functions)) } pub(super) fn use_in_formula(s: &mut Session, p: &Json) -> Result { diff --git a/crates/engine/src/cmd/extra/chart.rs b/crates/engine/src/cmd/extra/chart.rs index 2e16befa6..8d9af2064 100644 --- a/crates/engine/src/cmd/extra/chart.rs +++ b/crates/engine/src/cmd/extra/chart.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + CellRef, Chart, Color, EngineError, Json, LegendPos, RangeRef, Result, Session, Trendline, TrendlineKind, bad, bool_param, cell_param_on, edit, + f64_param, json, split_sheet, str_param, u32_param, +}; fn find_chart(s: &Session, p: &Json) -> Result<(usize, usize)> { let d = s.doc()?; @@ -40,6 +43,7 @@ pub(super) fn add_element(s: &mut Session, p: &Json) -> Result { } } "dataLabels" => c.data_labels = on, + "dataTable" => crate::cmd::insert::update_data_table(c, p, Some(on))?, "gridlines" => c.gridlines = on, "axisTitles" => { c.x_title = on.then(|| c.x_title.clone().unwrap_or_else(|| "Axis Title".into())); diff --git a/crates/engine/src/cmd/extra/data.rs b/crates/engine/src/cmd/extra/data.rs index cef840f21..9a1ae7417 100644 --- a/crates/engine/src/cmd/extra/data.rs +++ b/crates/engine/src/cmd/extra/data.rs @@ -1,4 +1,4 @@ -use super::*; +use super::{CellRef, EngineError, Json, RangeRef, Result, Session, TableColumn, TotalsFn, Value, bad, edit, json, ok, str_param, target_range}; pub(super) fn table_resize(s: &mut Session, p: &Json) -> Result { let r = str_param(p, "range").and_then(RangeRef::parse).ok_or_else(|| bad("table.resize", "missing `range`"))?; diff --git a/crates/engine/src/cmd/extra/objects.rs b/crates/engine/src/cmd/extra/objects.rs index 59ab8447a..fa5a5f438 100644 --- a/crates/engine/src/cmd/extra/objects.rs +++ b/crates/engine/src/cmd/extra/objects.rs @@ -1,5 +1,28 @@ +use gridcraft_model::ObjKind; + use super::chart::format_series; -use super::*; +use super::{ + Anchor, CellRef, Color, EngineError, Json, Result, Session, Shape, ShapeKind, Sheet, anchor_mode_name, bad, bool_param, edit, json, str_param, + u32_param, +}; + +/// The kind of object a `kind` parameter names: `shape` when the parameter is absent; one that +/// is present but not one of the names (`null`, a number, an unknown word) is an error, as for +/// the `object.*` commands, where the kind is required. +fn kind_param(p: &Json, cmd: &str) -> Result { + let err = || bad(cmd, "kind must be chart, image, shape, slicer or equation"); + match p.get("kind") { + None => Ok(ObjKind::Shape), + Some(Json::String(k)) => ObjKind::parse(k).ok_or_else(err), + Some(_) => Err(err()), + } +} + +/// Whether `kind` names the shape `sh`: an equation is a shape too, and a plain shape is not an +/// equation, as `Sheet::object_anchor` has it for the `object.*` commands. +fn shape_is(kind: ObjKind, sh: &Shape) -> bool { + kind != ObjKind::Equation || sh.kind == ShapeKind::Equation +} pub(super) fn shape_fill(s: &mut Session, p: &Json) -> Result { if str_param(p, "kind") == Some("shape") { @@ -31,7 +54,7 @@ fn with_shape(s: &mut Session, p: &Json, f: impl FnOnce(&mut Shape)) -> Result Result { let id = u32_param(p, "id").ok_or_else(|| bad("arrange", "missing `id`"))?; - let kind = str_param(p, "kind").unwrap_or("shape").to_string(); + let kind = kind_param(p, "arrange")?; let extreme = bool_param(p, "toFront").or(bool_param(p, "toBack")).unwrap_or(false); let si = s.doc()?.wb.active_sheet(); fn reorder(v: &mut Vec, i: usize, forward: bool, extreme: bool) { @@ -49,23 +72,17 @@ pub(super) fn arrange(s: &mut Session, p: &Json, forward: bool) -> Result } edit(s, |cx| { let sh = cx.sheet_mut(si)?; - match kind.as_str() { - "chart" => { - if let Some(i) = sh.charts.iter().position(|c| c.id == id) { - reorder(&mut sh.charts, i, forward, extreme); - } + // An id that is not an object of this kind is an error, so a stale id, or a shape's id + // given as an equation, never reorders something else or succeeds silently. + let found = match kind { + ObjKind::Chart => sh.charts.iter().position(|c| c.id == id).map(|i| reorder(&mut sh.charts, i, forward, extreme)), + ObjKind::Image => sh.images.iter().position(|c| c.id == id).map(|i| reorder(&mut sh.images, i, forward, extreme)), + ObjKind::Slicer => sh.slicers.iter().position(|c| c.id == id).map(|i| reorder(&mut sh.slicers, i, forward, extreme)), + ObjKind::Shape | ObjKind::Equation => { + sh.shapes.iter().position(|c| c.id == id && shape_is(kind, c)).map(|i| reorder(&mut sh.shapes, i, forward, extreme)) } - "image" => { - if let Some(i) = sh.images.iter().position(|c| c.id == id) { - reorder(&mut sh.images, i, forward, extreme); - } - } - _ => { - if let Some(i) = sh.shapes.iter().position(|c| c.id == id) { - reorder(&mut sh.shapes, i, forward, extreme); - } - } - } + }; + found.ok_or_else(|| bad("arrange", "no such object"))?; Ok(Json::Null) }) } @@ -75,7 +92,8 @@ pub(super) fn selection_pane(s: &mut Session, _: &Json) -> Result { let sh = d.wb.active().ok_or(EngineError::NoDocument)?; let mut v = Vec::new(); for c in &sh.shapes { - v.push(json!({"kind": "shape", "id": c.id, "name": format!("{:?} {}", c.kind, c.id), "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)})); + let kind = if c.kind == ShapeKind::Equation { "equation" } else { "shape" }; + v.push(json!({"kind": kind, "id": c.id, "name": format!("{:?} {}", c.kind, c.id), "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)})); } for c in &sh.images { v.push(json!({"kind": "image", "id": c.id, "name": format!("Picture {}", c.id), "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)})); @@ -83,33 +101,51 @@ pub(super) fn selection_pane(s: &mut Session, _: &Json) -> Result { for c in &sh.charts { v.push(json!({"kind": "chart", "id": c.id, "name": format!("Chart {}", c.id), "title": c.title, "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)})); } + // Slicers paint above the other objects. + for c in &sh.slicers { + let name = if c.caption.is_empty() { &c.column } else { &c.caption }; + v.push(json!({"kind": "slicer", "id": c.id, "name": name, "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)})); + } Ok(Json::Array(v)) } -fn anchors_of<'a>(sh: &'a mut Sheet, kind: &str) -> Vec<(u32, &'a mut Anchor)> { +/// The anchors of every object of `kind` on the sheet; under `equation`, only the equations. +fn anchors_of(sh: &mut Sheet, kind: ObjKind) -> Vec<(u32, &mut Anchor)> { match kind { - "chart" => sh.charts.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), - "image" => sh.images.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), - _ => sh.shapes.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), + ObjKind::Chart => sh.charts.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), + ObjKind::Image => sh.images.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), + ObjKind::Slicer => sh.slicers.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), + ObjKind::Shape | ObjKind::Equation => sh.shapes.iter_mut().filter(|c| shape_is(kind, c)).map(|c| (c.id, &mut c.anchor)).collect(), } } pub(super) fn align(s: &mut Session, p: &Json) -> Result { - let kind = str_param(p, "kind").unwrap_or("shape").to_string(); - let ids: Vec = - p.get("ids").and_then(Json::as_array).map(|a| a.iter().filter_map(|v| v.as_u64().map(|x| x as u32)).collect()).unwrap_or_default(); + let kind = kind_param(p, "arrange.align")?; + // `ids` may be absent (nothing to align) but, present, must be an array whose every entry is + // an object id naming an object of the kind: a stale or mismatched id, or a value of another + // shape, is an error, as for `arrange` and the `object.*` commands, not something silently + // left out. + let bad_ids = || bad("arrange.align", "ids must be an array of object ids"); + let ids: Vec = match p.get("ids") { + None => Vec::new(), + Some(Json::Array(a)) => a.iter().map(|v| v.as_u64().and_then(|x| u32::try_from(x).ok()).ok_or_else(bad_ids)).collect::>()?, + Some(_) => return Err(bad_ids()), + }; let how = str_param(p, "align").unwrap_or("left").to_string(); let si = s.doc()?.wb.active_sheet(); edit(s, |cx| { let sheet = cx.sheet_mut(si)?; let geo: Vec<(u32, f64, f64, f32, f32)> = { let shc = sheet.clone(); - anchors_of(sheet, &kind) + anchors_of(sheet, kind) .into_iter() .filter(|(id, _)| ids.contains(id)) .map(|(id, a)| (id, shc.col_left(a.cell.col) + a.dx as f64, shc.row_top(a.cell.row) + a.dy as f64, a.width, a.height)) .collect() }; + if let Some(missing) = ids.iter().find(|id| !geo.iter().any(|g| g.0 == **id)) { + return Err(bad("arrange.align", format!("no such object: {missing}"))); + } if geo.len() < 2 { return Ok(Json::Null); } @@ -118,7 +154,7 @@ pub(super) fn align(s: &mut Session, p: &Json) -> Result { let top = geo.iter().map(|g| g.2).fold(f64::INFINITY, f64::min); let bottom = geo.iter().map(|g| g.2 + g.4 as f64).fold(f64::NEG_INFINITY, f64::max); let shc = sheet.clone(); - for (id, a) in anchors_of(sheet, &kind) { + for (id, a) in anchors_of(sheet, kind) { let Some(g) = geo.iter().find(|g| g.0 == id) else { continue }; let (mut x, mut y) = (g.1, g.2); match how.as_str() { diff --git a/crates/engine/src/cmd/extra/scripts.rs b/crates/engine/src/cmd/extra/scripts.rs index 9f86837ab..e9b877672 100644 --- a/crates/engine/src/cmd/extra/scripts.rs +++ b/crates/engine/src/cmd/extra/scripts.rs @@ -1,10 +1,15 @@ -use super::*; +use super::{EngineError, Json, Result, SCRIPTS_NAME, Session, Workbook, bad, bool_param, edit, json, stash_get, stash_map, stash_set, str_param}; pub(super) fn record(s: &mut Session, p: &Json) -> Result { if bool_param(p, "stop").unwrap_or(false) || s.recording.is_some() { let Some(start) = s.recording.take() else { return Ok(json!({"recording": false})) }; - let cmds: Vec = - s.journal.iter().skip(start).filter(|(id, _)| !id.starts_with("automate.")).map(|(id, p)| json!({"command": id, "params": p})).collect(); + let cmds: Vec = s + .journal + .iter() + .skip(start) + .filter(|(id, _)| !id.starts_with("automate.") && !crate::record::NOT_RECORDED.contains(&id.as_str())) + .map(|(id, p)| json!({"command": id, "params": p})) + .collect(); let name = str_param(p, "name").map(str::to_string).unwrap_or_else(|| format!("Script {}", start)); s.execute("automate.newScript", json!({"name": name, "commands": cmds}))?; return Ok(json!({"recording": false, "saved": name, "steps": cmds.len()})); @@ -104,12 +109,16 @@ pub(super) fn run_script(s: &mut Session, p: &Json) -> Result { } let _sandbox = crate::io::ScriptSandbox::enter(); let mut n = 0; + // What ran, for a recorder in progress: the recording keeps these steps with the run, so + // editing or deleting the script before the recording stops does not change what it saved. + let mut ran_steps: Vec<(String, Json)> = Vec::new(); for st in &steps { let id = st["command"].as_str().unwrap_or(""); if id.starts_with("automate.") { continue; } let before = s.ui_requests.len(); + let wb_before = s.doc().ok().map(|d| (d.uid, d.wb.clone())); let ran = s.execute(id, st["params"].clone()); // A step that asks the host to open a URL leaves the app: drop the request and stop. if s.ui_requests.get(before..).is_some_and(|new| new.iter().any(|r| matches!(r, crate::UiRequest::OpenUrl(_)))) { @@ -120,17 +129,43 @@ pub(super) fn run_script(s: &mut Session, p: &Json) -> Result { }); return Err(bad("automate.runScript", format!("scripts can't open links (`{id}`)"))); } - ran?; + let result = ran?; n += 1; + if s.recorder.is_some() { + // The recorder's rule for each step, as for a command run on its own: a menu-style + // step that only opened a dialog is left out (playing it back would stop at the + // dialog), and a toggle keeps the state it set. + let opened_dialog = s.ui_requests.get(before..).is_some_and(|new| new.iter().any(|r| matches!(r, crate::UiRequest::Dialog(..)))); + let changed = wb_before.as_ref().is_some_and(|(uid, wb)| s.docs.iter().any(|d| d.uid == *uid && !std::sync::Arc::ptr_eq(&d.wb, wb))); + if !opened_dialog || changed { + ran_steps.push((id.to_string(), crate::record::with_toggle_state(id, st["params"].clone(), &result))); + } + } + } + if s.recorder.is_some() { + s.script_ran = Some(ran_steps); } Ok(json!({"ran": n})) } -/// Automate › Stop Action: rolls back the undo groups open on the active workbook, so Undo and -/// Redo work there again. +/// Redo work there again. A macro being debugged on that workbook is stopped with them. pub(super) fn stop_action(s: &mut Session, _: &Json) -> Result { + // Inside another command the open groups are that command's own (an action in progress): + // a step of it that stopped them would pull the workbook from under the rest of its run. + if s.depth() > 1 { + return Err(EngineError::Disabled("automate.stopAction".into(), "Stop Action runs on its own, not as a step of a script or macro.".into())); + } let uid = s.doc()?.uid; - Ok(json!({"stopped": s.abort_groups(uid)})) + let mut out = json!({"stopped": s.abort_groups(uid)}); + // A macro being debugged here had its group among them: its run is over. + if let Some((name, wrote_files)) = crate::macros::debug::end_stopped(s, uid) { + out["macro"] = json!(name); + out["message"] = json!(format!( + "Macro '{name}', which was being debugged, was stopped and rolled back. {}", + crate::macros::run::rolled_back(&wrote_files) + )); + } + Ok(out) } pub(super) fn delete_script(s: &mut Session, p: &Json) -> Result { diff --git a/crates/engine/src/cmd/extra/window.rs b/crates/engine/src/cmd/extra/window.rs index 19e92743f..b8129805f 100644 --- a/crates/engine/src/cmd/extra/window.rs +++ b/crates/engine/src/cmd/extra/window.rs @@ -1,4 +1,7 @@ -use super::*; +use super::{ + CellRef, EngineError, Json, Result, Session, SheetViewMode, VIEWS_NAME, WATCH_NAME, Workbook, bad, bool_param, edit, json, ok, split_sheet, + stash_get, stash_map, stash_set, str_param, target_sheet, u32_param, +}; /// Switches a sheet's view. Like zoom, the view is saved with the file but is not an edit: it /// neither records an undo step nor marks a clean workbook as changed. @@ -35,6 +38,35 @@ pub(crate) fn keep_watch_list(from: &Workbook, to: &mut std::sync::Arc wb.names.extend(keep); } +/// Normal, Page Layout and Page Break Preview belong to the window, not to the edits: Excel doesn't +/// undo a switch between them, and `set_view` is no undo step. So undo and redo keep each sheet's +/// current view when they swap in another snapshot of the workbook, and a view kept this way +/// doesn't make a saved workbook dirty. +pub(crate) fn keep_view_modes(d: &mut crate::DocState, now: &Workbook) { + // Sheets have no stable id. A name missing from `now` is a rename being undone or redone, which + // keeps the sheet count and position: the sheet is the one at the same index, under a name the + // restored snapshot doesn't have. + let renamed = |i: usize| { + (now.sheets.len() == d.wb.sheets.len()).then(|| now.sheets.get(i)).flatten().filter(|sh| !d.wb.sheets.iter().any(|s| s.name == sh.name)) + }; + let current = |i: usize, name: &str| now.sheets.iter().find(|sh| sh.name == name).or_else(|| renamed(i)).map(|sh| sh.view); + let changed: Vec<(usize, SheetViewMode)> = + d.wb.sheets.iter().enumerate().filter_map(|(i, sh)| current(i, &sh.name).filter(|v| *v != sh.view).map(|v| (i, v))).collect(); + if changed.is_empty() { + return; + } + let dirty = d.is_dirty(); + let wb = std::sync::Arc::make_mut(&mut d.wb); + for (i, v) in changed { + if let Some(sh) = wb.sheet_mut(i) { + sh.view = v; + } + } + if !dirty { + d.saved = d.wb.clone(); + } +} + pub(super) fn custom_views(s: &mut Session, p: &Json) -> Result { let d = s.doc()?; let mut views = stash_map(&d.wb, VIEWS_NAME); diff --git a/crates/engine/src/cmd/file.rs b/crates/engine/src/cmd/file.rs index 9af85b9dd..b3b515c2e 100644 --- a/crates/engine/src/cmd/file.rs +++ b/crates/engine/src/cmd/file.rs @@ -2,7 +2,7 @@ use serde_json::{Value as Json, json}; -use super::*; +use super::{CommandSpec, Result, Session, always, bad, bool_param, cmd, edit, has_doc, ok, str_param, target_sheet, u32_param}; use crate::DocState; pub fn specs() -> Vec { @@ -13,7 +13,7 @@ pub fn specs() -> Vec { cmd!(noundo "file.saveAs", "Save As…", ["File"], Some("Cmd+Shift+S"), "{path} (.xlsx, .csv, .tsv, .json, .html, or .pdf: a PDF of the active sheet)", has_doc, save_as), cmd!(query "file.saveBytes", "Encode Workbook", [], None, "{format?: xlsx|csv|tsv|json|html} → {base64}", has_doc, save_bytes), cmd!(noundo "file.close", "Close", ["File"], Some("Cmd+W"), "{force?: bool, document?: bool} (closes the active window, or with `document` the workbook and all its windows; the workbook closes with its last window)", has_doc, close), - cmd!(noundo "file.exportCsv", "Export as CSV", ["File", "Export"], None, "{path, sheet?}", has_doc, |s, p| export(s, p, "csv")), + cmd!(noundo "file.exportCsv", "Export as CSV", ["File", "Export"], None, "{path, sheet?, values?: bool} → {path, bytes}: writes the cells as displayed, like Excel's Save As CSV; `values: true` writes raw values instead (full-precision numbers, ISO 8601 dates and times, TRUE/FALSE)", has_doc, |s, p| export(s, p, "csv")), cmd!(noundo "file.exportHtml", "Save as Web Page", ["File", "Export"], None, "{path}", has_doc, |s, p| export(s, p, "html")), cmd!("file.properties", "Properties", ["File"], None, "{title?, subject?, author?, company?, keywords?, description?}", has_doc, properties), cmd!( @@ -191,7 +191,11 @@ fn export(s: &mut Session, p: &Json, ext: &str) -> Result { wb.set_active_sheet(target_sheet(s, p)?); } let target = if path.ends_with(&format!(".{ext}")) { path.clone() } else { format!("{path}.{ext}") }; - let bytes = crate::io::save_bytes(&wb, &target)?; + let bytes = if ext == "csv" && bool_param(p, "values").unwrap_or(false) { + crate::io::csv_values_bytes(&wb)? + } else { + crate::io::save_bytes(&wb, &target)? + }; crate::io::write_file(&target, &bytes)?; Ok(json!({"path": target, "bytes": bytes.len()})) } diff --git a/crates/engine/src/cmd/forecast.rs b/crates/engine/src/cmd/forecast.rs index b7472fd22..3c62e469f 100644 --- a/crates/engine/src/cmd/forecast.rs +++ b/crates/engine/src/cmd/forecast.rs @@ -11,7 +11,7 @@ use gridcraft_functions::Timeline; use gridcraft_model::{Anchor, Cell, Chart, ChartKind, Formula, LegendPos, Series, Sheet, StyleId, Table, TableColumn, TotalsFn}; use serde_json::{Value as Json, json}; -use super::*; +use super::{CommandSpec, EngineError, Result, Session, bad, bool_param, cmd, edit, has_doc, split_sheet, str_param}; use crate::selection::current_region; const CMD: &str = "data.forecastSheet"; @@ -315,7 +315,7 @@ fn forecast_sheet(s: &mut Session, p: &Json) -> Result { let vstyle = cx.wb.styles.derive(StyleId::DEFAULT, |st| st.num_fmt = vfmt.clone()); let mut sheet = Sheet::new(sheet_name.clone()); let put = |sheet: &mut Sheet, row: u32, col: u32, v: Value, style: StyleId| { - sheet.set_cell(CellRef::new(row, col), Cell { value: v, formula: None, style }); + sheet.set_cell(CellRef::new(row, col), Cell { value: v, formula: None, style, py: None }); }; let formula = |sheet: &mut Sheet, row: u32, col: u32, text: String, style: StyleId| { let mut cell = Cell::formula(Formula::new(&text)); @@ -424,6 +424,7 @@ fn forecast_sheet(s: &mut Session, p: &Json) -> Result { series, legend: LegendPos::Bottom, data_labels: false, + data_table: None, gridlines: true, style: 1, x_title: None, diff --git a/crates/engine/src/cmd/format.rs b/crates/engine/src/cmd/format.rs index 744b6d136..1066f48e2 100644 --- a/crates/engine/src/cmd/format.rs +++ b/crates/engine/src/cmd/format.rs @@ -1,10 +1,13 @@ //! Formatting: Home › Font, Alignment, Number, Styles, Cells › Format. use gridcraft_core::{CellRef, RangeRef}; -use gridcraft_model::*; +use gridcraft_model::{BorderLine, BorderStyle, Borders, Color, Fill, HAlign, NumFmt, Style, StyleId, Theme, Underline, VAlign, VertAlign}; use serde_json::{Value as Json, json}; -use super::*; +use super::{ + CommandSpec, EngineError, Result, Session, bad, bool_param, cmd, edit, f64_param, has_doc, ok, parse_lines, split_sheet, str_param, + target_ranges, target_sheet, +}; pub fn specs() -> Vec { vec![ @@ -87,8 +90,14 @@ pub fn specs() -> Vec { format_cells ), // Alignment - cmd!("home.alignLeft", "Align Left", ["Home", "Alignment"], Some("Cmd+L"), "{range?}", has_doc, |s, p| halign(s, p, HAlign::Left)), - cmd!("home.alignCenter", "Center", ["Home", "Alignment"], Some("Cmd+E"), "{range?}", has_doc, |s, p| halign(s, p, HAlign::Center)), + CommandSpec { + shortcut_mac: Some("Cmd+L"), + ..cmd!("home.alignLeft", "Align Left", ["Home", "Alignment"], Some("Cmd+L"), "{range?}", has_doc, |s, p| halign(s, p, HAlign::Left)) + }, + CommandSpec { + shortcut_mac: Some("Cmd+E"), + ..cmd!("home.alignCenter", "Center", ["Home", "Alignment"], Some("Cmd+E"), "{range?}", has_doc, |s, p| halign(s, p, HAlign::Center)) + }, cmd!("home.alignRight", "Align Right", ["Home", "Alignment"], None, "{range?}", has_doc, |s, p| halign(s, p, HAlign::Right)), cmd!("home.justify", "Justify", ["Home", "Alignment"], None, "{range?}", has_doc, |s, p| halign(s, p, HAlign::Justify)), cmd!("home.alignTop", "Top Align", ["Home", "Alignment"], None, "{range?}", has_doc, |s, p| valign(s, p, VAlign::Top)), @@ -647,8 +656,7 @@ fn merge(s: &mut Session, p: &Json, how: &str) -> Result { let others: Vec = sh.cells.iter_range(m).map(|(c, _)| c).filter(|c| *c != m.start).collect(); for c in others { if let Some(mut cell) = sh.cells.get(c).cloned() { - cell.value = gridcraft_core::Value::Empty; - cell.formula = None; + cell.clear_content(); sh.set_cell(c, cell); } touched.push((sheet, c)); @@ -665,8 +673,7 @@ fn merge(s: &mut Session, p: &Json, how: &str) -> Result { let others: Vec = sh.cells.iter_range(*r).map(|(c, _)| c).filter(|c| *c != r.start).collect(); for c in others { if let Some(mut cell) = sh.cells.get(c).cloned() { - cell.value = gridcraft_core::Value::Empty; - cell.formula = None; + cell.clear_content(); sh.set_cell(c, cell); } touched.push((sheet, c)); @@ -683,7 +690,10 @@ fn merge(s: &mut Session, p: &Json, how: &str) -> Result { let blocked = sh .cells .iter_range(reach) - .filter(|(_, x)| x.formula.is_some() && x.value == gridcraft_core::Value::Error(gridcraft_core::CellError::Spill)); + // A Python cell's result is blocked and unblocked like a formula's. + .filter(|(_, x)| { + (x.formula.is_some() || x.py.is_some()) && x.value == gridcraft_core::Value::Error(gridcraft_core::CellError::Spill) + }); touched.extend(blocked.map(|(c, _)| (sheet, c))); } cx.changed.extend(touched); diff --git a/crates/engine/src/cmd/format/cell_styles.rs b/crates/engine/src/cmd/format/cell_styles.rs index 53d378202..5ad0ccc15 100644 --- a/crates/engine/src/cmd/format/cell_styles.rs +++ b/crates/engine/src/cmd/format/cell_styles.rs @@ -1,4 +1,6 @@ -use super::*; +use super::{ + ACCOUNTING, BorderLine, BorderStyle, Borders, COMMA, Color, Fill, Json, NumFmt, Result, Session, Style, Theme, apply_style, bad, str_param, +}; /// Built-in cell styles (our own palette). pub fn builtin_cell_style(name: &str, theme: &Theme) -> Option