Files
gridcraft/docs/decisions.md
T
ca39b5ab9f XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel (#417)
* XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel; implied @ saved back without cm

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* XLSX: long operator chains analysed in full and unfinished legacy analysis leaves the formula unchanged (never saved as SINGLE); @ of a union keeps its parentheses; IF's condition intersected inside SUMPRODUCT

* XLSX: a legacy union already in parentheses saves back without a second pair (never as SINGLE); a name's definition analysed as an array throughout, inserting no @

* XLSX: drop the unreachable keep-unions retry; an @ of a bare union saves as a dynamic formula

* XLSX: an @ keeps parentheses the reader would not put back (redundant ones, a union's)

* XLSX: a redundant @ keeps a legacy formula legacy; IFERROR and IFNA intersected inside SUMPRODUCT; file parentheses and unfinished analyses saved as read

An edit could leave an implied @ on a single cell: deleting rows 2:3 under the legacy SQRT(A1:A3), read as SQRT(@A1:A3), gives SQRT(@A1:A1), and a name redefined to one cell does the same to SQRT(@Rng). The writer kept an @ only when re-analysing the formula without it put the same @ back, which it no longer does for one cell, so the formula was saved as SQRT(_xlfn.SINGLE(A1:A1)) with cm, readable only by dynamic-array Excel. The writer now analyses the formula with its own @ in place: an @ is accepted when the analysis implies it at that position or when its operand is a single value there (such an @ changes nothing), and any @ inserted elsewhere or any other @ still makes the formula dynamic. SQRT(@A1:A1) saves as SQRT(A1:A1), SQRT(@A1) as SQRT(A1) and @SUM(A1:A3) as SUM(A1:A3), all without cm; SUMPRODUCT(@SQRT(A1:A3)) stays dynamic because SQRT(A1:A3) is an array there.

Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT unless the formula is entered with Ctrl+Shift+Enter, like IF's condition: =SUMPRODUCT(IFERROR(VALUE(C19:C21),0)) returns one row's value or #VALUE!, and the accepted fix is CSE or N(+range) (https://stackoverflow.com/questions/59228707). They now get the same single-value parameter class as IF's condition (Param::Single), so the legacy SUMPRODUCT(IFERROR(A1:A3/A1:A3,0)) reads as SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0)), and a typed SUMPRODUCT(IFERROR(VALUE(A1:A3),0)), which dynamic-array Excel evaluates as an array, is saved with cm. IF's result arguments are left as arrays: in https://stackoverflow.com/questions/29682328, a non-CSE =SUMPRODUCT(--(B4=IF(NOT($C$4:$C$11),$B$4:$B$11))) in row 4 returns 2 (both "apple" rows of B4:B11 counted, so the result branch was not intersected), 0 in a row where C is TRUE and #VALUE! outside rows 4:11 (the condition was intersected).

The reader's @ in front of a whole-formula array result took no extra parentheses when the file already had them, so ({1,2}+1) and {1,2}+1 both read as @({1,2}+1), and the writer dropped the file's parentheses. An operand already in parentheses now gets a second pair, so ({1,2}+1) saves as written.

A legacy formula the analysis could not finish (node budget, nesting) was read unchanged, evaluated as a dynamic-array formula and saved with cm, so Excel spilled it. Names were analysed again at every use, so a 399-term name used six times exhausted the 4096-node budget; their kinds are now remembered per formula, and the budget is 16384 nodes. The nesting limit is 128, so Excel's 64 levels of nested functions are analysed. A formula that still exceeds the limits is saved as it was read, without cm.

* XLSX: an implied @ is dropped and any other @ of a single value kept as _xlfn.SINGLE in a legacy formula (as Excel writes it); a legacy formula the analysis cannot finish or whose semantics it does not know opens with a whole-formula @ and never spills; analysis depth follows the parser's limit

* XLSX: a legacy union given to a function's single-value parameter keeps its implied @, so it calculates to #VALUE! instead of spilling and saves as read; checked on a calculated workbook

* XLSX: only a one-cell range an @ applies to counts as one an edit shrank; a reading past the parser's nesting limit leaves the formula as read, and the writer saves the text a formula it cannot analyse was read from

* XLSX: a formula nested past the parser's limit for its reading is left as read (documented: it may spill); test covers it

* XLSX: a name's definition counts its one-cell ranges as collapsed only when it is itself a reference, so CHOOSE's index or IF's condition in it keeps the name legacy

* XLSX: an @ on a name or one-cell range counts as implied when ranges an edit may have shrunk, anywhere the name returns them (CHOOSE's choices, parentheses), make it so; deleting rows keeps such formulas legacy

* XLSX: the collapsed-range retry of an @ runs only when the first pass reached a one-cell range written as a range (tracked through cached names)

* docs: describe unfinished formula export fallback

* docs: the unfinished-analysis fallback differs for formulas with an explicit @

* Preserve main formula analyzer fixes

* Order file-format revision history

---------

Co-authored-by: neilcawse <8085959+neilcawse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Local contributor <contributor@localhost>
2026-11-10 12:43:07 +01:00

12 KiB
Raw Blame History

Decisions

Bound repeated XLSX column expansion (#1022)

Limit each worksheet to 1,000,000 nondefault column assignments across all <cols> blocks. This allows more than 60 full-grid passes while bounding repeated work on overlapping spans; counting only the final map size would not limit that work. Charge each clipped span before inserting it, after the existing default-record skip. Exceeding the limit returns the existing IoError::TooLarge, which aborts the import instead of silently dropping a partially read sheet. Keep the current last-wins semantics rather than introduce interval storage or normalization.

Formula localization uses the existing interface preference

PR #98 builds on the persisted UiState.language introduced by #31 and extended by #104. Español selects the Spanish function table and list/decimal separators; there is no competing locale setting. Formula syntax without a supplied translation table falls back to English. The editor translates to canonical English before engine commands, and translates back only for display. API, clipboard and XLSX syntax remain stable. Existing names override aliases.

The revision is based directly on main after #31/#104 and Portuguese interface support (#82) merged. Those locales retain their existing formula fallback; this change adds only Spanish.

Preserve array evaluation when exporting formulas (#64)

GridCraft evaluates ordinary formulas with array semantics even when the final result occupies one cell. XLSX export marks actual spills and formulas whose results may differ under legacy implicit intersection as dynamic, using the spill range or a single-cell array reference. Legacy CSE array ranges keep their existing representation.

The exporter inspects the parsed formula without evaluating it. It distinguishes range-taking functions such as SUM from scalar arguments and operators: SUM(A1:A3) and A1*2 stay ordinary, while SUM(A1:A3*B1:B3) and SQRT(A1:A3) retain array evaluation. Legacy array-aware contexts such as SUMPRODUCT do not need an extra marker. Explicit @ scalarizes its result without hiding array-sensitive operations inside it.

Names are resolved using workbook/sheet scope, and structured references distinguish one table-row cell from a column. Literal INDIRECT and INDEX arguments permit tighter classification; unresolved reference dimensions are conservatively treated as potentially multiple cells. Name/AST traversal is bounded. A formula whose array behaviour the analysis doesn't know (a user-defined or unknown function, an uncertain parameter, a recursive name) is saved with cm, unless it has an explicit @ (below). One without an explicit @ that the analysis cannot finish, but whose parts are all known, is saved as read, without cm. One with an explicit @ that the analysis cannot finish or doesn't know is saved as the legacy text the reader turns back into it (the formula without its @, or without a whole-formula @) when there is one, and with cm otherwise. This is an export classifier, not a claim of exact Excel metadata parity for every function or user-defined function. It stays within the format crate's existing dependencies.

The reader applies the same classification in reverse: a formula without dynamic-array metadata (cm) and not entered as a CSE array is a legacy formula, and gets the @ Excel implies where a range meets a single-value parameter, an operator or the cell result. Inside SUMPRODUCT and other array-evaluating parameters, IF's condition and IFERROR's and IFNA's arguments are still intersected, as legacy Excel does without Ctrl+Shift+Enter; IF's results are not. The writer drops an @ that legacy Excel would imply (also when an edit has shrunk its range to one cell) and saves that formula without cm; an @ of a single value stays _xlfn.SINGLE in that legacy formula, as Excel writes it; any other @ makes the formula a dynamic-array one. A legacy formula too large to analyse, or using functions whose legacy argument semantics are unknown (the array arguments of dynamic-array functions, LAMBDA), is read as it is with only a whole-formula @ when its result may be several values, so it never spills, and is saved as it was read, without cm. Which names are built-in functions and which parameters take a single value comes from the function library, which the format crate may not depend on: the caller passes a gridcraft_xlsx::FunctionInfo (read_xlsx_with, write_xlsx_with; the engine supplies the library's answers), and read_xlsx/write_xlsx/write_xlsx_as use the crate's own table alone. See Microsoft's Formula vs. Formula2 documentation.

Static cell pictures

Store immutable picture payloads in Sheet::cell_pictures, a sparse map keyed by CellRef, shared through Arc. Ordinary cells keep their original size. Content replacement removes the map entry; copy, fill, sort and structural edits explicitly transport it, and undo shares bytes. Serialize the map as address/payload pairs so JSON does not require struct-valued object keys. Keep the scalar #VALUE! fallback used by XLSX consumers that cannot interpret rich images. This first implementation does not extend formula evaluation to rich values or implement IMAGE(); references and text-only exports use the fallback.

Write picture value metadata and existing dynamic-array cell metadata into the same workbook metadata part, with independent index tables. Resolve input parts through their relationships. Support embedded PNG/JPEG files without fetching external resources. See cell pictures for the supported user workflow and current limits.

Picture metadata is optional during XLSX import: part-size limits, record limits and relationship read errors produce warnings and retain scalar cell fallbacks. The generic package size limits remain unchanged. Rebased on main before revision; ODS/XLSB import PRs are still unmerged, and Package::read_limited uses the same implementation as the incoming XLSB PR #101 to avoid duplicate helpers when that work lands.

The UI cache uses a 64 MiB texture budget with at most 4,096 sparse entries. Images used in the current or preceding frame are protected from eviction. If the visible set exceeds the budget, new images show the existing placeholder until capacity is available; header-based admission avoids repeatedly decoding overflow images. More than 128 small visible pictures now keep stable texture handles instead of cycling through the cache every frame.

Ctrl+wheel zoom steps along its own ladder, not Excel's 15-point steps

A Ctrl+wheel notch moves the zoom to the next step on grid::WHEEL_ZOOM_LADDER: 10, 12, 15, 18, 21, 25, 30, 35, 42, 50, 60, 70, 85, 100, 120, 140, 170, 200, 240, 280, 340, 400 (four steps per doubling, 17–25% apart). A step less than 5% away is skipped, except 100% and the 10% / 400% limits, which are never skipped: 121% out goes to 100%, and 99% or 101% towards 100% goes to 100%. The wheel's turning is counted in notches (egui eases each notch in over several frames, and fine-grained wheels send parts of one); a notch steps once 60% of it has turned. With animations on, the zoom then glides to the step over about a tenth of a second (as egui eases a scroll notch), and a notch turned during a glide steps on from the step it's going to. With View › Animations off it goes to the step at once. Whether the zoom glide should have a setting of its own is left to the maintainers.

Before, each frame multiplied the whole-number zoom by egui's eased factor and rounded, so a notch out never undid a notch in (100 → 122 → 101 → 123 → 102). Keeping the zoom unrounded fixed that for a wheel with exact notches, but not for one that sends uneven parts of a notch.

Excel (observed: Microsoft 365, Windows) steps 15 points a notch: 100 → 115 → 130, and from 112% to 115% or 100%. That's a large relative step at low zoom and a small one near 400%; equal relative steps keep each notch feeling the same, and the doublings land on 25, 50, 100, 200 and 400%. The status bar's Zoom In / Zoom Out buttons and Ctrl+Alt+= / Ctrl+Alt+- keep their 10% steps, so the wheel and the buttons now step differently. Whether they should share one ladder (this one, Excel's, or another) is left to the maintainers.

Equations are laid out by RaTeX, behind a guard

GridCraft lays equations out with RaTeX (ratex-parser, ratex-layout, ratex-types, ratex-font, MIT), pinned exactly at 0.1.14. The gate that chose it, rather than an in-house LaTeX subset:

  • Source scan. The five crates hold about 31,700 lines. Outside tests and the bundled command-line tools there are 95 sites that can panic (unwrap, expect, panic!, unreachable!), 87 of them in the parser. Most take an argument the parser has already counted (args.into_iter().next().unwrap()) or compile a fixed regular expression. There is no unsafe, no clock (Instant::now, SystemTime), no threads, and no file or environment access. Indexing is not counted; the fuzz run covers it.
  • Fuzzing. 70,000 random token strings and 50,000 mutations of valid equations: no panic, no input slower than 5 ms. Self-referencing and exponential macros stop at the parser's 1,000-expansion limit, which counts expansions and not their size: a definition whose argument doubles on every call never reaches it. The wrapper therefore refuses macro definitions (\def, \newcommand, \let and the like) before RaTeX sees the source, and caps the padded size of matrices at 2,500 cells. Nesting deeper than 32 is refused ("Recursion limit exceeded"), and the deepest accepted input needs under 256 KB of stack in a release build, well inside the smallest main-thread stack we run on (1 MB on Windows). Absurd sizes (\rule{1e308em}{1em}) lay out with huge dimensions, so the wrapper caps them.
  • wasm. The crates build for wasm32-unknown-unknown.
  • Regular expressions. ratex-parser pulls in fancy-regex (with bit-set 0.8 and a second bit-vec) for one module, its port of mhchem: 62 anchored patterns in src/mhchem/data/patterns_regex.json and a dozen in src/mhchem/patterns.rs tokenise the argument of \ce{…} and \pu{…}, and need lookahead, which the linear-time regex crate does not have (regex and regex-lite serve the parser's other, fixed patterns). Every pattern is compiled with fancy-regex's default backtrack limit of 1,000,000 steps per match; past it the match is reported as no match, so the macro fails instead of hanging. The limit is not reachable from outside the crate. gate.rs times 38 sources of exactly 4,096 bytes shaped to stress the parser and those patterns (nesting at the depth cap, runs of backslashes and braces, (a+)+b, wide and tall environments, chemistry such as \ce{+-+-…}, \ce{111…} and \pu{kJ//…}): the slowest takes 0.8 s in a debug build, with a 10 s bound per source.
  • Syntax. Fractions, roots, scripts, big operators and limits, \left/\middle/\right, matrices (matrix, pmatrix, bmatrix, vmatrix, cases, aligned, array), accents, braces, arrows, \mathbb/\mathcal/\mathfrak, \text, colours and mhchem all lay out.

Because a stray panic is still possible, RaTeX runs only inside gridcraft-math, under catch_unwind, with input capped at 4 KB and depth 32. Its output is converted to plain data and cached on the shape; painting (egui and PDF) reads only that cache, so nothing in a paint path can reach RaTeX. On the web build (wasm32-unknown-unknown) a panic aborts instead of unwinding, so there the caps and the fuzzed input space are the protection. crates/math/tests/gate.rs re-runs a smaller, deterministic version of the gate on every test run, so a version bump has to pass it. RaTeX positions glyphs with KaTeX font metrics; GridCraft never bundles the KaTeX fonts, so glyphs are drawn with system fonts and a glyph no font has is drawn as a [?] box.