Commit Graph
225 Commits
Author SHA1 Message Date
Neil Cawse 319a3e4c2e Merge remote-tracking branch 'upstream/main' into fix/main-ci-csv-tests 2026-11-17 04:15:59 +00:00
Valentin KleinandClaude Opus 5.5 ef99f1e51f Functions: GETPIVOTDATA (#284) (#290)
* Functions: GETPIVOTDATA (#284)

The engine records each PivotTable report value cell (value field,
row/column items, value) on refresh and when a workbook is opened;
GETPIVOTDATA is an evaluator special form that looks the value up by
value field and field/item pairs, #REF! when the report doesn't show it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* GETPIVOTDATA: shared lazy index, report values, hash lookup (review of #290)

- PivotTable::data_index is an Arc-shared PivotLookup (serde-skipped), so the
  per-command workbook clone no longer copies the index; refresh sets it, clear resets it.
- No index work on open: the first GETPIVOTDATA on a pivot without one builds it from the
  source via the engine-registered PIVOT_INDEXER; a failure is #REF!, not an empty index.
- The result is the value the sheet shows at the cell's report position, so formula and
  report cell agree (test with a source edited after the last refresh).
- Lookup by a pre-lowercased hash key; numbers/booleans fall back to a scan on a miss.
- GETPIVOTDATA is no longer volatile: it depends on its pivot's whole report area.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* GETPIVOTDATA: keep the index of the stored report; collision-free keys; align estimates

The lookup index describes the report as last refreshed and is saved with
the pivot. A pivot loaded without one (XLSX) only builds it when the
recomputed layout matches the stored labels. Lookup keys are length-prefixed
and distinguish Boolean from text items. Function-parity estimates match.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* pivot: import PivotDataCell, PivotItemRef and PivotLookup explicitly after upstream dropped the glob import

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* pivot: add an ignored measurement of edits on a sheet with a large report

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* pivot: keep a failed lazy GETPIVOTDATA build retryable, never save an empty index

A failed lazy index build stored unwrap_or_default() into the shared,
first-set-wins PivotLookup, so a valid state coming later (undo, a
source fix, a reopen) could never rebuild it, and the empty list was
saved as a restored index. Cache only validated builds, treat an empty
list as no index in serialization, reject reports with more value
cells than the index cap instead of truncating, register the pivot
indexer in gridcraft-cli check, and add two regression tests.

* GETPIVOTDATA: move cached offsets with edits inside the report; exact numeric items first

Inserting or deleting rows or columns inside a report remapped the cells
and last_range but kept the cached value-cell offsets, so a lookup could
read another item's cell. shift_sheet_features now maps each indexed
cell through the edit into a new index (the shared one stays with the
undo snapshots); a value cell deleted with its line drops out (#REF!).

A numeric item was accepted on a formatted-label hash hit, so 0.6 and 1
both shown as "1" returned the first. A number now prefers the cell
whose item has exactly that value, before any label match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix pivot chart test for the new Output.data field

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 08:59:28 +01:00
Valentin Klein ee24d50ac8 Calc: overlapping spills resolve by anchor order, incremental matches full recalc (#2483)
## What

When two array formulas want the same cells, the one that already has its spill keeps them, whichever anchor comes first (this is what Excel does, measured on this PR by Harrison-Booth). The other formula shows `#SPILL!` until the winner is cleared, then it spills. When neither has a spill yet (a first calculation with no remembered layout), the anchor first in reading order (row, then column) wins, so the result never depends on evaluation order.

Before, an incremental recalculation let the existing spill keep its cells, while a full recalculation laid everything out from scratch and gave them to the earlier anchor. The two disagreed. A full recalculation now starts from the layout the sheet held, so both agree.

Also fixed: clearing a spilling formula did not unblock an array blocked by its old area when that array sat below or to the right of it.

## Why

Issue #2480: on the sales sample, `cell.set C14 =TRANSPOSE(B18:B24)` followed by `range.fill =D14:E18` over E13:F13 left F13 as `#SPILL!` in memory but `0` after a full recalc (the fuzzer, `command_sequence_fuzz` seed 0, hits this on #795). Now C14 keeps its cells, F13 is `#SPILL!`, and a full recalc gives the same.

Known limits: the xlsx reader does not restore spill layouts, so a workbook opened from a file is laid out by reading order only (Excel keeps the layout the file records); and when both anchors already held a spill and one grows into the other, reading order decides (Excel keeps the cells the other owns). The rule is documented on `Standing` in `crates/calc/src/recalc.rs`.

## Test

In `crates/engine/src/tests.rs`: the two-command sequence from the issue; both entry orders of an overlap, plus clearing the winner; and a no-history layout, where the earlier anchor wins in both entry orders. All compare cells with a full recalc.

## How verified

- `cargo fmt --check`, `cargo clippy -p gridcraft-calc -p gridcraft-engine --all-targets -- -D warnings`, `cargo test -p gridcraft-calc -p gridcraft-engine` (includes the fuzz test): all pass.

Fixes #2480
2026-11-15 17:27:51 +01:00
Neil Cawse 027d5d1f43 Merge remote-tracking branch 'upstream/main' into fix/main-ci-csv-tests 2026-11-14 18:05:00 +00:00
Michael HigginsandClaude Opus 5.5 fdba683246 calc: linear dependency extraction for range-operator chains (#2183) (#2198)
* calc: linear dependency extraction for range-operator chains (#2183)

precedents() handled a range operator with non-reference operands by recursing into both operands and then letting Expr::walk descend into them again, so a left-associative chain of k operands cost 2^k (a 60-operand INDEX(...):INDEX(...) chain would hang on open). It is now a hand-written collector that visits every node once, growing the per-sheet bounding boxes of the enclosing range operator as it goes. Past a depth (4096) or node (1,000,000) bound it marks the formula dynamic (always recalculated) instead of recursing further.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* calc: drop wall-clock timing from precedents tests (clippy)

std::time::Instant::now is a disallowed method (clippy.toml: traps on wasm).
The regression test only needs to finish; the old 2^k walk would never return.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* calc: bound precedents to 100 ms and box plain-reference range chains (#2183)

- range_operator_chain_precedents_are_linear times the direct precedents
  call against the 100 ms bound in the issue.
- A Range node with two plain Ref operands (A1:B2:C3, which the parser
  can't fold) now collects its bounding box like any other range node, so
  an edit to C1 recalculates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:53:03 +01:00
Neil Cawse 2c2699d87b Merge remote-tracking branch 'upstream/main' into fix/main-ci-csv-tests
# Conflicts:
#	crates/numfmt/src/render.rs
#	docs/function-parity.md
2026-11-13 19:44:29 +00:00
Dylan M. Taylor a6c331b63f calc: one cumulative work budget per formula, and INDEX decides multi-cell picks from their shape (#3188)
* calc: INDEX decides a multi-cell position from its shape before reading it

* calc: charge each formula's evaluation against one cumulative work budget

* address review: a database function whose computed criterion ran out of work is #CALC!

* address review: give the test's criteria cell its stored value

* address review: stop matching database records once a computed criterion exhausts the work budget

* address review: skip the criteria-row scan once work is exhausted

* address review: stop between conditions once the work budget is spent
2026-11-11 07:17:04 -05:00
Dylan M. Taylor bc2dabc7ea FORMULATEXT and the in-cell editor keep Hidden formulas hidden (#3298)
* FORMULATEXT and the cell editor keep a hidden formula's source hidden

* address review: recalculate formulas when sheet protection changes

* address review: skip the full recalculation for a no-op unprotect

* address review: recalculate on protect/unprotect without marking the edit structural
2026-11-11 05:48:31 -05:00
FredApps a01ff3e7b3 Merge branch 'main' into fix/lookup-blank-result 2026-11-11 07:57:42 +01:00
Harrison Booth 50a9e9ede4 XLS: formulas open as legacy formulas with Excel's implicit @ (#410) (#1367)
.xls formulas are read as legacy formulas with Excel's implicit @ (apply_legacy via read_xls_with), one-cell Ctrl+Shift+Enter arrays keep their range, @ of a 2-D range meets the formula's row and column, and defined-name analyses are shared across the formulas of a pass and replayed exactly.

Closes #410
2026-11-10 15:05:02 +01:00
ca39b5ab9f XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel (#417)
* XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel; implied @ saved back without cm

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* XLSX: long operator chains analysed in full and unfinished legacy analysis leaves the formula unchanged (never saved as SINGLE); @ of a union keeps its parentheses; IF's condition intersected inside SUMPRODUCT

* XLSX: a legacy union already in parentheses saves back without a second pair (never as SINGLE); a name's definition analysed as an array throughout, inserting no @

* XLSX: drop the unreachable keep-unions retry; an @ of a bare union saves as a dynamic formula

* XLSX: an @ keeps parentheses the reader would not put back (redundant ones, a union's)

* XLSX: a redundant @ keeps a legacy formula legacy; IFERROR and IFNA intersected inside SUMPRODUCT; file parentheses and unfinished analyses saved as read

An edit could leave an implied @ on a single cell: deleting rows 2:3 under the legacy SQRT(A1:A3), read as SQRT(@A1:A3), gives SQRT(@A1:A1), and a name redefined to one cell does the same to SQRT(@Rng). The writer kept an @ only when re-analysing the formula without it put the same @ back, which it no longer does for one cell, so the formula was saved as SQRT(_xlfn.SINGLE(A1:A1)) with cm, readable only by dynamic-array Excel. The writer now analyses the formula with its own @ in place: an @ is accepted when the analysis implies it at that position or when its operand is a single value there (such an @ changes nothing), and any @ inserted elsewhere or any other @ still makes the formula dynamic. SQRT(@A1:A1) saves as SQRT(A1:A1), SQRT(@A1) as SQRT(A1) and @SUM(A1:A3) as SUM(A1:A3), all without cm; SUMPRODUCT(@SQRT(A1:A3)) stays dynamic because SQRT(A1:A3) is an array there.

Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT unless the formula is entered with Ctrl+Shift+Enter, like IF's condition: =SUMPRODUCT(IFERROR(VALUE(C19:C21),0)) returns one row's value or #VALUE!, and the accepted fix is CSE or N(+range) (https://stackoverflow.com/questions/59228707). They now get the same single-value parameter class as IF's condition (Param::Single), so the legacy SUMPRODUCT(IFERROR(A1:A3/A1:A3,0)) reads as SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0)), and a typed SUMPRODUCT(IFERROR(VALUE(A1:A3),0)), which dynamic-array Excel evaluates as an array, is saved with cm. IF's result arguments are left as arrays: in https://stackoverflow.com/questions/29682328, a non-CSE =SUMPRODUCT(--(B4=IF(NOT($C$4:$C$11),$B$4:$B$11))) in row 4 returns 2 (both "apple" rows of B4:B11 counted, so the result branch was not intersected), 0 in a row where C is TRUE and #VALUE! outside rows 4:11 (the condition was intersected).

The reader's @ in front of a whole-formula array result took no extra parentheses when the file already had them, so ({1,2}+1) and {1,2}+1 both read as @({1,2}+1), and the writer dropped the file's parentheses. An operand already in parentheses now gets a second pair, so ({1,2}+1) saves as written.

A legacy formula the analysis could not finish (node budget, nesting) was read unchanged, evaluated as a dynamic-array formula and saved with cm, so Excel spilled it. Names were analysed again at every use, so a 399-term name used six times exhausted the 4096-node budget; their kinds are now remembered per formula, and the budget is 16384 nodes. The nesting limit is 128, so Excel's 64 levels of nested functions are analysed. A formula that still exceeds the limits is saved as it was read, without cm.

* XLSX: an implied @ is dropped and any other @ of a single value kept as _xlfn.SINGLE in a legacy formula (as Excel writes it); a legacy formula the analysis cannot finish or whose semantics it does not know opens with a whole-formula @ and never spills; analysis depth follows the parser's limit

* XLSX: a legacy union given to a function's single-value parameter keeps its implied @, so it calculates to #VALUE! instead of spilling and saves as read; checked on a calculated workbook

* XLSX: only a one-cell range an @ applies to counts as one an edit shrank; a reading past the parser's nesting limit leaves the formula as read, and the writer saves the text a formula it cannot analyse was read from

* XLSX: a formula nested past the parser's limit for its reading is left as read (documented: it may spill); test covers it

* XLSX: a name's definition counts its one-cell ranges as collapsed only when it is itself a reference, so CHOOSE's index or IF's condition in it keeps the name legacy

* XLSX: an @ on a name or one-cell range counts as implied when ranges an edit may have shrunk, anywhere the name returns them (CHOOSE's choices, parentheses), make it so; deleting rows keeps such formulas legacy

* XLSX: the collapsed-range retry of an @ runs only when the first pass reached a one-cell range written as a range (tracked through cached names)

* docs: describe unfinished formula export fallback

* docs: the unfinished-analysis fallback differs for formulas with an explicit @

* Preserve main formula analyzer fixes

* Order file-format revision history

---------

Co-authored-by: neilcawse <8085959+neilcawse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Local contributor <contributor@localhost>
2026-11-10 12:43:07 +01:00
Neil Cawse 56a02dcc8c Merge origin/main into fix/main-ci-csv-tests 2026-11-09 01:11:15 +00:00
vangel76andClaude Opus 5.5 642fa97fa5 fix(functions): a blank element of a lifted argument is given, not omitted
CEILING.MATH(6.7,VLOOKUP({"k","k"},A1:B1,2,FALSE)) with B1 empty now gives 0 per
element, as the scalar call does, instead of treating the significance as left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 14:19:05 +01:00
vangel76andClaude Opus 5.5 63150810b3 fix(functions): a blank lookup given as an argument is not an omitted one
FILTER(1,FALSE,VLOOKUP(...)) with a blank lookup result returned #CALC!
because util::has read the blank value as a left-out argument. Arguments
written in the formula that compute a blank are now marked as given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 13:27:08 +01:00
vangel76andClaude Opus 5.5 0f5a860d74 Merge perf/calc-spill-index-2683 (with upstream/main) into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:23:30 +01:00
vangel76andClaude Opus 5.5 91bd22635e Merge upstream/main into fix/lookup-blank-result
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:23:30 +01:00
vangel76andClaude Opus 5.5 17131fb0e5 Merge upstream/main into perf/calc-spill-index-2683
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:18:31 +01:00
vangel76andClaude Sonnet 5.5 865a7f6a06 Merge upstream/main into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 19:09:51 +01:00
vangel76andClaude Sonnet 5.5 1458f45214 Merge upstream/main into perf/calc-spill-index-2683
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 19:09:31 +01:00
vangel76andClaude Sonnet 5.5 42370639e0 Merge upstream/main into perf/calc-range-memo-2681
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:29:04 +01:00
vangel76andClaude Sonnet 5.5 76002fe0bd Reuse remembered range sums for every COUNT block
COUNT never reads the running sum, so the zero-sum gate that keeps SUM
and AVERAGE additions in order does not apply to it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 16:58:19 +01:00
FredApps ee1b5df3db Merge pull request #2693 from vangel76/perf/calc-expr-arc-2685
calc: rebuild the dependency graph from shared syntax trees
2026-11-07 16:51:36 +01:00
Harrison BoothandClaude Sonnet 5.5 292b784f02 fix(calc): apply "Set precision as displayed" in calculation (#1366)
* ft-121.7.9: apply Set precision as displayed in calculation (#1343)

With the option on, a formula result (and each element of a spill) is
rounded to its cell's number format before it is stored and before other
formulas read it. Numbers typed, pasted or reformatted while it is on are
rounded at once, and turning it on through formulas.calculationOptions
{precisionAsDisplayed} rounds every number constant, as Excel does.

The rounding is NumberFormat::round_as_displayed in the numfmt crate,
which shares the renderer's digit logic (fixed, scientific, fraction).
General, date/time, text, digitless and conditional formats keep the full
value, as observed in Excel through COM. Nothing changes when the flag is
off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ft-121.7.9: round conditional formats by the sign's section under precision as displayed (#1343)

Excel does round formats with a condition when "Set precision as
displayed" is on; the PR kept them at full value. Observed through COM
with synthetic values (and in gridcraft-corpus
excel/precision-displayed/pad-on-conditional-sections.xlsx): conditions
are ignored and the sign picks the section, the first for positives and
the second for negatives. `[<1]0.00;0` shows 2+1/3 as 2 but stores 2.33,
and stores -10/3 as -3.

Two more observed details: with a condition and a single number section,
negatives keep the full value (`[<1]0.00`), unless an `@` section follows
(`[<1]0.00;@` rounds them by the first section); and a completely empty
section passes to the next one (`0.0;;0.000` stores -2.333 as -2.333,
`;0.0` stores 2.333 as 2.3), for formats with or without a condition.

Display is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* engine: recalculate spill anchors when a spill cell is reformatted under precision as displayed

round_edited_as_displayed only looked at stored-cell changes, but a spill
destination lives in Sheet::spill, so reformatting it (or its row or column) left
the anchor out of the changed set and the element unrounded. Add each spill anchor
whose range holds a changed cell or a changed row/column style to the changed set.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* numfmt: drop a cast the fraction merge made redundant

* precision as displayed: keep locked constants on protected sheets, round new sheets, fix manual-mode comment

* precision as displayed: name the input sheet in the new-sheet histogram test

* precision as displayed: use six values in the new-sheet histogram test

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:24:49 +01:00
Dylan M. Taylor 632e4a1bc5 formula, calc: LET/LAMBDA names looked up through a map; ragged array literals refused before reserving (#3312) 2026-11-03 22:57:52 -05:00
Dylan M. Taylor db05400e67 calc: list each dependent once when a change spreads (#3190)
* calc: list each dependent once when a change spreads

* calc: check the dependency closure against a walk of every cell

Random graphs of single-cell, narrow and wide range reads (some removed or unsettled), random seeds and changed areas: the closure with shared taken records and area lookups equals the one listing every dependent of every changed cell.
2026-11-03 22:51:29 -05:00
Dylan M. Taylor c0ea297615 calc: cap the text one & over arrays may build in all (#3189)
* calc: cap the text one & over arrays may build in all

* address review: concatenation over arrays stops at the byte budget without reserving the whole broadcast
2026-11-03 22:51:11 -05:00
Hector VillarrealandClaude Opus 5.5 2caf435a38 #2012: split a parsed sheet-qualified name at the last !, sheet as is
Addresses review comments 4455115540 and 4455115557. The parser stores
`'Sheet'!Name` as `Sheet!Name` with the sheet already unquoted:

- Evaluator::name split it at the first `!`, so a series named
  `'a!b'!Title` showed #NAME?.
- chart_name unquoted it again, trimming spaces, so a chart reference
  to `' Sales '!Data` was saved as `Sales!Data`.

Both now split at the last `!` and look the sheet up as is.

Tests: chart tests::a_series_named_by_a_name_on_a_sheet_with_a_bang_resolves,
xlsx tests::chart_book_names::chart_names_keep_spaces_around_a_quoted_sheet_name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 07:21:22 -06:00
vangel76andClaude Sonnet 5.5 bcad4a75cf Merge calc: register a Python result's spill in the spill index (#2694) into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:32:11 +01:00
vangel76andClaude Sonnet 5.5 e85b24df57 calc: register a Python result's spill in the spill index
python_result wrote into spills but not into spill_index, so indexed readers
(cell_value, range_values, visit_range) missed a Python array laid out in the
same pass. Insert on spill, remove when blocked, as the formula path does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:31:52 +01:00
vangel76andClaude Sonnet 5.5 00f565bc11 Factor out types clippy flags as too complex
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:44:16 +01:00
vangel76andClaude Sonnet 5.5 4b98b11996 Merge upstream/main into perf/calc-range-memo-2681
A Python cell is not a constant for the range_sum block memo.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:41:44 +01:00
vangel76andClaude Sonnet 5.5 21504859c8 Merge upstream/main into perf/calc-expr-arc-2685
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:36:41 +01:00
vangel76andClaude Sonnet 5.5 a9db1da90d Merge upstream/main into perf/calc-spill-index-2683
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:34:16 +01:00
vangel76andClaude Sonnet 5.5 96672e8ad5 Drop helpers the merged range reads no longer use
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:33:53 +01:00
vangel76andClaude Sonnet 5.5 628da5117a Merge upstream/main into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:28:52 +01:00
Neil Cawse d95c11b370 Merge upstream main into fix/main-ci-csv-tests 2026-11-02 00:46:06 +00:00
prxnvt 11b55e0c71 Calc: new Python code recalculates the readers of the cell's old spill
python.set, and Replace in a Python cell's code, removed the cell's spill
registration and its spilled values before the recalculation ran, so the
recalculation never saw the freed range and formulas reading those cells kept
their old values until a full recalculation. The commands now leave the spill
to the recalculation. When a Python cell's content changes and it has no array
result to lay out, cells_changed removes the spill and queues the freed range,
as it does for a formula replaced by a constant, so its readers are
recalculated; a cell that keeps its array result (the same code again) keeps
its spill and lays it out again. Clearing, pasting over and blocking a Python
spill already recalculated its readers, and undo and redo restore the values
along with the cells; a test covers them.
2026-11-01 14:41:48 -05:00
prxnvt cd51fe110d Calc: a range read walks only the spills it overlaps
A range read during recalculation walked every array laid out in the pass on
the sheet, whatever the range, and a block read (visit_range) gave up its fast
path as soon as any array was laid out there. A Python cell's cached result
can be up to 1,000,000 cells, so 1,000 formulas reading small ranges beside it
could visit about a billion values in one full recalculation; a large formula
array costs the same. Both reads now look only at the arrays whose area meets
the range, and only at the part inside it. The spilled values the sheet stores
from the last calculation are found row by row when the range has fewer rows
than there are spilled values, so a read beside them doesn't scan them either.
2026-11-01 14:06:02 -05:00
prxnvt b3a3590db6 Merge main into Python cells as data 2026-11-01 14:03:09 -05:00
prxnvt c03c61fd05 Merge main into Python cells as data 2026-11-01 12:00:09 -05:00
Max Murphy 81aee9e075 calc: form the MAP result size in u64 so wasm32 cannot wrap the cap (#2165)
`broadcast_len`, the MAX_CELLS guard shared by MAP, the binary operators,
TEXT and array IF, multiplied `rows` and `cols` as `usize`. On wasm32 a
broadcast such as 65536 x 65536 (2^32 cells) wraps that product to 0 and
slips past the cap, after which the allocation and loop run far past
MAX_CELLS. The product is now formed in u64 and converted back only after
the cap check, so the guard means the same on every target.

`single`, the per-cell scalar check in MAP and MAKEARRAY, compared
`rows * cols` with 1. A constructed array stores exactly `rows * cols`
values, so it now tests `data.len()` directly: the same answer without an
unchecked product.

Tests cover MAX_CELLS exactly, one cell over, the 2^32 pair that wraps to
zero on a 32-bit usize, a full column x row (2^34), and MAP itself just
over the cap and at 16384 x 16384.

Closes #2165
2026-10-31 16:41:24 +01:00
Neil Cawse cb4ac4d554 Merge main (1f4c67e) into the branch 2026-10-31 02:07:05 +00:00
Brandon Thomas c305315610 Merge PR #2719 2026-10-30 15:42:10 -04:00
vangel76 95ba076778 Merge remote-tracking branch 'origin/perf/calc-spill-index-2683' into HEAD 2026-10-30 00:04:34 +01:00
vangel76andClaude Sonnet 5.5 bdbd29e623 perf(calc): size the spill-index decision by the sheet's own changed cells
A batch spread over several sheets used its total length for every sheet, so a
one-cell sheet with many spills built the block index as if it had the whole
batch's lookups.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 00:04:25 +01:00
vangel76andClaude Sonnet 5.5 c1e1c7bdc9 Merge origin/perf/calc-spill-index-2683 into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:49:01 +01:00
vangel76andClaude Sonnet 5.5 6c912d2eb1 perf(calc): keep tall and wide spill areas out of the block index
A full-height column spill took 65,536 block registrations, on every pass
that produced it. Areas over 1024 blocks are now tested directly by each
lookup instead. Adds a randomized comparison against a scan of all areas.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:48:11 +01:00
vangel76 1801f4a7cf Merge upstream/main into perf/calc-spill-unblock-2684 2026-10-29 23:09:25 +01:00
vangel76 c3865b5dbd Merge upstream/main into perf/calc-spill-index-2683 2026-10-29 23:07:21 +01:00
vangel76andClaude Sonnet 5.5 b6bbdcd5cf calc: build precedents from a borrowed syntax tree, no Arc for text-only formulas
Formula::with_expr runs a closure on the cached tree, or on a freshly parsed
one that is dropped afterwards, so rebuild and cells_changed need neither a
tree copy nor an Arc allocation / reference count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 00:51:45 +01:00