A sheet set to First page number 7 has pageSetup firstPageNumber="7"
useFirstPageNumber="1"; neither attribute was read, so a save reset it
to Automatic. PrintSettings now holds it (None for Automatic), the
reader takes the number when useFirstPageNumber is on, and the writer
writes both in the schema's order. A JSON workbook leaves it out when
Automatic, so existing files are written as before.
Fixes#4293
* Functions: GETPIVOTDATA (#284)
The engine records each PivotTable report value cell (value field,
row/column items, value) on refresh and when a workbook is opened;
GETPIVOTDATA is an evaluator special form that looks the value up by
value field and field/item pairs, #REF! when the report doesn't show it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* GETPIVOTDATA: shared lazy index, report values, hash lookup (review of #290)
- PivotTable::data_index is an Arc-shared PivotLookup (serde-skipped), so the
per-command workbook clone no longer copies the index; refresh sets it, clear resets it.
- No index work on open: the first GETPIVOTDATA on a pivot without one builds it from the
source via the engine-registered PIVOT_INDEXER; a failure is #REF!, not an empty index.
- The result is the value the sheet shows at the cell's report position, so formula and
report cell agree (test with a source edited after the last refresh).
- Lookup by a pre-lowercased hash key; numbers/booleans fall back to a scan on a miss.
- GETPIVOTDATA is no longer volatile: it depends on its pivot's whole report area.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* GETPIVOTDATA: keep the index of the stored report; collision-free keys; align estimates
The lookup index describes the report as last refreshed and is saved with
the pivot. A pivot loaded without one (XLSX) only builds it when the
recomputed layout matches the stored labels. Lookup keys are length-prefixed
and distinguish Boolean from text items. Function-parity estimates match.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* pivot: import PivotDataCell, PivotItemRef and PivotLookup explicitly after upstream dropped the glob import
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* pivot: add an ignored measurement of edits on a sheet with a large report
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* pivot: keep a failed lazy GETPIVOTDATA build retryable, never save an empty index
A failed lazy index build stored unwrap_or_default() into the shared,
first-set-wins PivotLookup, so a valid state coming later (undo, a
source fix, a reopen) could never rebuild it, and the empty list was
saved as a restored index. Cache only validated builds, treat an empty
list as no index in serialization, reject reports with more value
cells than the index cap instead of truncating, register the pivot
indexer in gridcraft-cli check, and add two regression tests.
* GETPIVOTDATA: move cached offsets with edits inside the report; exact numeric items first
Inserting or deleting rows or columns inside a report remapped the cells
and last_range but kept the cached value-cell offsets, so a lookup could
read another item's cell. shift_sheet_features now maps each indexed
cell through the edit into a new index (the shared one stays with the
undo snapshots); a value cell deleted with its line drops out (#REF!).
A numeric item was accepted on a formatted-label hash hit, so 0.6 and 1
both shown as "1" returned the first. A number now prefers the cell
whose item has exactly that value, before any label match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Fix pivot chart test for the new Output.data field
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The package reader reports a part over the limit as "expands to more
than <limit> bytes"; the test added in #2491 expected the declared size
and failed on main.
Measured with crates/engine/examples/bench.rs (perf/xtask-bench), 100k rows, M2 Pro:
enter.formulas 465 -> 420 ms (23829d8c: 375), xlsx.save 540 -> 488 ms (466).
- number_to_text: the round-up-to-1E+15 check (f29ac7d9e) formatted every number with
exact {:.14e} and parsed it back; formula printing calls it once per literal. Run it
only for magnitudes that can round up (>= 9.9999999999999e14). Same output; boundary
tests added.
- Expr clone/map, xlsx prefix_params: a binary node whose left operand is not a chain
no longer allocates a spine Vec (2aeb21d4d); map rewrites the operands in their boxes.
Expr::walk reserves room for a typical formula up front.
- grow_autofilter checks the sheet has an AutoFilter before looking up the cell.
- retained_bytes_against looks up the newer band's row once per row, not per cell.
- The journal measures an entry's JSON size with a counting writer instead of building
the text (a big paste's params are MBs).
The rest of the enter.formulas gap and all of edit.fanout's (~+12%) is Undo's byte
budget (72db972de, #1227): sizing each snapshot and freeing evicted ones, an intended
cost of bounded memory.
Co-authored-by: nihilist-narwhal <240798405+nihilist-narwhal@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
read_cache turned every pivot cache error into absence with
optional_xml(part).ok().flatten(), so a cache over the part size limit
dropped its PivotTable with no warning, and a malformed or missing cache
only left a generic "skipped part" line. Each case now adds a warning that
the PivotTable was kept as plain cells, and the workbook keeps opening
(the oversized part is refused before it inflates).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
craft-xls reads Dv/DVal records (storytold/craft-libs#999, a port of #3636).
xls.rs maps each rule onto the model as the .xlsx reader maps a dataValidation:
a non-literal list source becomes =ref. The tests are #3636's, run through
read_xls; the cap test now expects craft-xls's skipped-rules warning.
Co-authored-by: kiwi4boy <kiwi4boy@gmail.com>
package.rs keeps GridCraft's Package, Rel and ZipOut, now on craft_opc::LazyPackage
and craft_opc::ZipOut (on craft-zip). Limits stay GridCraft's (200,000 entries,
512 MB a part, 1 GB a package) and so do the budget rules, which craft-opc took
from GridCraft. The budget tests moved with them. normalize, resolve and
split_dir are craft-opc's (the same code).
Every craft-libs crate moves to aa84415b. craft-xls now reads data validation;
until GridCraft maps it, a sheet with rules still warns that .xls data validation
is not imported. ods.rs matches entity names through a slice, since the new
dependencies make the old as_ref() call ambiguous.
Decrypting password-protected workbooks ([MS-OFFCRYPTO], Agile and Standard) moved
to storytold/craft-libs as craft-offcrypto (#931), on craft-cfb. GridCraft keeps
its DecryptError and the engine's messages; crypto.rs maps the crate's errors onto
it. The private compound-file reader and its tests are gone: sniffing looks for
the encryption streams with craft_cfb::has_entry_named (with the same fallback for
a damaged directory), and the test builders encrypt with craft-offcrypto and write
compound files with craft-cfb.
One difference, as craft-offcrypto has it: the dataIntegrity HMAC key is used at
the length it decrypts to (up to the hash size); GridCraft accepted only a
salt-size or hash-size key. Both verify what Office writes.
Text decoding (byte order marks, UTF-8, Windows-1256/1252), delimiter sniffing,
the record parser and field quoting moved to storytold/craft-libs as craft-csv
(#889), with the SYLK reader as craft_csv::sylk. GridCraft keeps typed values,
date-order inference, the used-range limit and the sheet writer in csv.rs, and in
io/sylk.rs the workbook a SYLK file becomes: formulas parsed (an unparsable one
is skipped and counted, as before), error values, number-format styles and column
widths.
The BIFF8 reader moved to storytold/craft-libs as craft-xls (#878). GridCraft
keeps the mapping onto its model in xls.rs: formulas parsed, Excel's implicit @
for legacy formulas, pixel widths and heights, styles interned per XF, sheet-name
checks, print areas and titles, calculation settings and GridCraft's warning text.
The workbook now opens on the sheet it was saved on when that sheet is visible,
else on the first visible sheet (from #3551 by @Morzilla, whose test is kept).
RString cells are read as text.
Every craft-libs crate moves to one revision, 119415f4.
* xlsx: share each custom number format across cell formats, cap font names at 31 characters
Every cell format record copied its font (name included) and built its
own copy of its number-format code, so many short records naming one
long font or code multiplied it in memory. Custom codes are now built
once and shared, and font names are cut to Excel's 31 characters.
* xlsx: drop stray worktree entries added by mistake
* address review: compare truncated theme font for the scheme link; intern inline dxf number formats
* xls: charge expanded shared-formula text to the import's text allowance
Shared-formula followers each kept their own printed formula text, but
none of it counted against the 256 MB text allowance, so compact PtgExp
records could expand into gigabytes of retained formulas. Each attach
now charges its text; past the allowance the formula is dropped and the
cell keeps its cached value.
* address review: keep resolving array formulas after the shared-formula text allowance is spent
* address review: keep the shared-formula text cut-off across worksheets
* address review: expand shared formulas only after every sheet's mandatory text is charged
* address review: refuse duplicate sheet names at import; adapt tests to read_xls_budgeted
* xlsx: resolve cell pictures through a per-structure layout and a relationship index
Every rich value rebuilt its structure's whole key list before checking
its value count, and scanned the image relationship table for a
duplicate id. Structures are now laid out once and relationships are
indexed by id (a repeated id stays invalid), so reading cell-picture
metadata is linear in its records.
* address review: wide-structure fixture uses the right count and loads a valid picture
* xlsx: analyse each kept chart sheet's graph once per save
chart_sheet_usable walked a chart sheet's relationship graph and parsed
its charts for every workbook entry, so many entries naming one shared
graph multiplied the work. The referenced sheet names are now cached by
relationship id.
* address review: cache the chart-sheet usability verdict per relationship id instead of every reference
* address review: cache chart-sheet analysis by the graph root, not the relationship id
* xlsx: find a reused picture through a content hash instead of comparing every media part
Out::image_part compared each new picture byte by byte with every
media part written so far, so a workbook of many distinct pictures
sharing a long prefix was exported in quadratic time. Media parts are
now indexed by extension and SHA-256.
* address review: index preserved media parts when they are added
* address review: do not hash an image twice when it is first added
* xlsx: one hyperlink allowance per sheet, and a copy limit for unparsable shared masters
Every <hyperlinks> element of a worksheet started a fresh cell and byte
allowance although all of them fill the same map. The sheet now keeps
one allowance across its elements.
Followers of a shared master the parser refuses each copied its full
text; a sheet full of followers of a long master multiplied that text
without bound. The copies are now charged against a 64 MiB per-sheet
limit, past which the followers keep their cached values.
* address review: keep the cached value of a follower dropped for the shared-formula copy limit inside array ranges
* Get Data: refuse an import whose dense rows would be too large
data.getData and data.fromTextCsv built every row densely before
writing it: JSON records were padded to the union of all keys (found
with a quadratic Vec::contains scan), and CSV cells were expanded over
the whole used rectangle. A few sparse records or one far-away CSV
cell therefore allocated billions of slots.
The importer now collects JSON keys with a HashSet, refuses more keys
than a sheet has columns, and checks the projected rows x columns
against MAX_IMPORT_CELLS (10,000,000) before building any row.
* address review: stop collecting JSON keys at the column limit and reject oversized row counts before building rows
* address review: enforce the CSV import cell limit while parsing
* Numfmt: one built-in format table, no dead render arms, fewer allocations (#817)
S1: the built-in format table lives only in gridcraft-xlsx (file formats do not
depend on numfmt); numfmt's unused copy and its exports are removed, xlsx gets
tests pinning its lookup rules.
S3: drop the render_number arm that printed what the fallback prints; literal
text is appended to the output instead of built as a String per call.
O3: render_number keeps pieces, text-before and literal replacements in one
buffer (Parts), fill_int writes digits left to right, Section stores the date
flags (sub-second digits, calendar, AM/PM) computed at parse time, render_date
writes into one buffer.
O4: significant_digits formats into a stack buffer; format_general_fit re-rounds
the 15 digits per width into reused strings.
An equality test compares format_value, TEXT() and format_general_fit with
digests recorded from the previous implementation.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* xlsx test: builtin ids 42 and 44 are written as custom formats like 5-8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* numfmt: push_str for short month/day names (clippy)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
.xls formulas are read as legacy formulas with Excel's implicit @ (apply_legacy via read_xls_with), one-cell Ctrl+Shift+Enter arrays keep their range, @ of a 2-D range meets the formula's row and column, and defined-name analyses are shared across the formulas of a pass and replayed exactly.
Closes#410
* XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel; implied @ saved back without cm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* XLSX: long operator chains analysed in full and unfinished legacy analysis leaves the formula unchanged (never saved as SINGLE); @ of a union keeps its parentheses; IF's condition intersected inside SUMPRODUCT
* XLSX: a legacy union already in parentheses saves back without a second pair (never as SINGLE); a name's definition analysed as an array throughout, inserting no @
* XLSX: drop the unreachable keep-unions retry; an @ of a bare union saves as a dynamic formula
* XLSX: an @ keeps parentheses the reader would not put back (redundant ones, a union's)
* XLSX: a redundant @ keeps a legacy formula legacy; IFERROR and IFNA intersected inside SUMPRODUCT; file parentheses and unfinished analyses saved as read
An edit could leave an implied @ on a single cell: deleting rows 2:3 under the legacy SQRT(A1:A3), read as SQRT(@A1:A3), gives SQRT(@A1:A1), and a name redefined to one cell does the same to SQRT(@Rng). The writer kept an @ only when re-analysing the formula without it put the same @ back, which it no longer does for one cell, so the formula was saved as SQRT(_xlfn.SINGLE(A1:A1)) with cm, readable only by dynamic-array Excel. The writer now analyses the formula with its own @ in place: an @ is accepted when the analysis implies it at that position or when its operand is a single value there (such an @ changes nothing), and any @ inserted elsewhere or any other @ still makes the formula dynamic. SQRT(@A1:A1) saves as SQRT(A1:A1), SQRT(@A1) as SQRT(A1) and @SUM(A1:A3) as SUM(A1:A3), all without cm; SUMPRODUCT(@SQRT(A1:A3)) stays dynamic because SQRT(A1:A3) is an array there.
Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT unless the formula is entered with Ctrl+Shift+Enter, like IF's condition: =SUMPRODUCT(IFERROR(VALUE(C19:C21),0)) returns one row's value or #VALUE!, and the accepted fix is CSE or N(+range) (https://stackoverflow.com/questions/59228707). They now get the same single-value parameter class as IF's condition (Param::Single), so the legacy SUMPRODUCT(IFERROR(A1:A3/A1:A3,0)) reads as SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0)), and a typed SUMPRODUCT(IFERROR(VALUE(A1:A3),0)), which dynamic-array Excel evaluates as an array, is saved with cm. IF's result arguments are left as arrays: in https://stackoverflow.com/questions/29682328, a non-CSE =SUMPRODUCT(--(B4=IF(NOT($C$4:$C$11),$B$4:$B$11))) in row 4 returns 2 (both "apple" rows of B4:B11 counted, so the result branch was not intersected), 0 in a row where C is TRUE and #VALUE! outside rows 4:11 (the condition was intersected).
The reader's @ in front of a whole-formula array result took no extra parentheses when the file already had them, so ({1,2}+1) and {1,2}+1 both read as @({1,2}+1), and the writer dropped the file's parentheses. An operand already in parentheses now gets a second pair, so ({1,2}+1) saves as written.
A legacy formula the analysis could not finish (node budget, nesting) was read unchanged, evaluated as a dynamic-array formula and saved with cm, so Excel spilled it. Names were analysed again at every use, so a 399-term name used six times exhausted the 4096-node budget; their kinds are now remembered per formula, and the budget is 16384 nodes. The nesting limit is 128, so Excel's 64 levels of nested functions are analysed. A formula that still exceeds the limits is saved as it was read, without cm.
* XLSX: an implied @ is dropped and any other @ of a single value kept as _xlfn.SINGLE in a legacy formula (as Excel writes it); a legacy formula the analysis cannot finish or whose semantics it does not know opens with a whole-formula @ and never spills; analysis depth follows the parser's limit
* XLSX: a legacy union given to a function's single-value parameter keeps its implied @, so it calculates to #VALUE! instead of spilling and saves as read; checked on a calculated workbook
* XLSX: only a one-cell range an @ applies to counts as one an edit shrank; a reading past the parser's nesting limit leaves the formula as read, and the writer saves the text a formula it cannot analyse was read from
* XLSX: a formula nested past the parser's limit for its reading is left as read (documented: it may spill); test covers it
* XLSX: a name's definition counts its one-cell ranges as collapsed only when it is itself a reference, so CHOOSE's index or IF's condition in it keeps the name legacy
* XLSX: an @ on a name or one-cell range counts as implied when ranges an edit may have shrunk, anywhere the name returns them (CHOOSE's choices, parentheses), make it so; deleting rows keeps such formulas legacy
* XLSX: the collapsed-range retry of an @ runs only when the first pass reached a one-cell range written as a range (tracked through cached names)
* docs: describe unfinished formula export fallback
* docs: the unfinished-analysis fallback differs for formulas with an explicit @
* Preserve main formula analyzer fixes
* Order file-format revision history
---------
Co-authored-by: neilcawse <8085959+neilcawse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Local contributor <contributor@localhost>
* M12.AR4: preserve Arabic logical text in bidi display and editing
* M12.AR4: drop the stale "no bidi" note from the RTL sheet status
* M12.AR4: open the Arabic editors and place the caret through key and pointer events in tests
* M12.AR4: skip the conditional-format lookup in the formula bar unless a rule sets reading order
* M12.AR4: test that the caret after typed Arabic is drawn at the text's left end (#1070)
* M12.AR4: pass the theme body font in the reading-order style test
* M12.AR4: adapt the Arabic UI tests to egui 0.37 (row direction, editor ids, Ctrl+End to the logical end)
* M12.AR4: read the active sheet through Workbook::active_sheet() in the formula bar
main made `Workbook::active_sheet` an `ActiveSheet` (shared between windows on one
workbook); the formula bar's reading-order lookup takes the index from the accessor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M12.AR4: keep explicit Left on wrapped right-to-left cells and trailing spaces in the editor
Wrapped cell text lays out every row from the left when the painted alignment is Left, instead of aligning right-to-left rows to their start. The cell and formula bar editor keep trailing whitespace in their layout so a right-to-left entry ending in a space keeps the caret inside the text.
* M12.AR4: keep Right and General on right-to-left sheets aligned right in wrapped cells
* M12.AR4: keep right-aligned Arabic with a marked first letter inside its cell
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
textNoShape is the standard no-warp value, so it alone no longer makes a
shape WordArt: only a rectangle that is a text box (as the writer saves
WordArt) or has no fill and no outline reads as WordArt. A filled ellipse
with an explicit textNoShape now reads and saves as the same ellipse.
The PDF/print path drew WordArt text without its outline, so Mint
Outline (white text, green outline) printed as a blank page. The outline
is now drawn like on screen: the text in the outline colour, offset in
eight directions, under the filled text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* xlsx: stop memory amplification from declared sizes (#2173)
Hyperlinks: drop targets over 8,192 bytes before copying them into each cell, cut ScreenTips to 255 characters, and count the 10,000-cell cap across all <hyperlinks> blocks of a sheet. Conditional formats: cap the sqref copies made by extra rules at 1M ranges per sheet. Styles: build each custom number format once and share its Arc; cut font names to 255 characters. Shared formulas: keep cached values for dependents of over-long masters, and charge each expansion against a 2 GiB per-workbook budget.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Import the hyperlink caps in sheet_read/parts after the merge
Upstream switched parts.rs to an explicit super:: import list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* xlsx: charge conditional-format ranges per sheet; cap number format codes (#2173)
- read_cf keeps a cumulative per-sheet count of the range cells retained
rules hold, instead of multiplying the current block by the rule count,
and charges only rules that are kept.
- Custom and inline number format codes longer than 255 characters (Excel's
limit) are dropped with a warning; their cells show General.
- The shared NumFmt map is built only for ids a cell or differential
format refers to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* xlsx: keep over-long number format codes as stored (#2173)
an_overlong_number_format_code_does_not_stop_the_open requires a code
past 255 characters to round-trip; sharing one NumFmt per referenced id
already removes the amplification. Keep the referenced-id sharing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* xlsx: bound only what a shared formula costs beyond an ordinary cell; count characters for the length warning
Review follow-up for #2197 (@skylartaylor):
- Each dependent of a shared formula now pays only its cost beyond an ordinary formula cell
(SHARED_FORMULA_FREE_COST, 4 KiB by shared_cost), and the per-workbook budget for that excess
drops from 2 GiB to 256 MiB. Many ordinary shared formulas still all expand; a long master
copied into thousands of cells stops at the budget and keeps the cached values.
- The over-long-master warning now uses the parser's own test (bytes, then characters), so an
unparseable multibyte master no longer gets a false length warning.
Refs #2173
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
* Charts: draw every doughnut series as a ring (#1827)
* Charts: submit doughnut hole size and first slice angle when the drag ends (#1827)
---------
Co-authored-by: alosslessdev <alosslessdev@users.noreply.github.com>