Commit Graph
22 Commits
Author SHA1 Message Date
Dylan M. Taylor ef87aaa590 test(xlsx): retain XML predicate for escaping oracle 2026-11-16 19:45:33 -05:00
Dylan M. Taylor faef1a06c8 xlsx: use shared SpreadsheetML string codec in reader and writer 2026-11-16 19:40:39 -05:00
Dylan M. Taylor f22b4ed9dd xlsx: import shared append escaping only for tests 2026-11-15 14:07:48 -05:00
Dylan M. Taylor d593a8f99d Use existing craft-xml escaping for XLSX output 2026-11-15 10:19:11 -05:00
FredApps 73fad54671 Merge origin/main into pr/form-controls 2026-11-05 04:05:43 +01:00
skylar taylor-barrick e8bdd0efd6 Merge pull request #2527 from RedBearAK/xml-node-cap
XLSX: the XML element cap counts only elements kept in the tree
2026-11-05 00:29:00 +01:00
prxnvt 02df071b4c XLSX: a kept VML drawing drops a shape as a note only when its client data's ObjectType is Note, so a control whose caption or other attribute reads like one is kept 2026-11-02 15:18:54 -05:00
prxnvt 295c57535c XLSX: form controls (control properties, <controls> and VML ClientData), alongside kept controls
Buttons, scroll bars (sliders) and spin buttons are written the three
ways the published formats describe: a control properties part
(xl/ctrlProps/ctrlPropN.xml, formControlPr with objectType, fmlaLink,
min, max, inc, page, val and horiz), a <control> in the sheet's
<controls> inside mc:AlternateContent with its anchor, print and locked
flags and a button's macro="[0]!Name", and a VML shape with
x:ClientData. Notes and controls share one VML drawing per sheet; shape
ids come from 1024-id blocks listed in the drawing's idmap (after the
blocks kept header and footer drawings use), notes first, so each
control's shapeId names its VML shape.

Reading prefers the properties part, takes the caption from the VML
text box and falls back to the VML alone for files that only have it.
Values are clamped to whole numbers 0..30000 and repaired to pass the
model's rules.

Form controls of other kinds (check boxes, list boxes, drop-downs) stay
with pass-through: the reader returns the shape numbers it modelled,
and pass-through keeps the rest of the <controls> element and VML
drawing without them. On save, modelled controls join the kept
<controls> element and take a fresh shape block in the kept drawing, so
a sheet with both round-trips without duplicates. Properties part names
are reserved so kept parts of other sheets don't collide with them.

Button actions xlsx has no place for (commands, fixed inputs, output
cells) travel in a chunked hidden name keyed by sheet and position, and
are taken back out on open. The streaming sheet parser now splits only
shallow <row> elements, so an anchor's <xdr:row> inside <controls> stays
in the tree.
2026-10-31 16:11:54 -04:00
NetterSilverandClaude Opus 5.5 393a524aa3 Speed up XLSX worksheet writing
Saving a large workbook spent about half its time building worksheet XML,
mostly in small per-cell allocations. The writer now appends to the part
directly:

- Cell references, row numbers, style ids, shared string indexes and
  numbers are appended without temporary strings (push_a1, push_u64,
  push_num), and text is escaped in place (push_esc, with a fast path for
  plain ASCII).
- sheetData walks the cell store in its own row-major order instead of
  copying every address into a BTreeSet and a per-row BTreeMap and looking
  each cell up again. Cells outside the store (pictures, spill values,
  table headers) still go through the sorted set.
- Formulas with nothing the file spells differently (no LET/LAMBDA, no
  _xlfn function, no spill, @, trim operator or structured reference) are
  printed from the shared expression instead of being cloned and rebuilt
  twice by the file rewrite.
- The A1 printer in gridcraft-formula appends references to the output
  instead of formatting each anchor into its own string.

The output is byte-for-byte unchanged.

write_perf, 200,000 rows (2.0M cells, 14.2 MB package), release, Ryzen
16 threads / Linux, 36 interleaved runs each:
  before: median 2083.9 ms, min 1615.7 ms
  after:  median 1501.3 ms, min 1192.3 ms  (-28.0 % median, -26.2 % min)

What remains is mostly deflate (zlib-rs, level 6), about 70 % of the save.

Tests pin the exact sheetData of a workbook covering row settings between
cell rows, spill values, every value kind, number spellings, escaping and
formulas with and without a file spelling (generated with the previous
writer), and check each appending helper and the formula fast path against
the code they replace.

The formula fast path is guarded by a test that runs both paths over every
function in the gridcraft-functions registry (a new dev-dependency, allowed
by the layer rules), in 15 surrounding shapes covering every operator and
rewrite, so a function or rewrite added later without updating the trigger
list fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 16:46:25 +01:00
3b65b9af58 Sonar maintainability: explicit imports, method paths, smaller functions (#2412)
* Sonar: explicit imports, fn paths for closures, extract helpers for cognitive complexity

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Fix imports

* Fix serde imports

* Fix engine imports

* Fix clippy gate and a type error from the explicit-import change

Remove the unused imports the review flagged (cf.rs, pivot.rs, tables.rs, cmd/sheet.rs) so clippy -D warnings passes, and point the backstage parent-folder lookup at Path::to_str, which the fn-path rewrite had broken.

Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Apply rustfmt to the explicit-import cleanup so the fmt gate passes

* Share the per-directory font path expansion in theme.rs

The candidate font lists repeated the same loop over the Linux font
directories in every role. A single helper expands them in the same order,
so the lookup order is unchanged and the duplicated blocks go away.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Read the Windows fonts folder in one place in theme.rs

Every script-font list repeated the same WINDIR lookup. A single helper
returns the same path, so the duplicated lines go away without changing
which font files are tried.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Build the light and dark UI palettes from one table in theme.rs

Tokens::light() and Tokens::dark() listed the same 40 colour fields in the
same order, so every change had to be made twice. A single palette
constructor takes the mode and picks each colour's value, so each field is
written once and both modes keep their exact colours.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Share the Noto CJK font list between the CJK script lists in theme.rs

The Japanese, Simplified and Korean lists each repeated the same loop over
the Linux CJK directories with a different locale file name. One helper
builds that loop, and each list keeps its own face index, so the lookup
order is unchanged.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Share Linux Noto directory list across Thai and Hebrew font lookups

Removes the duplicated directory arrays flagged by SonarCloud new-code duplication.

* Reuse in_dirs for Thai and Hebrew Noto font candidates

Removes the remaining duplicated per-directory loops flagged by SonarCloud.

* Share one font-list builder for the Thai and Hebrew faces

The two per-script functions repeated the same Windows, macOS, Linux and
variable-font setup. A single script_fonts helper now builds both lists,
keeping the candidate order unchanged, which clears the Sonar duplication gate.

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
2026-10-25 12:32:02 -04:00
RedBearAKandClaude Fable 5.1 0b7deaa3d8 XLSX: the XML element cap counts only elements kept in the tree
`parse_streaming` counts every element it attaches to a parent against
MAX_XML_NODES, but the cells of a worksheet are attached to their `row`
while the row is open, and the row is then handed to the callback and
dropped. So the count grew with every cell of a sheet although the tree
never held more than one row, and a workbook with a sheet of about one
million ordinary cells (12,274 rows by 103 columns) was reported damaged
after 6,310 rows since #2413. Shared strings, streamed by `si`, had the
same exposure: a table of about a million plain strings, or fewer
rich-text ones.

The count now returns to its value at the start of a split element when
that element is handed out, so a streamed part of any size passes while a
part that keeps a million elements in its tree is still refused.
Regression test: 5,001 streamed rows of 100 cells are read whole; the same
part kept in a tree is refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-24 17:45:15 -08:00
Dylan M. TaylorandCodebuff 6fac430f55 M2.9: cap xlsx import work and gate hostile datetime field lengths (#2413)
Security-sweep findings from the scan at revision ac8da7d4 (storytold-
gridcraft-agent-findings.json), no longer exploitable on main at 72c1d57d.
Each change caps work or storage per sheet/part so a hostile file cannot buy
unbounded work with ordinary bytes; each has a regression test:

- finding 2 (package.rs): a part rejected by the MAX_PART size gate is
  charged its declared compressed size up front, so a loop of oversized
  parts still ends the allowance instead of re-entering the package.
- finding 4 (xml.rs): elements past the kept-node cap are consumed and
  dropped instead of kept — parse cost stays linear, memory stops growing
  with the same input (a small compressed workbook previously asked for
  gigabytes of declared-sized metadata tree).
- finding 10 (records.rs): .remaining() caches the suffix of a shared
  string's Continue parts, so crossing N Continue records is O(N) instead
  of O(N^2) in records scanned per character.
- finding 9 (xls shared formulas): the SHRFMLA master template is parsed
  once and re-emitted per dependent cell instead of decoded and reprinted
  for every referencing cell.
- findings 3, 6, 8 (sheet_read.rs): hyperlink-cell expansion, conditional-
  format kept range cells and dynamic-array (deleted-spill) checks are
  capped with one import warning each instead of the compact-workbook
  multi-gigabyte allocation or the scanned-per-cell cost.
- finding 31 (core/parse.rs): datetime recognition no longer reparses a
  prefix at every space with a time-shaped tail; an all-digit field length
  is gated before the derivation machinery runs, staying linear in the
  text.

🤖 Generated with Codebuff

Co-authored-by: Codebuff <noreply@codebuff.com>
2026-10-24 05:46:28 -04:00
aiongg d3201e7cea Integration: track t08
# Conflicts:
#	ROADMAP.md
#	crates/engine/src/tests.rs
#	docs/file-format-parity.md
#	docs/localization-parity.md
2026-10-22 03:03:55 +00:00
aiongg 02640d6626 Merge PR #1495: XLSX: preserve carriage returns in saved text
Conflicts:
- crates/xlsx/src/xml.rs: kept both appended tests (surrogate pairs, carriage returns)
- docs/file-format-parity.md: kept main's header and log rows, added this PR's change phrase and row once
2026-10-22 02:34:51 +00:00
midasdf e42ebf9311 XLSX: keep a phoneticPr that has no rPh runs, and document the limits
si_body returned as soon as no valid run was written, so a shared string with a
<phoneticPr> and no <rPh> — a shape §18.4.8 allows — lost its properties on
save even though the reader keeps them. The runs and the properties are now
written independently: valid runs as before, then the valid <phoneticPr> when
the workbook has one; with neither, the <si> is exactly the plain t_el output.
property_only_shared_string_survives_open_and_save covers read, save and
reopen, and the si_body damaged-data test now expects the property-only output.

The write-time checks the reader also runs (sb/eb clamped into the UTF-16
length, empty readings skipped, type/alignment filtered through the shared
PHONETIC_KINDS/PHONETIC_ALIGNMENTS) were already in the head; the tests above
pin them, so no further change there.

Docs: the .xlsx row and the revision history now state that the shared string
table holds each text once, so duplicate <si> items with the same text but
different readings keep only the first reading (reported as a warning on open),
and that a <phoneticPr> without runs is kept. Both parity docs' "Last updated"
headers match their 2026-10-20 revision rows; the earlier rows keep their dates.
2026-10-21 22:35:13 +09:00
midasdf 40803e6aca XLSX: keep Japanese phonetic runs (ふりがな) on open and save
xl/sharedStrings.xml can carry phonetic runs (<rPh sb="0" eb="2"><t>ふりがな</t></rPh>)
and a <phoneticPr> beside a string item. GridCraft read only the text and wrote every
string back as <si><t>…</t></si>, so a plain open + save deleted a Japanese workbook's
furigana.

Both are now read — offsets validated against the base text, runs with bad or missing
offsets, sb >= eb or no reading dropped with a warning, and never used to index — and
kept on the workbook as Workbook::phonetics, one entry per string text. The writer
emits them next to the <si> of the same text. The saved table holds each text once, so
two identical texts cannot keep two different readings: the reader keeps the first and
warns. Strings without phonetic data are written byte for byte as before.

Behaviour reference: ISO/IEC 29500-1 §18.4.6 rPh and §18.4.7 phoneticPr (Microsoft
Learn PhoneticRun/PhoneticProperties). Tests: crates/xlsx/src/tests/phonetic_tests.rs
(round trip, plain string, hostile offsets, shared text, unused data, comments and
inline strings). Docs: file-format-parity.md, localization-parity.md.
2026-10-20 13:39:48 +09:00
derechopiano 9097170b08 XLSX: preserve carriage returns in saved text 2026-10-15 10:33:57 +01:00
MarekSpodBiedry 8939e7817f Fix xlsx: combine escaped UTF-16 surrogate pairs when decoding strings (#1025) 2026-10-13 18:04:09 +02:00
Brandon Thomas 79fce91359 Merge remote-tracking branch 'origin/main' into fix-333
# Conflicts:
#	ROADMAP.md
#	crates/model/src/lib.rs
#	crates/xlsx/src/sheet_read.rs
#	docs/file-format-parity.md
#	docs/gaps.md
2026-10-11 12:56:45 -04:00
vangel76andClaude Opus 5.5 ed62390244 XLSX: keep unmodelled parts on save (#330)
Pass-through of package content GridCraft does not model: on read the raw
parts are kept with their content types and relationship graph per owner
(package root, workbook, sheet); on write they get fresh part names where
needed and new relationship ids, and the r:ids in kept XML are rewritten.

Kept: custom document properties, ribbon customisation, add-in task panes,
custom XML, links to external workbooks with <externalReferences> (never
followed), connections, sheet query tables, printer settings with the
<pageSetup r:id> (#324), form and ActiveX controls, OLE objects and sheet
backgrounds with their VML shapes (merged with GridCraft's note shapes).
Content whose graph reaches parts GridCraft rewrites is dropped with a
warning; sheet content goes with its sheet and is not copied by Copy Sheet.

Fixes #324

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 13:12:56 +02:00
pkibiego 2f078962a0 xlsx: bump quick-xml to 0.41 for the namespace-declaration DoS (RUSTSEC-2026-0195)
The ODS and XLSB readers parse untrusted package metadata and content through
`quick_xml::NsReader`. In 0.38.4 that is RUSTSEC-2026-0195 (CVSS 7.5): the namespace
resolver pushes a binding for every `xmlns` / `xmlns:*` declaration on a start tag, with
no cap on how many there are, and `NsReader::read_event` reaches that path - not only
`read_resolved_event`.

None of our caps bound it. MAX_PART (512 MiB), MAX_TOTAL (1 GiB), MAX_ENTRIES,
MAX_CELLS, MAX_TEXT and MAX_DEPTH bound part size, entry count, cells, text and nesting,
never declarations per tag.

Measured with `gridcraft-cli info` on crafted .ods files, before this change:

     754 B, 0 declarations  ->    8.6 MB, exit 0
     24 KB, 1,000,000       ->   52.6 MB, exit 0
    187 KB, 8,000,000       ->  367.8 MB, exit 0
    770 KB, 33,000,000      ->    1.46 GB, exit 0

About 45 bytes of resident memory per declaration, roughly 1,900x the file size, and no
error at all. At MAX_PART a ~1 MB file reaches ~1.9 GB, and every part is held in memory
while the package is read. Both readers share the path, so a downloaded .ods or .xlsb
was enough. Nothing panicked, so `Session::execute` never saw it.

0.41.0 caps declarations per start tag at 256 (DEFAULT_MAX_DECLARATIONS_PER_ELEMENT,
raisable through NamespaceResolver::set_max_declarations_per_element) and returns
`NamespaceError::TooManyDeclarations` beyond it. The same fixtures after:

     754 B ->    8.6 MB, ok         24 KB ->   19 MB, error
    187 KB ->  101.8 MB, error     770 KB ->    397 MB, error

with "start tag declares more than 256 namespace bindings". The residual peak is the
part itself: MAX_PART still lets one part decompress to 256 MiB, the existing intended
bound.

API changes for 0.41 (0.x bumps may break; all four are in the xlsx crate):
- `resolve_element` and `resolve_attribute` moved off `NsReader` on to the resolver:
  `reader.resolver_mut().resolve_element(..)`. `attr` and `tag` in ods.rs take
  `&mut NsReader` for that.
- `Attribute::unescape_value` is deprecated in favour of `normalized_value`, which also
  applies the attribute-value normalization the XML specification requires.
- `BytesText::xml_content` now takes the document's XML version.

This also clears RUSTSEC-2026-0194, the quadratic duplicate-attribute check. xml.rs
already passed `with_checks(false)`, but ods.rs and xlsb.rs used the checked iterator, so
they were on the quadratic path as well.

Regression test: `ods_tests::ods_namespace_declaration_bomb_is_rejected` asserts that a
tag with thousands of declarations is an error and that a document with a handful still
reads. It fails on 0.38.4 (the tag is accepted) and passes on 0.41.0.

Verified: `cargo xtask ci`; the full xlsx suite (80 tests) plus the malformed-input
tests; and the four measurements above, before and after, with no panics either way.
2026-10-11 03:39:32 +03:00
Brandon ThomasandClaude Opus 5.5 1ba9daa263 M0: foundation — core, numfmt, formula, functions (478), model, calc, xlsx/csv, engine
Clean-room spreadsheet engine: addresses/values/dates, Excel number formats,
formula parser/printer/reference adjustment, worksheet function library,
copy-on-write workbook model, dependency-driven recalculation with dynamic
arrays, XLSX/CSV I/O, and a command engine (cell entry, clipboard, fill,
formatting, sheets, sort/filter, tables, charts, CF, names, view, file).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:18:10 -04:00