mirror of
https://github.com/storytold/gridcraft.git
synced 2026-11-07 02:51:33 +00:00
74d9641e25ce63fc2aefda4ea2591ba7aed48085
345
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brandon Thomas
|
74d9641e25 | Merge PR #3173 | ||
 ljkx
|
fb6dc688bc |
Rewrite rule formulas for the new top-left cell when cells are removed from a rule
Clearing validation from cells (or setting a new rule, or pasting a validation over them) and Clear Rules on selected cells shrink a rule's areas. When that removes the rule's top-left cell, the relative formulas were left written for the old cell, so the cells left checked other cells: clearing A1 off a custom `A1>10` on A1:A9 made A2 check A1. Rewrite the relative references of validation formulas, and of conditional format expressions and comparison values, for the new top-left cell. Absolute references and literal lists stay as they are. |
||
|
ljkx
|
adcb66697c |
Data validation and conditional formats: anchor relative formulas on the top-left cell of all areas (#3152)
Excel writes the relative formulas of a multi-area rule for the top-left cell of the whole sqref (lowest row, lowest column), whatever order the areas are listed in. GridCraft anchored them on the first area listed, so a validation on `H62:H68 H26:H27 H93:H95` with `MOD($H26,$F26)=0` checked row 26 - 36 at H26 and refused valid input. Add `RangeRef::top_left` and use it in validation checks, list sources, pasting a validation, conditional format lookups and the formulas the xlsx writer generates for text, blank and error rules. |
||
 prxnvt
|
7ddcb800ea |
Merge origin/main (6f52cc50) into the Python cells branch
Main gained #471 (equations) and #2729 (command palette shortcuts). Conflicts in ROADMAP.md, docs/architecture.md and docs/file-format-parity.md only: header lines combined, both sides' sections and rows kept. |
||
 PranavandFredApps
|
6f52cc5090 |
Equations: Insert › Equation with a LaTeX editor, Office Math in xlsx and equations in PDF (#471)
* Objects: one selection for charts, pictures, shapes and slicers, with eight resize handles
The sheet tracked the selected object as a bare id in `selected_chart`, so
only charts and text boxes drew a selection frame, only the bottom-right
corner resized, and the kind of the selected object had to be guessed from
the id. The selection is now `SheetApp::selected_object: Option<ObjRef>`,
with `selected_chart()` kept for the Chart Design tab and the Format Chart
pane.
- Model: `ObjKind` (chart, image, shape, slicer) and `ObjRef { kind, id }`;
`Sheet` helpers find, move and remove an object by `ObjRef` and walk every
object's anchor in a fixed order. `next_object_id` saturates instead of
overflowing on an id at `u32::MAX` read from a file.
- Commands: `object.delete`, `object.move` and `object.setAnchorMode` accept
the four kinds through one parameter check, refuse an unknown kind or id
instead of succeeding silently, and clamp hostile offsets. `object.delete`
replies with the deleted object. Arrange and the Selection Pane include
slicers, which stack above the other objects.
- Sheet: the selected object of any kind draws a frame with eight handles
over every object. Each handle resizes only its own edges, which never
cross; the body moves the object, and the first drag frame already applies
the pointer's movement. The pointer shows the handle's resize cursor or a
move cursor, with the diagonals swapped on a right-to-left sheet, where the
frame moves with its object.
- Keys: Escape drops the object selection; Delete and Backspace keep
deleting the selected object (#213). A selection whose object disappears
(undo, redo, delete, sheet switch) is cleared.
- Double-click: `open_object` routes by kind (a text box edits in place, a
chart opens Format Chart).
- Control protocol: `ui.inspect` reports `selectedObject {kind, id}` and
keeps `selectedChart`; `ui.object.select` selects any object and replies
with its rect. The snapshot example takes `--object <id>`.
Tests: engine tests for every kind by kind and id, the Selection Pane and
hostile parameters; UI tests for selecting each kind (a slicer over a chart
takes the click), all eight handles on four kinds, cursors, Delete,
Backspace and Escape, undo and redo, double-click routing, and resizing on a
right-to-left sheet. Existing tests use the new selection API.
* Objects: every kind follows edits; chart series follow cut-moves and sheet deletes
Structural edits and sorts kept three hand-indexed lists (charts, images,
shapes) in step in two places, and slicers were left out of both, so a
slicer stayed put when rows were inserted above it. Chart series were
rewritten for inserted and deleted lines and for sheet renames, but not for
cut-moves or sheet deletes, which left them pointing at the old cells.
- Inserting and deleting rows and columns and sorting walk
`Sheet::object_anchors` and `object_anchors_mut`, one fixed order over
every kind, so slicers now follow their cells by their anchor mode like
the other objects, and a new kind cannot be forgotten in one of the
passes.
- `rewrite_cell_links` is the one place that rewrites the references
floating objects hold (chart series values, categories and names,
and the source block a chart was made from).
`rewrite_all_formulas` calls it, which covers inserting and deleting
lines and cells and cut-moves on one sheet or across sheets; renaming and
deleting a sheet call it too, so a series into a deleted sheet becomes
#REF!, as a formula does.
- Copying a sheet gives its charts, pictures, shapes and slicers fresh ids,
keeping ids unique across the workbook, which object selection and the
`{kind, id}` commands rely on.
Tests: every kind through row and column inserts and deletes, an absolute
shape, a sort and undo; a slicer through inserted rows; chart series
through a cut-move, a sheet rename, a sheet delete and undo; ids after a
sheet copy.
* Objects: bubble sizes follow edits like the other series references
`rewrite_cell_links` rewrote a chart series' values, categories and name
but not its bubble sizes, so inserting or deleting lines, a cut-move or a
sheet rename left a bubble chart's sizes pointing at the old cells. They
are now rewritten with the rest.
Test: bubble sizes through an inserted row, a cut-move and a sheet rename.
* Objects: a resize handle under the headers leaves the press to them
A selected object's handles reach a few points outside it, and a press
was matched against them before any check of where it landed, so a press
on the column or row header beside an object at the sheet's edge resized
the object instead of selecting the column or row. The cursor already
ignored handles outside the cells.
`chartview::selected_handle` finds the selected object's handle under a
point only over the cell area, as `hit` does, and not while objects are
hidden; the press and the cursor both use it.
Test: presses within a handle's reach over the column header and the row
header neither move nor resize the object.
* Objects: each drag of an object is its own undo step
A drag merges its frames into one undo step by dropping a new step whose
label matches the one before it. Its first frame did the same, so a second
drag of an object (a move after a move, or a resize after a move) merged
into the first drag's "Move Object" step, and one undo took back both.
`Drag::Object` records whether the drag has made its step; only the later
frames of the same drag merge into it.
Test: a move and then a resize of a picture take two undos to reverse,
the first restoring the moved picture.
* Objects: the parity and gap docs no longer list chart resizing as missing
gaps.md (§4), target-app-parity.md (the Essentials tier's work, the
Mainstream chart row, the interaction discount and the Essentials chart
task) and chart-parity.md (the element table and the work list) still
described charts as impossible to resize (#190). Charts move and resize
from eight handles like every floating object, so those statements now
say so; the 2026-10-10 revision rows and the 10-10 re-check stay as
history. Scores and the 40-60 h chart estimate are unchanged (resizing
was 1-2 h, within the rounding).
* Objects: one chart evidence paragraph in gaps.md; trendlines are not missing
gaps.md §4 carried two evidence paragraphs, an older one saying PRs #158
and #161 were open and a newer one saying all 2-D kinds survive XLSX round
trip, both repeating the resize, smooth-line, Switch Row/Column and
trendline statements. It is one paragraph with the current status again.
The Mainstream chart row in target-app-parity.md listed trendlines as
missing; they are read, written and drawn (`Series::trendlines`, the XLSX
chart reader and writer, `chart.addElement`), as chart-parity.md records.
The row now says so and keeps "no axis options". No score change.
* Objects: only the selected object's handles resize; a press elsewhere on an object moves it
A primary drag looked for a handle under the press on whatever object it
grabbed, so pressing near the edge or corner of an object that was not
selected started a resize, while the pointer over that spot showed the
Move cursor (handles are only shown, and only offered by the cursor, on
the selected object). The press now takes its handle from
`chartview::selected_handle`, the same lookup the cursor uses; a press on
any other object selects and moves it.
Test: dragging an unselected shape by its bottom-right corner shows the
Move cursor, then selects and moves the shape without resizing it.
* Equations: gridcraft-math lays LaTeX out with RaTeX, behind caps and a panic guard
Equations need a LaTeX layout engine. RaTeX (ratex-parser, ratex-layout,
ratex-types, ratex-font; MIT) turns LaTeX into positioned glyphs, rules and
paths. Before adopting it, a gate compared it with writing an in-house
subset:
- Source scan: about 31,700 lines, 95 non-test panic sites (87 in the
parser, mostly unwraps on arguments already counted and fixed regular
expressions), no unsafe code, no clocks, threads, file or environment
access.
- Fuzzing: 70,000 random token strings and 50,000 mutations of valid
equations with no panic and no input slower than 5 ms. Macro bombs stop
at 1,000 expansions; nesting deeper than 32 is refused, and the deepest
accepted input needs under 256 KB of stack.
- The crates build for wasm32-unknown-unknown.
- Syntax coverage: 56 of 57 probes of everyday notation lay out.
The gate passes, so RaTeX is used, pinned exactly at 0.1.14. The new
gridcraft-math crate (layer 1, registered in the layering table) is the only
code that calls it:
- `layout` turns LaTeX into a `MathList` of plain glyph, rule and path data
in em units, with faces (roman, italic, bold, sans, mono, large operator
sizes), explicit colours and a plain fallback character for glyphs a font
may lack. `check` only parses. Both cap the source at 4 KB and the nesting
at 32 before RaTeX sees it, cap coordinates and item counts in the result,
and run under `catch_unwind`, so a panic becomes a `MathError` with the
offending span when RaTeX reports one.
- `to_omml` writes the common subset as Office Math (ECMA-376 Part 1, 22.1):
runs and font styles, fractions and binomials, radicals, scripts, n-ary
operators with limits, functions and limits, delimiters, matrices,
accents, bars and braces. Anything else is an error, so the caller can
save the equation as text instead.
crates/math/tests/gate.rs re-runs a smaller, deterministic version of the
gate (fuzzing, depth, macro bombs, coverage) on every test run, so a version
bump has to pass it. docs/decisions.md records the evidence and the guard
design. RaTeX positions glyphs with KaTeX's font metrics (MIT); no KaTeX
font files are bundled.
* Equations: Insert › Equation, an equation editor, and equations on the sheet
Equations are shapes (`ShapeKind::Equation`) whose text is their LaTeX
source; they select, open and delete as their own object kind
(`ObjKind::Equation`).
- Model: `Shape::math`, an `EquationCache` holding the layout of the
source (or the reason it has none), keyed by a hash of the source. It is
never saved and is ignored when shapes are compared.
`Workbook::refresh_equations` lays out only the equations whose source
changed, copying only the sheets that hold one. Every new or opened
document (`DocState::new`, `DocState::from_saved`) and every commit
call it, so files, pastes and undo snapshots always arrive with a layout
and painting never lays anything out.
- Commands: `insert.equation {latex, cell?|at?, size?, color?}` sizes the
frame for the requested font size (11 pt by default) and anchors it to
move, but not size, with cells. `equation.edit {id, latex}` keeps the
scale the equation is drawn at and refits the frame, in one undo step.
Missing source, or source that does not lay out, is an error naming the
character position ("equation error at character 5: Undefined control
sequence: \foo"). An equation is drawn to fit its frame, so resizing the
frame scales it.
- Painting reads only the cache: glyphs in system maths font roles
(`theme::MATH`, `MATH_ITALIC`, `MATH_BOLD`: STIX Two, Cambria Math, Latin
Modern, DejaVu or Liberation Serif, then Times; nothing is bundled), large
operators and delimiters stretched to their TeX boxes, rules, and
outlines filled by ear clipping. A glyph no face draws becomes a [?] box;
since egui's `has_glyph` misreports the glyphs of the face that supplies
the replacement glyph, a negative answer is confirmed by comparing atlas
slots. Ink defaults to the sheet's text colour. Source that does not lay
out (from a file) is shown as text in a dashed frame.
- Ribbon: Insert gains a Symbols group with an Equation split button before
Symbol. The main half (access keys N E, previously reserved in
keytips.json) opens the editor; the menu has Insert New Equation... and a
gallery of GridCraft's own examples (`gridcraft_math::GALLERY`), each
drawn as a preview and inserted with one click. The new labels are
translated for every interface language, and the Equation icon is drawn
in code.
- Editor (dialog "equation", also driven by `ui.dialog.set` and
`ui.dialog.confirm`): quick-insert buttons for common constructs, Greek
letters and relations, inserted at the caret with the caret moved into the
first `{}`; the code editor in LaTeX mode with the error's span underlined
(`code_editor_marked` and `code_editor::underline`); the error with its
character position, or a live preview on a sheet-coloured card; OK is
disabled while the source has an error, and Cmd/Ctrl+Enter runs
`insert.equation` at the active cell or `equation.edit`. Double-clicking an
equation opens the editor on its source.
- xlsx writes an equation as a text box holding its source for now.
The keytip test that used `insert.equation` as its stand-in for a pending
command now uses `insert.object`. Docs: architecture (the math crate and the
cache), UI, gaps and target-app parity, ROADMAP; the parity checklist is
regenerated.
Tests: engine tests for insert sizing, size and colour, undo, hostile and
bad source with positions, edit scale and refit, no-op edits, refusals,
protected sheets, the cache rebuilt on load and not saved, a bad source
from a file, and equations in the object tests; UI tests for opening the
editor from the ribbon and with N E, typing an equation in, the error
position blocking OK, quick-insert at the caret, double-click editing as
one undo step, the gallery, an equation in the selection and resize tests,
ear clipping, curve flattening and underline splitting.
* Equations: Office Math in xlsx, and the Symbol font in PDF
xlsx: an equation is written as mc:AlternateContent. The a14 choice is a
text box holding the equation as Office Math (a14:m with m:oMathPara, from
gridcraft_math::to_omml at the size the equation is drawn at, in its ink
colour); the fallback is the same text box with the LaTeX as plain text, so
other readers still show the source. Both carry
descr="gridcraft:latex:<source>". Source outside the Office Math subset is
written as the text box alone.
On reading, the descr source wins, exactly (quotes and newlines included).
Without it, as in files from other writers, the new xlsx::omml reader
converts the common subset back to LaTeX: runs (upright, text,
double-struck, script, Fraktur, sans and bold styles, operator names),
fractions, radicals, scripts, n-ary operators with limits, delimiters with
separators, matrices, equation arrays, accents, bars, group characters,
lower and upper limits, functions, border boxes and phantoms. Anything else,
or a result that does not parse, stays a text box. The reader is
depth-bounded. The drawing reader now takes an mc:Choice that holds Office
Math, not only chart frames. xlsx may now depend on gridcraft-math
(FORMAT_ALLOWED), a pure layer-1 crate.
PDF: crates/pdf gains Font::Symbol (the thirteenth standard font, written
without WinAnsiEncoding) with its Unicode-to-Symbol encoding table and the
advance widths of Adobe's Core 14 Symbol.afm, cited in symbol.rs as
metrics.rs cites the other faces; Font::can_encode tells callers what a face
can show. Symbol text is written as one run in Symbol's own encoding, not
split off to the CID font that the text faces use for characters WinAnsi
lacks. Printing and PDF export draw an equation's cached layout: letters
and digits in Times (italic for variables) or Helvetica and Courier for sans
and monospace styles, Greek letters and operators in Symbol, large operators
and delimiters stretched to their TeX boxes, the radical fitted to meet its
vinculum, rules as filled rectangles and outlines as filled polygons. A
glyph outside WinAnsi and Symbol (after its plain fallback, R for a
double-struck R) prints as a [?] box.
Docs: file-format parity, gaps, target-app parity and ROADMAP.
Tests: xlsx round trips (choice and fallback markup, descr, colour, anchor
mode, an equation outside the subset, hostile sources, a text box left
alone), Office Math without a source read back to the same glyphs, the
reader's subset, unsupported constructs and the depth bound, and gallery
write-then-read; Symbol encoding and widths for every mapped character; a
PDF with Symbol and Times-Italic, the summation and alpha codes, and a [?]
box.
* Equations: the error underline snaps to char boundaries, so a stale mark cannot crash the editor
The equation editor computes the error's byte range from the source as it
was before this frame's edit, and `code_editor::underline` only clamped the
range to the text's length. Typing a multibyte character in front of the
error (`a\foo` with its error at 1..5, then `é` at the start) put a section
boundary inside the `é`, and egui's layout panicked slicing the text there,
in the paint path, where no guard catches it.
`underline` now moves the mark's start down and its end up to the nearest
char boundaries, so any byte range is safe.
Test: a_stale_mark_inside_a_multibyte_char_snaps_to_char_boundaries marks
every range over texts with two-, three- and four-byte characters, checks
that every section is on char boundaries and covers the text, and lays each
job out; it fails without the fix.
* Equations: the editor focuses its source on every opening, however the last one closed
The editor kept a "focused once" flag in egui memory and removed it only
on OK, Cancel and Cmd/Ctrl+Enter. Escape and the window's close button
also close the dialog, and left the flag set, so the next Insert ›
Equation or Edit Equation opened without focus on the LaTeX source and
typing went nowhere.
The flag is now the dialog's own `focus_initialized` (made visible to the
crate), as Command Search uses it: every opening builds a new dialog with
it unset, whatever closed the previous one.
Test: reopening_after_escape_focuses_the_source_again opens the editor
twice, types into it and closes it with Escape each time; it fails
without the fix (the second round's typing is lost).
* Equations: Office Math the reader cannot convert keeps the writer's fallback
The drawing reader took an `mc:Choice` whenever its text box held Office
Math, before knowing whether that Office Math converts to LaTeX. Office
Math outside the reader's subset, without GridCraft's source in `descr`,
then became a text box with no text, and the `mc:Fallback` (the picture or
text the writer saved for readers like this one) was never read.
The choice is now taken only when `omml::to_latex` converts its Office
Math; otherwise the fallback is read. GridCraft's own files are
unaffected: their fallback carries the same source in `descr`.
Test: office_math_outside_the_subset_reads_the_fallback replaces a saved
equation's Office Math with a pre-subscript (outside the subset), removes
the source and gives the fallback its own text; it fails without the fix
(the shape is read with empty text).
* Equations: hostile LaTeX through the guarded entry points and the xlsx round trip
The gate fuzzes RaTeX itself. These tests fuzz what GridCraft's callers
reach:
- the_guarded_entry_points_survive_hostile_input runs `layout`, `check`
and `to_omml` over random token soup (any Unicode scalar included),
inputs at and past the nesting and length caps, huge rules and a macro
bomb: none panics, every layout stays within `MAX_EM` and `MAX_ITEMS`,
every error span is on char boundaries inside the source, and what lays
out also checks.
- hostile_sources_round_trip_exactly saves 200 sources built from XML and
LaTeX specials (quotes, `<`, `&`, `]]>`, comments, CR, LF, tab, emoji,
CJK, unbalanced braces) to xlsx and reads each back as the same equation
source.
Both pass on the current code; they guard the caps and the writer's
escaping against regressions.
* Equations: opening a file lays out a bounded amount of equation source
`Workbook::refresh_equations` laid out every stale equation on load and
on every commit. One 4 KB equation can take about a tenth of a second to
lay out, so a hostile or simply large file with thousands of equations
held the app for minutes before it showed anything.
One call now lays out at most `MAX_EQUATION_LAYOUT_BYTES` (128 KB, about
2,500 typical equations) of source. Equations past it get an error
(`EquationCache::refuse`, without running the layout engine), so they show
their source in the dashed frame and later commits do not retry them.
`equation.edit` with an unchanged source now lays out an equation that has
no layout, so OK in the editor brings one back. Architecture docs note the
budget.
Test: a_file_with_too_many_equations_lays_out_a_bounded_amount opens a
workbook whose equations exceed the budget, checks that the laid-out
source stays within it, that the rest carry the budget's error and are not
retried by a later edit, and that `equation.edit` lays one out; it fails
without the budget (every equation is laid out).
* Equations: the layout cache answers only for the source it was built from
`EquationCache` matched an entry by a 64-bit hash of the source
(`DefaultHasher`, fixed keys). A crafted pair of sources with the same
hash, for example one in a file and one typed by the user, would make the
cache return one source's drawing for the other.
An entry now keeps its source and compares it in full on every lookup;
the hash and length are only a quick first check. A source over the
4 KB length cap is not copied into the entry: it never lays out, so its
entry is always an error, and it is matched by length and hash, and only
by another source over the cap. Kept sources are shared (`Arc<str>`), so
copies of a sheet and undo snapshots do not duplicate them.
Tests: a_hash_collision_never_returns_another_sources_layout forges a
collision by giving an entry for `x^2` the hash of `y^3` (same length) and
checks that `y^3` misses, is laid out afresh, and that a refusal does not
answer for a colliding source either; it fails without the fix (`y^3` gets
the layout of `x^2`). sources_over_the_cap_are_not_copied_and_never_lay_out
checks the cap path.
* Equations: a text box with Office Math beside text or other maths stays a text box
The drawing reader made any shape whose text held Office Math into one
equation, taken from the first `m:oMath`. A text box from another writer
with text beside the maths ("Solve: " and an inline equation), or with two
equations, lost the text, the other equation and the shape's formatting.
The `mc:Choice` filter used the same check, so the writer's fallback was
skipped as well.
`omml::equation_source` now converts a text body only when it holds one
equation and nothing else: no text in its runs or fields and no second
`m:oMath`/`m:oMathPara`. The Choice filter and `read_shape` both use it, so
a mixed box reads the fallback (inside alternate content) or stays a text
box with its text.
Test: office_math_beside_text_or_other_math_is_not_an_equation covers a
text run before the maths in the a14 choice (the fallback's text is read),
the same box without alternate content (a text box that keeps "Solve:"),
and two equations in one box (not an equation). It fails without the fix
(the first case reads an equation); the two-equation case fails when only
the second-equation check is removed.
* Equations: a new object id is a free one even when a file used the largest id
`Workbook::next_object_id` returned one more than the largest id in use,
saturating at `u32::MAX`. A workbook read from a JSON file can carry an
object with that id; every later insert (an equation, chart, picture,
shape, table or slicer) then got `u32::MAX` again, and so did each object
of a copied sheet, whose ids also came from a saturating counter. Lookups
and selection take the first object with an id, and `object.delete`
removes every one.
`Workbook::free_object_ids(count)` now returns distinct unused ids: those
above the largest in use while they last, then the lowest free ones from
1. `next_object_id` takes the first of them, and the sheet copy takes one
per copied object (and refuses if the id space were ever exhausted). The
model test that recorded the saturation now checks the free ids instead.
Test: ids_stay_unique_when_a_file_used_the_largest_id opens a workbook
whose equation has id u32::MAX, inserts two equations and copies the
sheet; every id is distinct. It fails without the fix (both new equations
get 4294967295). object_ids_never_overflow_or_repeat covers the gaps and
the order.
* Equations: the control protocol and UI parity list equation among the object kinds
`ui.inspect` reports a selected equation as `selectedObject: {kind:
"equation", id}`, and `object.delete`, `object.move`,
`object.setAnchorMode` and the `arrange.*` commands take `equation` as a
`kind`, but docs/control-protocol.md listed only chart, image, shape and
slicer. It now lists equation too, as does the floating-objects row of
docs/ui-parity.md.
* Docs: ui.object.select says what it replies when it clears the selection
* Objects: a test pins that both anchor accessors visit the same objects in the same order
Sorting and row or column edits read every anchor through
`object_anchors`, compute the new ones and write them back through
`object_anchors_mut` in step. Nothing checked that the two accessors
visit the same objects in the same order; a kind of floating object
reaching only one of them would have paired objects with other objects'
anchors silently.
Test: two objects of every kind are inserted in a mixed order, each
anchor is marked with its position through the mutable accessor, the
marks are read back through the shared one in the same sequence, the
count matches the per-kind collections, and the mark is found on the
object `object_anchor` names by kind and id. The test's constructor
matches on `ObjKind` exhaustively, so a new kind does not compile until
the test covers it.
* Objects: sorting keeps the size of a "move and size" picture whose lines swap places
A picture anchored to move and size with cells and spanning two rows
(or columns) was re-fitted between its top-left and bottom-right lines
after a sort. When the sort put the bottom line above the top one, the
fitted extent was negative and the picture collapsed to a zero height
or width. It now keeps its old size on that axis and travels with its
top-left line, like a "move only" picture.
Tests: for a sort by rows and one by columns, "move and size" and
"move only" pictures inside one line travel with it (the first
re-fitted to the same line, the second at its own size), an absolute
picture stays while the cells move under it, a two-line picture whose
lines swap keeps its size, and undo restores every anchor.
* Equations: Office Math whose LaTeX passes the source cap reads the writer's fallback
The OMML reader (crates/xlsx/src/omml.rs) bounded nesting (`MAX_DEPTH`
64, `find_math` depth 8) but not the length of the LaTeX it produced: a
drawing with 100,000 sibling runs, or one run of megabytes of text, was
converted in full and only then refused by `gridcraft_math::check`,
which stops at `MAX_SOURCE_BYTES` (4,096). A hostile file paid for the
whole string, and each backslash in a `\text` run grew to
`\textbackslash ` first.
`convert` (the conversion before the final check) now stops with `None`
as soon as the LaTeX produced passes `MAX_SOURCE_BYTES`: after every
node in a sequence, and after every character of a run's text, in the
escaped `\text{…}` form as well as the plain one. The check reads the
output, not the run's byte length, because a character may become a
longer macro (`\alpha `) or a shorter sign (a minus sign becomes `-`).
The caller then keeps the writer's text fallback, as for any equation
outside the subset. An equation exactly at the cap still reads back.
Regression test: `omml::tests::latex_past_the_source_cap_stops_converting_and_falls_back`.
Without the checks it fails with `left: Some("xxx…")` (a 100,000-byte
string), `right: None`. The test reads the clock, which clippy
disallows for the sake of wasm; tests never run there, and the lint is
allowed on that function.
* Equations: the hostile-source test covers equation.edit as well as insert.equation
`insert.equation` and `equation.edit` both lay a source out through
`cmd::equation::lay_out`, so both refuse a source over 4,096 bytes or
nested deeper than 32 levels, but only the insert side was asserted.
`insert_refuses_missing_bad_or_hostile_source` now edits an equation
with the same over-long and over-deep sources, expects the same errors,
and checks that the equation's source is unchanged afterwards.
* Equations: ATTRIBUTION.md records the Core 14 AFM numbers behind the PDF Symbol font
PDF export draws an equation's Greek letters and operators with the
standard Symbol font and its letters with the Times faces; no font file
is embedded (the base-14 fonts are the reader's own, and text outside
WinAnsiEncoding uses a predefined, non-embedded Adobe-GB1 CID font). What
the repository holds are numbers: Symbol's character codes and advance
widths in crates/pdf/src/symbol.rs, and the Helvetica, Times and Courier
widths in crates/pdf/src/metrics.rs, typed in from Adobe's Core 14 AFM
metrics, which Adobe distributes for use, copying and distribution for
any purpose (the notice is quoted in metrics.rs). ATTRIBUTION.md now
carries a row for those numbers, their source and that notice, so the
licence is recorded where every other third-party input is.
* Equations: the gate times pathological sources at the cap, chemistry included
`ratex-parser` uses `fancy-regex`, a backtracking engine, for its port of
mhchem: anchored patterns with lookahead tokenise the argument of `\ce`
and `\pu`, so user LaTeX does reach them. The engine stops a match after
its default 1,000,000 backtracking steps and the match counts as none,
and `gridcraft-math` caps the source at 4,096 bytes before the parser
sees it, but nothing measured how long a source built to make those
patterns (or the hand-written parser) work hardest could take.
`pathological_source_at_the_cap_finishes_in_bounded_time` in
crates/math/tests/gate.rs lays out, checks and converts 38 sources of
exactly `MAX_SOURCE_BYTES`: groups, delimiters, fractions, roots,
scripts, accents, `\text` and matrices nested to `MAX_DEPTH`; runs of
`\\`, `\{`, `{}`, primes, thin spaces and alignment tabs; a `(a+)+b`
shape; wide and tall environments; macro definitions, colours and
combining marks; and chemistry (`\ce` with digit runs, decimals,
parentheses, arrows, scripts, dollars, formulas, nested `\ce`, charges
and fractions; `\pu` with charges, exponents and unit slashes). Each
must finish inside 10 s through the guarded entry points; the slowest,
`\ce{+-+-…}`, takes 0.8 s in a debug build on an Apple M series
machine, so catastrophic backtracking would fail the test without a
slow CI runner failing it. The test reads the clock, which clippy
disallows for the sake of wasm; tests never run there, and the lint is
allowed on that function. docs/decisions.md records the finding in the
RaTeX gate.
* Objects: a copied sheet's objects get free ids even when a file used the largest id
`Workbook::next_object_id` returned one more than the largest id in use,
saturating at `u32::MAX`. A workbook read from a file can carry an object
with that id; every later insert (a chart, picture, shape, table or
slicer) then got `u32::MAX` again, and so did each object of a copied
sheet, whose ids also came from a saturating counter. Lookups and
selection take the first object with an id, and `object.delete` removes
every one.
`Workbook::free_object_ids(count)` now returns distinct unused ids: those
above the largest in use while they last, then the lowest free ones from
1. `next_object_id` takes the first of them, and the sheet copy takes one
per copied object (and refuses if the id space were ever exhausted). The
model test that recorded the saturation now checks the free ids instead,
and a new one covers a file that used the largest ids: the free ids come
from below, 100,000 of them distinct and in order.
Test: object_ids_stay_unique_when_a_file_used_the_largest_id gives a
shape id u32::MAX, inserts two shapes and copies the sheet; every shape id
is distinct across both sheets. It fails without the fix (both new shapes
get 4294967295).
* Equations: PDF prints an equation without a layout one source line at a time
When an equation has no layout (its source does not parse, or the file
was past the open-time layout budget), PDF export printed the source in
one `pg.text` call. A newline is not a WinAnsi character, so each one
became a literal `?`, and a multi-line source ran on in one line. The
xlsx plain-text fallback already writes one paragraph per line.
`fallback_lines` in crates/engine/src/cmd/print/objects.rs splits the
source at newlines and yields the lines the frame holds at 12 pt per
line (at least the first), never more than `MAX_FALLBACK_LINES` (500);
the fallback arm prints each on its own baseline.
Regression test:
`cmd::print::tests::an_equation_without_a_layout_prints_its_source_one_line_per_line`.
Without the change it fails at "the first line is its own text run"
(the PDF holds `(frac{?second line) Tj`).
* Equations: Office Math runs drop the characters XML cannot carry
`esc` in crates/math/src/omml.rs dropped C0 controls other than tab and
newline but kept U+FFFE, U+FFFF and the carriage return as they were.
The xlsx writer embeds the OMML in the drawing part as it is, so a
source holding one of them produced XML no parser accepts, where the
rest of the drawing part (escaped through the xlsx crate's `esc`, which
filters through `is_xml_char`) would have dropped it.
`esc` now keeps exactly the XML 1.0 character set the xlsx escapes keep
(tab, newline, carriage return, U+0020 to U+D7FF, U+E000 to U+FFFD and
the supplementary planes), writes the carriage return as ` ` and
drops the rest. The predicate is a copy of the xlsx one: the math crate
sits below the xlsx crate and cannot import it.
Regression test: `omml::tests::characters_xml_cannot_carry_are_dropped_from_runs`.
Without the change it fails with `left: "a\u{fffe}b\u{ffff}cde\u{7f}f"`,
`right: "abcde\u{7f}f"`.
* Equations: the matrix writer writes every row RaTeX gives it
The OMML matrix writer skipped a row whose only cell was an empty
`OrdGroup` whenever the matrix had more than one row, meant to drop the
empty last row a trailing `\\` leaves. Read against RaTeX 0.1.14 the
branch was unreachable: the parser removes that trailing empty row
itself (its `environments.rs`, "Check for trailing empty row and remove
it"), and every cell it does give is a `Styling` node around the group,
never a bare `OrdGroup`. A blank row in the middle, written as `\\ \\`
or as `{}`, was therefore already kept, but the code said otherwise and
would have dropped such a row had the parser's shape matched.
The skip is gone; the writer writes every row it is given. The new
tests pin the behaviour so that a RaTeX release that stopped dropping
the trailing row would show up: `omml::tests::only_a_trailing_empty_row_is_dropped`
(two rows for `a \\ b \\`, three for `x \\ \\ y` and for `a \\ {} \\ b`,
one for a lone `\\`) and, through the xlsx reader,
`omml::tests::a_blank_middle_row_comes_back_and_a_trailing_one_does_not`.
* Equations: to_omml takes the ink colour instead of the xlsx writer patching its output
The xlsx writer coloured the Office Math runs by replacing the exact
text `<a:latin typeface="Cambria Math"/></a:rPr>` in `to_omml`'s output
with the fill in front of it. Had that text changed, `replace` would
have done nothing, the Office Math choice would have lost its colour,
and nothing would have reported it; the round-trip test could not have
caught it either, because the plain-text fallback carried the same
colour and the reader would have returned it.
`gridcraft_math::to_omml` now takes the ink as `Option<[u8; 3]>` and
writes `<a:solidFill><a:srgbClr …/></a:solidFill>` into every run's
properties itself; the xlsx writer passes the resolved shape colour and
the string patch is gone. Tests: `omml::tests::runs_carry_the_ink_colour_when_one_is_given`
(three coloured runs for `x+1`, none without an ink), and
`equations_write_office_math_with_a_text_fallback_and_read_back` now
asserts the colour inside the `<a14:m>` paragraph. Without the fill in
the writer both fail (`left: 0, right: 3`; "the Office Math runs carry
the ink colour").
* Equations: what shows a source as text shows its first 4,096 bytes
The PDF fallback for an equation without a layout capped the number of
lines it printed but not their length: `fallback_lines` scanned the
whole source and handed a complete line to `Page::text`, which formats
it before the page-content limit can refuse it. A source read from a
file is not bounded by `MAX_SOURCE_BYTES` (the editor and the commands
refuse a longer one, but a drawing part may be 512 MiB), so a hostile
file put megabytes into one text run. The sheet painter laid the whole
source out every frame for the same reason.
`cmd::equation::source_excerpt` returns the first `MAX_SOURCE_BYTES`
bytes of a source, cut at a char boundary; a longer source never lays
out, so that is all a reader of the text can use. `fallback_lines`
splits the excerpt, and the sheet painter's dashed frame shows it.
The stored source is left as the file wrote it: the xlsx reader keeps a
`descr` source whole, as it keeps every text box's text, so a file is
not changed by opening it, and `check_caps` refuses an over-long source
by its length before reading it, so opening costs nothing more.
Regression tests: `cmd::print::tests::an_equation_source_of_a_megabyte_prints_a_bounded_excerpt`
(a 1 MiB single-line source: the longest text run in the PDF is 4,096
bytes; without the excerpt it is 1,048,576) and
`tests::equation::the_source_excerpt_is_at_most_the_cap_and_whole_chars`.
* Equations: the layout cache answers for a source over the cap without reading it
`EquationCache::get` matched a source by length, then by a hash of the
whole source (`DefaultHasher`), then in full. For a source over
`MAX_SOURCE_BYTES` the entry is always the same error, yet every lookup
still hashed it: the sheet painter asks the cache every frame and the
PDF export every draw, and a source read from a file can be megabytes,
so a hostile file was rescanned on every repaint.
`get` now answers for a source over the cap at once, by its length,
with `gridcraft_math::SOURCE_TOO_LONG` (one shared value, the same error
`check_caps` returns), whatever the entry holds; `holds` and `store`
neither hash nor copy such a source. Within the cap the lookup is
unchanged. Both callers already route anything but `Some(Ok(_))` to the
4,096-byte excerpt, so the painter and the PDF fallback do no work on
the rest of the source. `Workbook::refresh_equations` no longer carries
a branch for over-cap sources: they are never stale, so it never saw
them. Nothing else reads a source per frame: `source_excerpt` slices at
most 4,096 bytes and `check_caps` tests the length first.
Tests: `equation_cache_tests::a_source_of_a_megabyte_is_looked_up_without_being_read`
(a 1 MiB source looked up 1,000 times: no hash, counted by a test-only
counter in `source_key`, and under 0.5 s; on the previous cache it fails
with `left: 1000, right: 0` after 0.20 s) and
`sources_over_the_cap_are_not_copied_hashed_or_laid_out`.
* Equations: the source cap's message says bytes, which is what is counted
check_caps compares the UTF-8 byte length with MAX_SOURCE_BYTES, so 2,049 two-byte characters are refused although the message said 4,096 characters were allowed. The message now names the unit the check uses.
* Equations: ATTRIBUTION.md records Unicode's terms for the Symbol encoding table
The Unicode values of Symbol's character codes in crates/pdf/src/symbol.rs
come from `symbol.txt` ("Adobe Symbol Encoding to Unicode", table version
1.0, 2011-07-12, at unicode.org/Public/MAPPINGS/VENDORS/ADOBE/), which
carries Unicode, Inc.'s own copyright and terms, not Adobe's Core 14
metrics notice: Unicode grants the right to freely use the information
in the file in products supporting the Unicode Standard, and to copy the
file as long as its notice remains attached, as-is and without
warranties. ATTRIBUTION.md now has a row for that table with those
terms, separate from Adobe's row, which keeps to the AFM character codes
and widths; the module comment of symbol.rs credits both.
* Equations: a literal space in text mode reaches Office Math
RaTeX gives the space in `\text{if } x` as a spacing node whose text is
" ", and the OMML writer's spacing match mapped that to "", which `run`
omits, so the saved equation read `ifx`. The match now maps " " to a
space, and a run that starts or ends with whitespace is written with
`xml:space="preserve"` on its `m:t`, so Word keeps it; the reader never
trimmed, so it reads the run back as `\text{ }`.
Tests: `omml::tests::a_literal_space_in_text_mode_is_kept` (the space is
its own marked run in `\text{a b}` and `\text{if } x`; runs without
surrounding whitespace are not marked) and, through the xlsx reader,
`omml::tests::a_space_in_text_mode_survives_the_round_trip` (the same
number of space glyphs before and after). Without the mapping both fail;
the round trip came back as `\text{i}\text{f}x`.
* Equations: the PDF row says what the writer does with fonts and scripts
docs/file-format-parity.md said PDF export used "only the 14 standard
PDF fonts" and that "non-Latin text does not export". That wording
predates this branch, but the branch added the equation note to the same
row, and both halves were wrong against crates/pdf: the writer has 13
standard-font entries (the Helvetica, Times and Courier faces and
Symbol), none embedded, and sets characters outside WinAnsiEncoding in a
predefined, non-embedded CID font (STSong-Light, Adobe-GB1,
`UniGB-UTF16-H`), so Simplified Chinese displays where the reader
supplies that font, while scripts outside Adobe-GB1 export as code
points and show as the reader's missing-glyph box. The row and the
equation revision row now say so, with Symbol for the Greek letters and
operators it has, the Times faces for letters and digits, and `[?]` for
the rest. "Would pass the cap" reads "would exceed the cap" in the
revision row and the matching code comments.
* Equations: the gate checks nesting that macro expansion builds
`check_caps` counts the braces, `\left`s and `\begin`s of the source
text, so nesting that macros build reaches RaTeX, whose recursion guard
is the one that must hold. `nesting_built_by_macros_is_refused_by_the_parser`
builds `\left(`/`\right)` pairs and `\frac` levels through `\def`: 32
levels lay out, 64 are refused with "Recursion limit exceeded", inside
the 1,000-expansion limit, and the wrapper refuses the 64-level source
as well.
* Equations: the description carries a marker of the Office Math, so an edit made in Excel reads back
The xlsx reader took GridCraft's `gridcraft:latex:` description whenever
a shape had one. Excel edits an equation's Office Math and leaves the
description alone, so a file edited there came back as the old equation,
and the next save overwrote the edit.
The writer now puts a marker of the text body it writes into the
description, `gridcraft:latex:<marker>:<source>`: 16 hex digits of an
FNV-1a 64-bit hash (`crate::omml::body_marker`) over what carries the
equation's meaning, the OMML element names in document order and the
text of the runs. It leaves out what a re-serialisation may change
without changing the equation: property elements (`*Pr`), extension
lists, the DrawingML containers, attributes, namespace prefixes and
whitespace between elements; adjacent runs are joined, so a writer that
splits or merges runs changes nothing, while a text change, an added run
or a structural change does. The hash is computed over the parsed body,
the same way on both sides, and FNV is fixed, so the marker survives
Rust and library versions.
The reader takes the source only when the shape holds Office Math and
the marker matches the body now in the file; otherwise it converts the
Office Math with `equation_source` and drops the stale description,
falling back as before when the maths is outside the subset. A body
without Office Math (the plain-text fallback of an equation outside the
subset) keeps the source as it did, and every branch stays
`ShapeKind::Equation`. A description without a well-formed marker is not
GridCraft's source.
Tests: `office_math_edited_elsewhere_reads_back_as_edited` (an `m:t`
changed in the drawing part reads back converted, and a save writes the
edited equation with the old source gone; with the marker ignored it
fails with `left: "x^2", right: "{y}^{2}"`),
`a_marker_over_garbage_office_math_falls_back_without_panicking`,
`omml::tests::the_body_marker_follows_the_equation_not_its_serialisation`,
and the existing round trips, whose exact sources still come back.
* Equations: the PDF fallback text is clipped to the equation frame
The source lines printed for an equation without a layout were drawn at
full width under the block's clip alone, so a long line ran past the
frame over neighbouring content. The lines are now drawn inside a clip
of the frame, saved and restored around the loop, which nests in the
block's clip.
Test: `cmd::print::tests::the_fallback_text_is_clipped_to_the_equation_frame`
(a 40 by 100 px frame gives a `30 75 re W n` clip before the text run,
restored after it); without the clip it fails at "a clip the size of the
frame".
* Equations: the body marker hashes the properties the converter reads
`body_marker` skipped every `*Pr` element, yet `to_latex` reads several
property values: a run's `sty`, `scr` and `nor`, a fraction's `type`, a
radical's `degHide`, an operator's `chr`, `limLoc`, `subHide` and
`supHide`, a delimiter's `begChr`, `sepChr` and `endChr`, an accent's
`chr`, a bar's `pos` and a group character's `chr` and `pos`. An edit in
Excel to one of them (upright to bold, parentheses to brackets, a
hidden limit) left the marker unchanged, so the stale source won.
The properties the converter reads are one table, `PROPERTIES`, that
both sides use: `val` and `on` read a property only when the table lists
it (a read of an unlisted property is absent, which the conversion tests
would show), and `mark` hashes, for each property element, exactly the
listed properties present as `name=val` in the table's order, closing
the run text gathered so far, so the same property moving to another
run is a change too. `ctrlPr`, extension lists and properties the
converter never reads (DrawingML size, colour and fonts) stay out, so
Excel's re-serialisation still leaves the marker alone.
`the_body_marker_follows_the_equation_not_its_serialisation` now checks
that a changed `sty`, a style moved between runs, an accent or delimiter
character, and a hidden operator limit change the marker, while an
added `ctrlPr`, an extension list, DrawingML run properties and an
unread `rFonts` do not. Without the property hashing it fails at the
`sty` case with two equal markers.
* Equations: the clip test checks the graphics state at a later object
The test for the fallback's frame clip counted `q` lines in two ranges
that always added up, and looked for any `Q` after the text, which the
block's own restore supplies, so a missing `pg.restore()` in the
fallback passed it. It now puts a text box below the equation in the
same block and compares the net `q`/`Q` depth at three points: the
text is drawn one level inside the clip's state, and the later object
at the level before it. With the fallback's restore removed it fails
with `left: 2, right: 1` at the later object.
* Equations: the body marker hashes a styled run as a unit, as the converter converts it
The marker joined the text of adjacent runs so that Excel's splitting
and merging of runs would not change it. The converter, though,
converts each run on its own and recognises function names per run: an
upright `sin` becomes `\sin `, while an upright `s` beside `in`, styled
or not, becomes `\mathrm{s}in` or `\mathrm{s}\mathrm{in}`. An edit that
split a styled run therefore kept the marker equal and the stale `\sin `
source was reused.
A run with a property the converter reads is now hashed as a unit: the
property already closed the text gathered before it, and the run's end
closes its own text as well, so a styled run split in two, or a styled
run merged with a plain neighbour, changes the marker. Runs without
read properties convert character by character and stay joined, so
splitting or merging them still changes nothing. This mirrors the
converter exactly, including adjacent runs with identical properties,
which it converts separately.
`the_body_marker_follows_the_equation_not_its_serialisation` now checks
that upright `sin` differs from upright `s` plus plain `in` and from
upright `s` plus upright `in`, that `sin` + `x` differs from `s` +
`inx`, and that plain `ab` equals plain `a` + `b` beside a styled run or
alone. Without the after-run flush it fails at the first new case with
two equal markers.
* Equations: a brace keeps the script on its other side in Office Math
The writer put the label of `\underbrace{x}_{n}` in `m:limLow` and
that of `\overbrace{x}^{n}` in `m:limUpp`, then returned, so a script
on the brace's other side was dropped: `\underbrace{x}_{n}^{2}` lost
the exponent in the Office Math. The source in `descr` hid this in a
GridCraft-to-GridCraft round trip; an application reading the Office
Math saw an incomplete equation.
The other script now wraps the whole: `m:sSup` around `m:limLow` for
an underbrace with an exponent, `m:sSub` around `m:limUpp` for an
overbrace with a subscript. A brace with only the other script
(`\underbrace{x}^{2}`) gets the script around the bare `m:groupChr`,
with no empty `m:lim`; a brace with its label alone, or with neither
script, is written as before. Each wrapper is a level of the depth
accounting.
The reader converted a limit over a brace into `\underset`/`\overset`
around `\underbrace`/`\overbrace`. It now reads a limit whose base is a
brace alone as the brace with the label as its own script, the
writer's inverse, so `\underbrace{x}_{n}^{2}` reads back as
`{\underbrace{x}_{n}}^{2}`; a limit over any other base is still
`\underset` or `\overset`. The body marker hashes element names and
the properties the converter reads, which this does not change: the
stored source is still taken while the marker matches, and an
equation edited elsewhere still reads back as edited.
Tests: `an_underbrace_with_both_scripts_keeps_its_exponent` and
`an_underbrace_without_a_label_keeps_its_superscript` in the math
crate check the exact Office Math for both braces and for nested ones;
`a_labelled_brace_reads_back_with_a_script_on_its_other_side` checks
the reader on hand-written Office Math and the LaTeX round trip (same
glyphs); `a_brace_with_scripts_on_both_sides_round_trips` saves an
xlsx, reads it with the source (taken as written) and without it
(converted, exponent kept). Without the fix the first two fail on the
missing `m:sSup`, the third reads `\underset{n}{\underbrace{x}}`, and
the fourth finds no `m:sSup` in the drawing.
* Objects: main's chart data table compiles with the object selection and the model's object test
Two leftovers of the merge of origin/main (
|
||
|
prxnvt
|
e8f2becf4a |
Engine: a deleted reference drops a Python cell's kept result; spills read past the scan budget stay values
A structural edit that deleted the data an xl() reference names rewrote the reference to #REF! but kept the cell's cached result, so the cell went on showing a result for data that is gone. rewrite_python_refs now makes such a cell new code (#BLOCKED!, no kept result), and the recalculation removes its spill. A rename or a move, after which the reference names the same data, still keeps the result. When the reader's range-scan budget ran out part way through a Python spill, the rest of its cells were read as plain cells, and the result rebuilt on open was then blocked by them (#SPILL!). On such a sheet every Python spill now keeps its values as plain values, with no live spill. Tests: deleting_the_data_python_code_reads_drops_its_kept_result and python_spills_read_past_the_scan_budget_keep_their_values_as_plain_values; both fail without the fix. |
||
|
prxnvt
|
ad6aca08bf |
XLSX: a Python spill is recorded at its anchor, and one too large keeps its values
A Python cell's spill range was recorded only when the file held a record for one of its other cells, so a result whose other cells are blank, which a file may leave out, opened with no spill and no rebuilt result. The range is now recorded when the anchor is read, if a result can fill it. A Python range larger than a result may be (more than 1,000,000 cells, but within what the reader tracks) emptied the cells it covered, although it is no spill and file-format-parity.md says such cells keep the values the file shows. They are now kept as plain values; a formula's spilled values are still dropped, since recalculation fills them again. Tests: a_python_spill_whose_blank_cells_have_no_records_is_still_a_spill and a_python_spill_range_larger_than_a_result_keeps_its_values_as_plain_values; both fail without the fix. |
||
|
prxnvt
|
b78afc9b3c |
Merge origin/main (25881a8e) into the Python cells branch
Conflicts: ROADMAP.md and docs/file-format-parity.md only (header lines combined, both sides' rows kept). |
||
|
Brandon Thomas
|
e361f922d5 |
Merge PR #3130
# Conflicts: # docs/file-format-parity.md # docs/target-app-parity.md # docs/ui-parity.md |
||
  Ingmar LübbeandClaude Opus 5.5
|
e2c523881f |
Page Layout: Page Setup's page order (Down, then over / Over, then down)
A sheet more than one page wide and tall always printed down, then over. PrintSettings gains the page order (serde default Down, then over), pageLayout.pageSetup takes pageOrder, and the pagination numbers the pages in it, so printing, PDF export, the print preview, Page Break Preview, Page Layout view and the &P/&N, first-page and odd/even headers follow the printed sequence. XLSX reads <pageSetup pageOrder> and writes pageOrder="overThenDown" only, as Excel does. The Page Setup dialog gets a Page order choice with a small picture of the order and runs pageLayout.pageSetup, so the order and the margins are one undo step; the labels are translated where the dialog is, and the dialog is now in German too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj |
||
  campbellcodyandCodex
|
b6c4500fa7 |
M7: add data tables beneath charts
Add command and UI controls for chart data tables, shared chart and PDF rendering, and native XLSX persistence. Cover live values, blanks, formatting, alignment, undo/redo, bounds and table flags with regression tests. Co-Authored-By: Codex <noreply@openai.com> |
||
|
prxnvt
|
b3a3590db6 | Merge main into Python cells as data | ||
|
prxnvt
|
84d4f6aa84 |
XLSX: a Python script's length is measured after its escapes are decoded
A script's code was cut at the character limit before its ST_Xstring escapes were decoded, so a script whose escapes (`_x0009_`, `_x000D_`) made the file's text longer than the limit lost its end without a warning, although its code fits. The code is now decoded first and then cut one character past the limit, so a script that is too long once decoded still fails the check and its cell keeps only its value, as before. Decoding is linear and reads at most a bounded number of encoded characters, enough for any code that fits. |
||
|
prxnvt
|
e82e91b451 |
XLSX: a Python cell without its code keeps its whole cached result as values
A PY formula whose script the file doesn't hold (the Python part missing or damaged, or the script over the code budget) keeps only its cached value, but its array range stayed recorded, so the cached values over the rest of its spill were dropped as a formula's spill would be, and nothing filled them again. The range of such a cell is now forgotten when it is read: the anchor and the cells of its spill keep their cached values as plain values, as the file shows them, with no live spill. An ordinary formula's dynamic-array spill is handled as before. |
||
|
prxnvt
|
113aeff07c |
XLSX: a damaged Python part is a warning, and the workbook opens with its values
A Python part whose zip entry failed to read (a bad checksum or compressed stream) made the whole workbook fail to open: the error was passed up, and the failed read had also ended the package's read budget, so even a warning would have left the sheets unread. The part is now read with Package::read_optional_limited, which charges a failed read only that part's allowance, and every error other than TooLarge is a warning. The Python cells then keep only their cached values, as when the part is missing. Malformed XML in the part was already a warning; the test covers it too. |
||
|
Brandon Thomas
|
e129c072a1 |
Merge PR #3075
# Conflicts: # crates/xlsx/src/read.rs # docs/file-format-parity.md |
||
|
Brandon Thomas
|
bc173ead8f |
Merge PR #3062
# Conflicts: # ROADMAP.md # docs/file-format-parity.md |
||
|
Brandon Thomas
|
7f1574b573 |
Merge PR #3052
# Conflicts: # docs/file-format-parity.md |
||
|
prxnvt
|
c03c61fd05 | Merge main into Python cells as data | ||
 scks1
|
60a267b4ef | #3057: select visible workbook views and normalize tab ratios | ||
 Harrison SmithandClaude Opus 5.5
|
a1e0383056 |
#2871: unwrap Google Sheets' wrapper only when GridCraft calculates the formula
A formula calling a Google-only function (GOOGLETRANSLATE, QUERY, ARRAYFORMULA...) opened as #NAME? where the wrapper had shown the cached value, as Excel does. The XLSX reader now unwraps a cell formula only when every function inside is known: read_xlsx_with takes the predicate (the xlsx crate can't depend on the function library) and the engine passes gridcraft_calc::is_known_function. Unwrapping stays in the reader so shared formulas are shifted from the unwrapped master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Harrison SmithandClaude Opus 5.5
|
2c0a00fbae |
#2871: open Google Sheets' DUMMYFUNCTION export wrapper as the original formula
Google Sheets' XLSX export writes a formula that uses a function it thinks
Excel lacks as IFERROR(__xludf.DUMMYFUNCTION("<formula, quotes doubled>"),
<cached value>). GridCraft showed the wrapper as the formula.
fmla::from_file, which every cell formula read goes through (plain, shared
master and t="array"), now recognises exactly that shape: IFERROR (with or
without _xlfn.) of a single-string __xludf.DUMMYFUNCTION call, any case.
If the string parses, the formula inside replaces the wrapper, as the whole
formula or nested in a larger one (parenthesised unless it is a single
operand). The cell keeps the file's cached value.
Bounded on hostile input: inner text over MAX_FORMULA_LEN is not parsed,
only one level is unwrapped, and a result that no longer parses (nesting
past the parser's limit) keeps the file's formula. Text that doesn't parse
keeps the wrapper.
Saving writes the real formula with Excel's _xlfn. prefixes, as Excel does
with the _xlfn functions it knows. Keeping cached results for functions
GridCraft can't calculate is #2770 and not part of this change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Simon Koch-Sultan
|
9276827b1a |
xlsx: keep the workbook view settings across an open + save (#3057)
The reader took only activeTab from <workbookView> and the writer wrote that alone back, so a workbook Excel saved with the sheet tabs and both scroll bars hidden (and a non-default tab split) opened with all three shown and lost the settings on save. Workbook gains a view field (WorkbookView: show_sheet_tabs, show_horizontal_scroll, show_vertical_scroll, tab_ratio; the format defaults are shown/shown/shown/600). The reader fills it from the <workbookView> attributes; the writer emits each attribute only when it differs from the default, so an ordinary workbook file is unchanged. Regression test: workbook_view_settings_survive_a_round_trip reads a package carrying the hidden settings, checks the model, and checks the written workbook.xml keeps all four attributes (and that a default workbook still writes the bare element). Scope: this is the data-loss half of #3057. The app does not act on the settings yet and File > Options has no controls for them; that is the other half the ticket was split into. |
||
 Daniel LubrykaandClaude Sonnet 5.5
|
d25445f4e3 |
M2.1: document the .xls style helpers
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Daniel LubrykaandClaude Sonnet 5.5
|
879ea7385f |
M2.1: test that .xls fonts keep their family and charset
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Daniel LubrykaandClaude Sonnet 5.5
|
f2a75eca1b |
M2.1: keep BIFF8 font family and charset when importing .xls fonts
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
 Max Murphy
|
72b1310a5a | #2765: CSV and text open detect each column's date order | ||
|
Brandon Thomas
|
ac27e3da14 | Merge PR #2793 | ||
 Harrison Booth
|
b612141726 | xlsx: keep intl date functions in xlfn list | ||
|
Harrison Booth
|
c82a138462 | M2.1: fix XLS style test clippy lints | ||
|
Harrison Booth
|
591c6ed6f9 | Fix XLS style test Clippy complexity | ||
|
Harrison Booth
|
9587f08253 | Merge upstream main into gc-xls-cell-styles | ||
 truongitandClaude Opus 5.5
|
94433a3c4f |
Document the Agile decryption steps
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
truongitandClaude Opus 5.5
|
9e0f14cab9 |
Open Excel's password-protected workbooks again: accept a hash-size integrity key (#2712)
Since #2425 checks Agile encryption's dataIntegrity, every workbook Excel encrypts fails with "the decrypted data failed its integrity check". The reader keyed the HMAC with the decrypted encryptedHmacKey cut to the salt size (16 bytes), as [MS-OFFCRYPTO] 2.3.4.14 describes it, but Excel's key is as long as the hash: 64 bytes for SHA-512. The check now accepts a key of either length, so spec-literal writers still open and a modified package still fails. The test encryptor writes the integrity key as Excel does, so every encrypted fixture now has Excel's layout; a new test covers both key lengths, a key of neither length, and tampering with each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
 Dylan M. Taylor
|
84e684e586 | Refactor xlsx for strict Clippy checks | ||
|
Harrison Booth
|
f6015a7e6e | Pin COPILOT as written without an _xlfn. prefix | ||
|
Harrison Booth
|
25816f0009 | xlsx: write _xlfn. for STOCKHISTORY, TRANSLATE, DETECTLANGUAGE; none for NETWORKDAYS.INTL, WORKDAY.INTL | ||
|
Harrison Booth
|
0e643e9026 | Merge remote-tracking branch 'upstream/main' into gc-xls-cell-styles | ||
 NetterSilverandClaude Opus 5.5
|
393a524aa3 |
Speed up XLSX worksheet writing
Saving a large workbook spent about half its time building worksheet XML, mostly in small per-cell allocations. The writer now appends to the part directly: - Cell references, row numbers, style ids, shared string indexes and numbers are appended without temporary strings (push_a1, push_u64, push_num), and text is escaped in place (push_esc, with a fast path for plain ASCII). - sheetData walks the cell store in its own row-major order instead of copying every address into a BTreeSet and a per-row BTreeMap and looking each cell up again. Cells outside the store (pictures, spill values, table headers) still go through the sorted set. - Formulas with nothing the file spells differently (no LET/LAMBDA, no _xlfn function, no spill, @, trim operator or structured reference) are printed from the shared expression instead of being cloned and rebuilt twice by the file rewrite. - The A1 printer in gridcraft-formula appends references to the output instead of formatting each anchor into its own string. The output is byte-for-byte unchanged. write_perf, 200,000 rows (2.0M cells, 14.2 MB package), release, Ryzen 16 threads / Linux, 36 interleaved runs each: before: median 2083.9 ms, min 1615.7 ms after: median 1501.3 ms, min 1192.3 ms (-28.0 % median, -26.2 % min) What remains is mostly deflate (zlib-rs, level 6), about 70 % of the save. Tests pin the exact sheetData of a workbook covering row settings between cell rows, spill values, every value kind, number spellings, escaping and formulas with and without a file spelling (generated with the previous writer), and check each appending helper and the formula fast path against the code they replace. The formula fast path is guarded by a test that runs both paths over every function in the gridcraft-functions registry (a new dev-dependency, allowed by the layer rules), in 15 surrounding shapes covering every operator and rewrite, so a function or rewrite added later without updating the trigger list fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
 aiongg
|
7f1ccf2e81 | refactor(xlsx): move sheet_read.rs items to sheet_read/ | ||
|
aiongg
|
af54396e64 | refactor(xlsx): move sheet_write.rs items to sheet_write/ | ||
|
aiongg
|
d1c8de48e7 | refactor(xlsx): move passthrough.rs items to passthrough/ | ||
|
aiongg
|
e8f3b6f0cf | refactor(xlsx): move pivot.rs items to pivot/ | ||
|
aiongg
|
b3e2a6a2be | refactor(xlsx): move crates/xlsx/src/tests/mod.rs to crates/xlsx/src/tests.rs | ||
|
aiongg
|
24a18620d2 | refactor(xlsx): move crates/xlsx/src/xls/mod.rs to crates/xlsx/src/xls.rs | ||
|
prxnvt
|
fcbbfa5142 |
XLSX: Python cells as _xlfn._xlws.PY() formulas, code in the Python part, cached values; opening limits
A Python cell is written to xlsx as [MS-XLSX] 2.3.10 specifies: its formula is _xlfn._xlws.PY(i,0|1), where i is the zero-based index of the cell's script in the workbook's Python part and 0|1 the return type, 0 for an Excel value and 1 for a Python object. The part (2.1.23) is xl/python.xml, content type application/vnd.ms-excel.python+xml, related from the workbook part with http://schemas.microsoft.com/office/2023/09/relationships/Python; its root python holds pythonScripts, one pythonScript per Python cell in the order the cells are written, with the code in code. The code keeps its xl() references as text, so the formula has no further arguments. The cell keeps its cached value. A spilling result is written like a dynamic array (cm="1", t="array" ref=...) with the spilled values cached, so other applications show the values (and #NAME? if they recalculate). The reader turns a PY( formula with or without the _xlfn. and _xlws. prefixes into a Python cell. An index names a script of the Python part, and each %Px% in the script becomes the text of the formula's argument x (2.3.10.4). The string-literal form, PY("chunk"&"chunk",0|1), still reads, from the formula's own text so line breaks in the code survive. A formula naming a script the part doesn't hold keeps only its cached value, with a warning: no calculation can evaluate it. The reader keeps the cached spilled values as the cell's result, so a workbook from Excel shows its arrays without running anything. Python errors are cached as #VALUE!, as #SPILL! and #CALC! already are. Native JSON keeps Cell.py whole: code, return type, the last result and the exact error. Calc lays out a Python cell's kept result: Python cells are nodes of the dependency graph that read nothing, and a pass takes a dirty Python cell's result instead of evaluating it, so its array spills, is blocked (#SPILL!) and unblocked, and feeds its dependents exactly like a formula's. An array that is malformed or has more than 1,000,000 cells (only a crafted file holds one) shows #SPILL!. A dirty Python cell is pending in the pass, as a dirty formula is, so a formula, a range read or a cell-by-cell range walk that reaches it before its turn lays it out first rather than reading its old value. Opening a file can't make the app loop over a claimed size: the reader keeps a Python cell's spill range only when it has at most 1,000,000 cells (a range claiming the whole sheet is dropped with its values, as a formula's spilled values are), and io::adopt_python rebuilds results for at most 4,000,000 cells over the whole workbook, with a warning for the cells it skips. The Python part is read up to 64 MB and 1,000,000 scripts, a script is kept up to one character past what a cell may hold, and the cells take at most 64 MB of code from the part in all, since several formulas can name one script. io::enforce_python_limits drops the code of a cell whose code doesn't fit in 8,192 characters as PY(...) (the cell keeps its cached value, with a warning) and a kept result that isn't a well-formed array within the cap, each with its spill range and spilled values, so the sheet and a save no longer show the old spill. Formats that keep PY() as an ordinary formula (xls, xlsb, ods) are adopted the same way. Opening a file runs nothing. The command-sequence fuzzer sets Python cells (code with xl() references, quotes, line breaks, and code at and over the size limit), and its xlsx round-trip check allows the Python errors becoming #VALUE!, which is what the file format stores. Tests: the written formula naming its script, the Python part with its content type and relationship, return types, a spilling table saved as a dynamic array, Python errors as #VALUE!, the round trip of values, code, long code with quotes and line breaks and the error, and no Python part without Python cells; a workbook laid out as the format specifies (an index, a parameter named as %P2%) read as Python cells with their spilled values and saved back with its code in a Python part; the string-literal form; a formula naming a missing script, with and without a Python part; 20,000 cells naming one long script against the code budget; the part's reader and writer (escaping, odd %P tokens, a long script, a repeated parameter); a blocker in the spill area giving #SPILL! and clearing it spilling again; the native JSON round trip; a spill range claiming the whole sheet; the workbook-wide budget; hostile JSON (code over the limit, a result claiming far more cells than it holds); a malformed result that never spills; PY() formulas at 8,079 and 8,080 characters of code. Calc: formulas, MAX, SUM, COUNTA and SUMPRODUCT read a dirty Python cell whose array is blocked before its turn, in a full and an incremental recalculation, and see the array once it is unblocked; enforce_python_limits removes the spill of dropped code and of a dropped result, and keeps a sound one. |
||
|
prxnvt
|
44c663aea2 |
Model: Python cells, and the #PYTHON!, #BUSY!, #TIMEOUT! and #BLOCKED! errors
Cell gains `py: Option<Arc<PyCode>>`, read with a serde default and not
written for ordinary cells, so older native files load and ordinary
cells serialize as before. PyCode holds the code, the return type (an
Excel value or a Python object), the last result (value, type name,
preview, whether it converted to cells) and a stale flag for manual
calculation. The cell's `value` stays the cached result, so formulas,
display and saving read a Python cell like any other. is_blank,
CellStore::has and input_text (which returns the code) take the code
into account; Cell::has_content and clear_content cover value, formula
and code together.
The model reads and writes the PY function's syntax as Microsoft
documents it, PY("chunk"&"chunk",0|1), with string literals of at most
255 characters and the return type 0 for an Excel value, 1 for a Python
object. The reader takes the `_xlfn._xlws.` prefix (and `_xlfn.` or
`_xlws.` alone, in any letter case); to_file_formula writes it. (An xlsx
file keeps the code in its Python part instead; the xlsx commit adds
it.) The size limit is Excel's 8,192 characters for that formula, not a
count of code characters: doubled quotes and the "&" between literals
count, so code with many quotes fits in fewer characters (8,079
characters of plain code fit exactly). The model also finds the
xl("...") literals in code, skipping comments and other strings; a plain
literal may escape a quote or a backslash. A rewritten reference that
holds the literal's quote (a sheet name that needs quotes, 'Q4
2026'!B2) takes the other quote as the literal's delimiter, or an
escaped quote when it holds both, so it never ends the string early.
Edits: cell.set over a Python cell, Clear Contents, Clear All and the
cells a merge empties remove the code; Clear Formats keeps it. Copy and
paste (All, Formulas), Fill, sort, cut-move, sheet copy and undo carry
it; a cut to another sheet first qualifies its xl() references with the
source sheet, as a moved formula's references are, so they keep
reading the same cells. Paste Values and Paste Link do not carry it,
and paste operations do not combine numbers with a Python cell. Fill
copies Python cells instead of extending them as a series. cell.set
with =PY(...) is refused with a message that points to python.set, and
Text to Columns keeps a piece that reads as =PY(...) as text instead of
clearing it. Goal Seek refuses a Python cell as its changing cell, and
Solver as a variable cell, as both refuse formulas: the number written
there would replace the code. The sheet-protection check compares the
code, so a locked Python cell cannot be changed. cell.get reports
{python: code, output, typeName, preview, stale}.
The xl() literals follow reference-moving edits: rewrite_all_formulas
(row and column insert and delete, cell insert and delete, cut-move),
sheet rename and sheet delete adjust each literal like a formula
reference, and replace it only when it changed.
Errors: CellError gains Python, Busy, Timeout and Blocked, and ALL has
14 entries, so the lexer reads them as literals. ERROR.TYPE gives 11 for
#BLOCKED!, the number Excel returns. Microsoft documents no ERROR.TYPE
number for #PYTHON!, #BUSY! or #TIMEOUT!; they take 19, 20 and 21, after
the numbers Excel is known to use. The xlsx writer caches the four as
#VALUE!, as it already does #SPILL! and #CALC!, because they are not
error codes of the file format.
Tests cover the =PY refusal, removal by entry and clearing, undo, copy,
paste values, fill and sort, xl() rewrites through row insert, sheet
rename (also to names that need quotes, with one quote and with both),
cut-move (also to another sheet) and column delete, Goal Seek and
Solver refusing Python cells, Text to Columns keeping a =PY piece,
native JSON round trips, the new errors, the prefixes, escaped quotes in
literals, and the encoded size limit at 8,192 characters exactly and
with doubled quotes.
|
||
|
aiongg
|
f102fe29a1 |
Merge g3 into g4
# Conflicts: # ROADMAP.md # crates/engine/src/cmd/edit.rs # crates/engine/src/cmd/formulas.rs # crates/engine/src/cmd/inspect.rs # crates/engine/src/cmd/pivot.rs # crates/ui-egui/src/lib.rs # crates/ui-egui/src/tabs.rs # docs/gaps.md # docs/ui-parity.md |
||
|
Dylan M. Taylor
|
52cf825395 |
refactor: split complex functions for Sonar S3776 (#2609)
Extract helpers from auto_sum (selection and single-cell guess), update_cursor (drag, hover, freeze, and header-edge cursors) and read_styles (dxf reader). No behaviour change. |
||
|
3b65b9af58 |
Sonar maintainability: explicit imports, method paths, smaller functions (#2412)
* Sonar: explicit imports, fn paths for closures, extract helpers for cognitive complexity Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Fix imports * Fix serde imports * Fix engine imports * Fix clippy gate and a type error from the explicit-import change Remove the unused imports the review flagged (cf.rs, pivot.rs, tables.rs, cmd/sheet.rs) so clippy -D warnings passes, and point the backstage parent-folder lookup at Path::to_str, which the fn-path rewrite had broken. Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Apply rustfmt to the explicit-import cleanup so the fmt gate passes * Share the per-directory font path expansion in theme.rs The candidate font lists repeated the same loop over the Linux font directories in every role. A single helper expands them in the same order, so the lookup order is unchanged and the duplicated blocks go away. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Read the Windows fonts folder in one place in theme.rs Every script-font list repeated the same WINDIR lookup. A single helper returns the same path, so the duplicated lines go away without changing which font files are tried. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Build the light and dark UI palettes from one table in theme.rs Tokens::light() and Tokens::dark() listed the same 40 colour fields in the same order, so every change had to be made twice. A single palette constructor takes the mode and picks each colour's value, so each field is written once and both modes keep their exact colours. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Share the Noto CJK font list between the CJK script lists in theme.rs The Japanese, Simplified and Korean lists each repeated the same loop over the Linux CJK directories with a different locale file name. One helper builds that loop, and each list keeps its own face index, so the lookup order is unchanged. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Share Linux Noto directory list across Thai and Hebrew font lookups Removes the duplicated directory arrays flagged by SonarCloud new-code duplication. * Reuse in_dirs for Thai and Hebrew Noto font candidates Removes the remaining duplicated per-directory loops flagged by SonarCloud. * Share one font-list builder for the Thai and Hebrew faces The two per-script functions repeated the same Windows, macOS, Linux and variable-font setup. A single script_fonts helper now builds both lists, keeping the candidate order unchanged, which clears the Sonar duplication gate. --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Codebuff <noreply@codebuff.com> |