Commit Graph
3258 Commits
Author SHA1 Message Date
Neil Cawse 24f58a2ee4 Merge remote-tracking branch 'upstream/main' into fix/main-ci-csv-tests 2026-11-17 13:24:58 +00:00
Neil Cawse 319a3e4c2e Merge remote-tracking branch 'upstream/main' into fix/main-ci-csv-tests 2026-11-17 04:15:59 +00:00
Dylan M. Taylor d60709a173 pdf: follow up shared writer cutover with merged-library pin (#4407) 2026-11-16 22:44:24 -05:00
Dylan M. Taylor 0e5a82f084 ci: lock workflow actions through an actions.lock lockfile (#3879) 2026-11-16 21:58:36 -05:00
Dylan M. Taylor 4c943670cf pdf: move object and xref envelope writing to craft-pdf (#4398) 2026-11-16 21:58:31 -05:00
Dylan M. Taylor c8e69968c7 Merge pull request #4262 from vangel76/fix/ratex-parse-messages-i18n
Equation parse errors follow the interface language
2026-11-16 20:36:10 -05:00
Dylan M. Taylor 79742499b5 Merge pull request #3626 from vangel76/fix/i18n-hardcoded-strings
Status bar statistics, resize tips, sheet notes and macro toasts follow the interface language
2026-11-16 20:35:16 -05:00
Dylan M. Taylor d0f20a31da Merge pull request #4165 from vangel76/fix/4161-equation-fallback-font
Equation fallback text in xlsx is drawn at a normal font size
2026-11-16 20:32:51 -05:00
Dylan M. Taylor 7e4235c85b Merge pull request #4399 from storytold/dot/shared-xstring-20261117
xlsx: share SpreadsheetML string reader/writer codec (cutover stage 1)
2026-11-16 20:00:52 -05:00
Dylan M. Taylor ef87aaa590 test(xlsx): retain XML predicate for escaping oracle 2026-11-16 19:45:33 -05:00
Dylan M. Taylor 4ebc9a5127 Merge remote-tracking branch 'origin/main' into dot/shared-xstring-20261117 2026-11-16 19:44:21 -05:00
Dylan M. Taylor fc7698eb11 Merge pull request #4383 from Hamza0101/fix/table-hidden-total-row-keeps-calc
XLSX: a hidden Total Row keeps its calculations
2026-11-16 19:43:58 -05:00
Dylan M. Taylor 058ea183ec Merge pull request #4382 from Hamza0101/fix/xlsx-first-page-number
XLSX: keep Page Setup's first page number
2026-11-16 19:43:34 -05:00
Dylan M. Taylor faef1a06c8 xlsx: use shared SpreadsheetML string codec in reader and writer 2026-11-16 19:40:39 -05:00
vangel76 0c8e2f850e Merge fix/i18n-hardcoded-strings into fix/ratex-parse-messages-i18n 2026-11-16 21:55:34 +01:00
vangel76 54ebc232fb Merge upstream/main into fix/i18n-hardcoded-strings 2026-11-16 21:52:37 +01:00
Hamza 8d8716cdc0 XLSX: a hidden Total Row keeps its calculations
A table whose Total Row was turned off still holds each column's total:
Excel writes totalsRowFunction, totalsRowLabel and a custom
totalsRowFormula without totalsRowCount, and restores them when the row
is turned back on. The reader took them, but the writer wrote them only
while the row was shown, so a save lost a custom formula or an Average
and Excel brought back SUBTOTAL(109,...) instead. They are now written
whether the row is shown or not.

Fixes #4296
2026-11-16 20:10:00 +05:00
Hamza 496c13e7b4 XLSX: keep Page Setup's first page number
A sheet set to First page number 7 has pageSetup firstPageNumber="7"
useFirstPageNumber="1"; neither attribute was read, so a save reset it
to Automatic. PrintSettings now holds it (None for Automatic), the
reader takes the number when useFirstPageNumber is on, and the writer
writes both in the schema's order. A JSON workbook leaves it out when
Automatic, so existing files are written as before.

Fixes #4293
2026-11-16 20:04:07 +05:00
FredApps abcfc19153 Merge pull request #4364 from storytold/docs/shared-library-guidance
docs: prefer shared Craft libraries for reusable features
2026-11-16 13:51:45 +01:00
Dylan M. Taylor 3ca81a761b docs: prefer shared Craft libraries for reusable features 2026-11-16 06:27:17 -05:00
FredApps 1476d4e7ab Merge pull request #4184 from ZIDAN44/fix/3945-subtotal-grand-total-outline
engine: Subtotal groups every row under the grand total (#3945)
2026-11-16 10:25:43 +01:00
FredApps 91f153de5b Merge pull request #4247 from gmrmad1/docs/roadmap-2026-11-15
docs: bring ROADMAP.md and parity assessment current through 2026-11-15
2026-11-16 10:24:34 +01:00
FredApps 3443e06bfc Merge pull request #4183 from ZIDAN44/fix/3934-paste-row-down-narrow-selection
engine: paste repeats a copied row down a taller, narrower selection (#3934)
2026-11-16 10:24:21 +01:00
FredApps e1147239fc Merge pull request #4182 from ZIDAN44/fix/3949-custom-list-unlisted-sort
engine: custom-list sort puts unlisted values A to Z after the list (#3949)
2026-11-16 10:21:26 +01:00
Valentin KleinandClaude Opus 5.5 ef99f1e51f Functions: GETPIVOTDATA (#284) (#290)
* Functions: GETPIVOTDATA (#284)

The engine records each PivotTable report value cell (value field,
row/column items, value) on refresh and when a workbook is opened;
GETPIVOTDATA is an evaluator special form that looks the value up by
value field and field/item pairs, #REF! when the report doesn't show it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* GETPIVOTDATA: shared lazy index, report values, hash lookup (review of #290)

- PivotTable::data_index is an Arc-shared PivotLookup (serde-skipped), so the
  per-command workbook clone no longer copies the index; refresh sets it, clear resets it.
- No index work on open: the first GETPIVOTDATA on a pivot without one builds it from the
  source via the engine-registered PIVOT_INDEXER; a failure is #REF!, not an empty index.
- The result is the value the sheet shows at the cell's report position, so formula and
  report cell agree (test with a source edited after the last refresh).
- Lookup by a pre-lowercased hash key; numbers/booleans fall back to a scan on a miss.
- GETPIVOTDATA is no longer volatile: it depends on its pivot's whole report area.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* GETPIVOTDATA: keep the index of the stored report; collision-free keys; align estimates

The lookup index describes the report as last refreshed and is saved with
the pivot. A pivot loaded without one (XLSX) only builds it when the
recomputed layout matches the stored labels. Lookup keys are length-prefixed
and distinguish Boolean from text items. Function-parity estimates match.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* pivot: import PivotDataCell, PivotItemRef and PivotLookup explicitly after upstream dropped the glob import

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* pivot: add an ignored measurement of edits on a sheet with a large report

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* pivot: keep a failed lazy GETPIVOTDATA build retryable, never save an empty index

A failed lazy index build stored unwrap_or_default() into the shared,
first-set-wins PivotLookup, so a valid state coming later (undo, a
source fix, a reopen) could never rebuild it, and the empty list was
saved as a restored index. Cache only validated builds, treat an empty
list as no index in serialization, reject reports with more value
cells than the index cap instead of truncating, register the pivot
indexer in gridcraft-cli check, and add two regression tests.

* GETPIVOTDATA: move cached offsets with edits inside the report; exact numeric items first

Inserting or deleting rows or columns inside a report remapped the cells
and last_range but kept the cached value-cell offsets, so a lookup could
read another item's cell. shift_sheet_features now maps each indexed
cell through the edit into a new index (the shared one stays with the
undo snapshots); a value cell deleted with its line drops out (#REF!).

A numeric item was accepted on a formatted-label hash hit, so 0.6 and 1
both shown as "1" returned the first. A number now prefers the cell
whose item has exactly that value, before any label match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix pivot chart test for the new Output.data field

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 08:59:28 +01:00
FredAppsandFredApps 19f07f3a25 Native JSON workbook: read and write through craft-gridcraft-format (#4320)
* Native JSON workbook: golden bytes before the move to craft-libs

A workbook with text, numbers, a formula, bold, a number format and a macro is
saved as native JSON. Its bytes, the bytes saved after a newer-format macro is
kept, and the error for a non-workbook file are golden files written by the
current code. Opening the golden file as .json and .scjson and saving it again
gives it back.

* Native JSON workbook: read and write through craft-gridcraft-format

The `.json`/`.scjson` extension check, the reader and its error text, and the
pretty writer with newer-format macros written back into `macros` now come from
craft-gridcraft-format (storytold/craft-libs#1749), pinned at its PR tip. The
Workbook model, the style-index and cell-band rebuilds after a read, and macro
reading and checks stay here.

The golden tests from the previous commit pass unchanged.

Part of storytold/craft-libs#778.

* Native JSON workbook: pin craft-gridcraft-format to the craft-libs merge commit 9a6d8942 (storytold/craft-libs#1749)

---------

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-16 02:14:28 +01:00
Dylan M. Taylor 407e0e1fa9 Merge pull request #4314 from storytold/cutover/shared-pdf-runlength
pdf: use shared RunLength encoding for image streams
2026-11-15 17:36:23 -05:00
Dylan M. Taylor dbdec081e2 Merge pull request #4304 from storytold/extract/shared-xml-escaping
xlsx: reuse shared XML byte escaping
2026-11-15 17:19:16 -05:00
Dylan M. Taylor 24e40de94f Pin PDF RunLength dependency to merged shared fix 2026-11-15 17:02:36 -05:00
Dylan M. Taylor 44050e9447 Merge pull request #3023 from AxylumRust/fix/insert-modules-compiled
Engine: compile the insert submodules again (#2950)
2026-11-15 16:40:38 -05:00
Dylan M. Taylor d217e8272d Fix shared codec dependency for external GridCraft builds 2026-11-15 14:33:42 -05:00
Dylan M. Taylor f22b4ed9dd xlsx: import shared append escaping only for tests 2026-11-15 14:07:48 -05:00
Dylan M. Taylor 75c4b01db3 Use shared RunLength encoding for PDF image streams 2026-11-15 13:53:04 -05:00
Mouhand Kaddo 1d0b311b4a Preserve Arabic cell direction and RTL sheet layout in PDF exports (#955)
**Stacked on #940.** It contains all of #940 (Arabic shaping in PDF) and adds cell direction and right-to-left sheet geometry (`M12.AR5`). Review the [diff against #940's branch](https://github.com/Mouhand-Kaddo/gridcraft/compare/fix/arabic-pdf-text...fix/arabic-pdf-reading-direction); once #940 merges, this PR is up to date with it.

## Summary

PDF export now honors cell Context, Left-to-Right and Right-to-Left settings. Wrapped lines keep their paragraph direction. Right-to-left sheets place columns, repeated headings, merged cells and objects in the same order as the grid, and explicit Left and Right alignment stay on the side the grid shows them.

```diff
 PDF export
- infer each line separately; place every sheet left to right
+ preserve cell direction across lines; mirror RTL sheet positions
+ reject oversized text before appending an incomplete PDF block
```

Depends on #940 (Arabic PDF glyphs). The cell reading direction it reads landed with #947.

## Evidence

- **Before:** PDF ignored explicit reading order and RTL sheet layout. A public size-limit probe returned a PDF with an unmatched marked-content block.
  **After:** Direction, wrapped paragraphs, numeric brackets, RTL columns, headings, objects, clipping and sparklines have focused regressions. The same size-limit probe returns an error; its regression failed before the fix and passes afterward.
- At the current head, with Noto Sans Arabic installed and no `CRAFT_FONTS_DIR`: the `gridcraft-pdf` and `gridcraft-engine` tests, workspace clippy and fmt pass. The full `cargo xtask ci` with the craft-fonts bundle relies on the hosted gate.
- Poppler rendering inspected for LTR and RTL sheets. Bounding boxes confirm reversed column positions; logical Arabic strings remain searchable. No physical printer test.

## Merge Danger

**Door:** Two-way. Revert the commit to restore prior export behavior.

**Blast Radius:** PDF. Explicit cell direction is shared with XLSX persistence. Latin text retains the standard-font path, including forced RTL numeric text without an Arabic font.
2026-11-15 17:55:04 +01:00
Mouhand Kaddo 92ff21b9f2 Shape and preserve Arabic text in PDF exports (#940)
## Summary

Arabic text reached the PDF writer intact, then WinAnsi encoding replaced its characters with question marks. This fixes #923 through the shared PDF text path, including cells, headers and chart labels.

```diff
 PDF text
- encode Arabic through WinAnsi
+ shape with harfrust and Unicode bidi
+ embed licensed glyph outlines with matching weight coordinates
+ preserve logical source using ToUnicode and ActualText
+ fall back to the standard and CID fonts for characters the Arabic face lacks
```

Font binaries stay in the optional craft-fonts build input. Native unbundled builds can use installed Noto Sans Arabic. Explicit cell reading-direction integration follows separately.

**Web build:** shaped Arabic on web needs the bundled craft fonts. The current release workflow doesn't enable `web-craft-fonts`, so until a maintainer enables it, web sets Arabic in the CID font, as main does today, which has no Arabic glyphs, and shows a message that no Noto Sans Arabic face was found. The same applies to a desktop without Noto Sans Arabic, and to bold Arabic where only the regular face is installed.

## Evidence

- **Before:** Arabic exported as `?` (#923, v0.6.1); main now sets it in the CID font, which has no Arabic glyphs, so readers show blanks or boxes.
  **After:** native and bundled exports show joined letters, marks, Western/Arabic digits, bold/italic, wrapping, rotation and chart labels. Fresh `pdfinfo`, `pdffonts` and `pdftotext -bbox` checks accept the files; eight original Arabic strings match extracted logical text exactly.
- At the current head, with Noto Sans Arabic installed and no `CRAFT_FONTS_DIR`: the `gridcraft-pdf`, `gridcraft-engine` and `gridcraft-xlsx` tests, clippy, fmt, layers, assets, parity and the wasm check pass. The full `cargo xtask ci` with the craft-fonts bundle relies on the hosted gate.
- Characters the Arabic face lacks (emoji, Cyrillic) are set like non-Arabic text, and without an Arabic face for the text's weight Arabic takes the CID path main uses, with one message per export. Only a broken font or a size limit fails the export. Size limits cover text, glyph subsets, outlines and page content.

## Merge Danger

**Door:** two-way

**Blast Radius:** printing

`PdfDoc::finish` now returns a Result and every caller handles it. Latin-only rendering stays on its existing path. This is a glyph-outline PDF solution; platform printer and physical screen-reader behavior are not claimed.
2026-11-15 17:29:22 +01:00
Valentin Klein ee24d50ac8 Calc: overlapping spills resolve by anchor order, incremental matches full recalc (#2483)
## What

When two array formulas want the same cells, the one that already has its spill keeps them, whichever anchor comes first (this is what Excel does, measured on this PR by Harrison-Booth). The other formula shows `#SPILL!` until the winner is cleared, then it spills. When neither has a spill yet (a first calculation with no remembered layout), the anchor first in reading order (row, then column) wins, so the result never depends on evaluation order.

Before, an incremental recalculation let the existing spill keep its cells, while a full recalculation laid everything out from scratch and gave them to the earlier anchor. The two disagreed. A full recalculation now starts from the layout the sheet held, so both agree.

Also fixed: clearing a spilling formula did not unblock an array blocked by its old area when that array sat below or to the right of it.

## Why

Issue #2480: on the sales sample, `cell.set C14 =TRANSPOSE(B18:B24)` followed by `range.fill =D14:E18` over E13:F13 left F13 as `#SPILL!` in memory but `0` after a full recalc (the fuzzer, `command_sequence_fuzz` seed 0, hits this on #795). Now C14 keeps its cells, F13 is `#SPILL!`, and a full recalc gives the same.

Known limits: the xlsx reader does not restore spill layouts, so a workbook opened from a file is laid out by reading order only (Excel keeps the layout the file records); and when both anchors already held a spill and one grows into the other, reading order decides (Excel keeps the cells the other owns). The rule is documented on `Standing` in `crates/calc/src/recalc.rs`.

## Test

In `crates/engine/src/tests.rs`: the two-command sequence from the issue; both entry orders of an overlap, plus clearing the winner; and a no-history layout, where the earlier anchor wins in both entry orders. All compare cells with a full recalc.

## How verified

- `cargo fmt --check`, `cargo clippy -p gridcraft-calc -p gridcraft-engine --all-targets -- -D warnings`, `cargo test -p gridcraft-calc -p gridcraft-engine` (includes the fuzz test): all pass.

Fixes #2480
2026-11-15 17:27:51 +01:00
Dylan M. Taylor d593a8f99d Use existing craft-xml escaping for XLSX output 2026-11-15 10:19:11 -05:00
vangel76andClaude Opus 5.5 566eeef456 Equation parse errors from RaTeX follow the interface language
RaTeX reports parse errors only as finished English text. Its fixed
messages are now catalog keys, and its messages that carry a piece of the
source are read back into templates (i18n::unfill) and filled in again
after translating. A message of any other shape stays as RaTeX wrote it.
66 new keys in all 18 catalogs.

Fixes #4261

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 13:26:02 +01:00
skylar taylor-barrick 8bde636f27 Sandbox: test the socket owner ioctls with a real pointer (#4245)
The merged test asserts FIOSETOWN and FIOASYNC with the ioctl argument passed
by value, where the kernel reads the number as an address and fails with
EFAULT whether or not the filter refuses it, so the refusal alone is not the
reason the call came back. Passing a pointer, as a C caller does, the call
succeeds unconfined and is EPERM confined.

This covers FIOSETOWN and SIOCSPGRP that way on a socket the parent made and
the program inherited as a standard stream, after reading its socket name so a
pipe cannot pass for one. SIOCSPGRP is in the rules and in the filter's count
assertion but in no test until now.

Deleting either rule line fails this test at that ioctl's assertion; restored,
it passes. The test above is untouched.
2026-11-14 22:04:39 -06:00
Madhav Agarwal bfcb3a515a docs: bring ROADMAP.md current through 2026-11-15 2026-11-15 09:12:02 +05:30
skylar taylor-barrick 2f52ae5c7e Sandbox: refuse fcntl's asynchronous notification on Linux (#3846)
* Sandbox: refuse fcntl's asynchronous notification on Linux

* Sandbox: tests for fcntl's notification rules and the new probe

* Sandbox: the socket ioctls that notify another process, too
2026-11-14 21:12:49 -06:00
Dylan M. Taylor a62a532753 Merge pull request #4242 from storytold/ci/sonar-pinned-fonts
ci: provision pinned fonts for coverage tests
2026-11-14 20:53:30 -05:00
Dylan M. Taylor 69219c0d5c ci: provision pinned fonts for coverage tests 2026-11-14 20:38:23 -05:00
Zinadin Zidan da46633db7 Merge remote-tracking branch 'origin/main' into fix/3945-subtotal-grand-total-outline 2026-11-15 03:15:01 +06:00
Zinadin Zidan b01be008a0 Merge remote-tracking branch 'origin/main' into fix/3934-paste-row-down-narrow-selection 2026-11-15 03:14:59 +06:00
Zinadin Zidan 4ea6f081d7 Merge remote-tracking branch 'origin/main' into fix/3949-custom-list-unlisted-sort 2026-11-15 03:14:57 +06:00
Neil Cawse 027d5d1f43 Merge remote-tracking branch 'upstream/main' into fix/main-ci-csv-tests 2026-11-14 18:05:00 +00:00
FredApps f2044337c7 Merge pull request #4190 from storytold/fix/oversized-pivot-cache-warning-text
Tests: the oversized pivot cache warning names the part limit (main is red)
2026-11-14 18:29:41 +01:00
Cédric Legrand 569707e3f0 fix(engine): M8: compile insert submodules 2026-11-14 18:25:22 +01:00
FredApps 34ab393c72 Tests: the oversized pivot cache warning names the part limit
The package reader reports a part over the limit as "expands to more
than <limit> bytes"; the test added in #2491 expected the declared size
and failed on main.
2026-11-14 18:20:30 +01:00