Commit Graph
2140 Commits
Author SHA1 Message Date
el-j cdf4a2fa6e craft-audio: native FLAC decoder (flac-decode)
Adds flac::decode for whole files and flac::Decoder for single frames:
RFC 9639 subframes (constant, verbatim, fixed, LPC), Rice and escaped
partitions, wasted bits, all stereo modes, fixed and variable block
sizes, 4 to 32 bits, CRC-8 and CRC-16 checked. Not yet wired into
craft_audio::decode, which still uses symphonia for FLAC.

Part of #1546
2026-11-15 15:11:43 +01:00
Dylan M. Taylor fe6b443acf Merge pull request #1505 from storytold/fix/svg-embedded-image-accounting
craft-svg: account for malformed embedded image data
2026-11-15 06:37:46 -05:00
Dylan M. Taylor 55b1d7f3c5 Merge pull request #1501 from storytold/extract/apng-container
craft-image-codecs: extract bounded APNG container assembly
2026-11-15 06:30:57 -05:00
keithcsqueensu 5a0caf286e craft-xlsx: .xlsb cell formulas, shared, array and dynamic arrays (ports of gridcraft#3810, #3818, #3821) (#1166)
craft-xlsx decodes `.xlsb` cell formulas, including shared, array and dynamic-array formulas, into the text an `.xlsx` file holds for them. Until now they were skipped and only their saved values kept. Part of #778.

Ported from storytold/gridcraft#3810, #3818 and #3821 (issue gridcraft#3806), my own series; the commits keep me (kiwi4boy) as their author. The three PRs are one series, each containing the one before.

## Commits

1. **The XLSB function table** (gridcraft#3810): `xlsb/ftab.rs`, [MS-XLSB] 2.5.98.10. It is craft-xls's table up to 379 (0x17B), which matches GridCraft's before the PR, plus the PR's 0x17C–0x1E4 entries (CUBE*, the Analysis ToolPak functions, IFERROR, COUNTIFS, SUMIFS, AVERAGEIF(S)). It is craft-xlsx's own copy, not shared with craft-xls: the formats follow separate specifications and their readers develop separately. They can share one table later once both settle.
2. **Cell formulas** (gridcraft#3818, with its two review fixes): `xlsb/formula.rs` is a BIFF12 decoder, separate from craft-xls's BIFF8 one, because nearly every operand differs. `xlsb/expr.rs` is a copy of craft-xls's expression tree and A1 printer, with `BinOp::precedence` added for grouping. A formula cell's `CellFormula.text` is in the file's spelling (`_xlfn.CONCAT`, `_xlpm.x`, no implicit `@`), and a `BrtCellMeta` before a cell sets `cell_metadata`, as `cm` does. The workbook's sheet links, supporting links and names are read with the workbook part, and `Package` keeps them for its sheets. A new `Limits::binary_formula_text` (128 MiB) bounds decoded text.
3. **Shared, array and dynamic-array formulas** (gridcraft#3821, with its review fixes): `PtgExp` cells, `BrtShrFmla` and `BrtArrFmla`. See the mapping below.
4. **A shared formula stops decoding copies once one is kept as its value**: a fix for CodeRabbit's finding that is still open on gridcraft#3821, so it isn't carried over.
5. **README**: what the decoder gives, what keeps its saved result, and the new limit.
6. **Review fixes**: 468 (0x1D4) is CONVERT. The specification's table has no row for it, but Excel writes CONVERT as `PtgFunc` 0x1D4. A shared formula's tokens are borrowed from the part instead of copied. A formula kept as its value loses its cell metadata.
7. **Review fixes, round 2**: a chain of strings joined with `&` is flattened only up to the decoder's depth bound (256 strings), so a balanced tree of thousands can't become a chain that overflows the stack when printed or dropped. A first cell whose range record never comes loses its cell metadata.
8. **Excel's spelling, from a second Excel round:**
   - Spaces and line breaks typed in a formula (`PtgAttrSpace`) are kept where `.xlsx` keeps them (`1 +  2`, `SUM( A1, B1 )`), and the text is trimmed as the `.xlsx` reader trims it.
   - A number is written out in full when that takes at most 21 characters without the sign, and in scientific notation otherwise (`100000000000000000000`, `1E+21`), as Excel writes it. The decoder switched at 1E+15 and 1E-9.
   - ISO.CEILING, ECMA.CEILING, NETWORKDAYS.INTL and WORKDAY.INTL are called through hidden `_xlfn.` names in `.xlsb`, but `.xlsx` spells them without the prefix.
9. **Macro sheets and spans over chart sheets:** a reference to an XLM macro sheet (`Macro1!$A$1`) is named, since macro sheets have cells. A 3-D span with a chart sheet between its ends is spelled by its ends, as `.xlsx` has it. An end with no cells still keeps the saved result.

## How it maps onto craft-xlsx's model

- A formula's text is what the `.xlsx` reader gives for the same formula. GridCraft's reading of that text stays in GridCraft: `fmla::from_file`, the implicit `@` on legacy formulas, its parser check, and Python-cell adoption.
- **Shared formulas:** every cell of the group gets `FormulaKind::Shared` with the group's `si` (counted per sheet), and the first cell also gets the range. Unlike `.xlsx`, where the other cells have empty text, each cell here gets its own text, decoded from the shared tokens at that cell. [MS-XLSB] defines it that way, with relative offsets that wrap around the grid. Shifting the first cell's text is not exact; the review of the GridCraft PR found that. `CellFormula.text`'s doc says so.
- **Arrays:** the first cell gets `FormulaKind::Array` and its range. The other cells keep their cached values and have no formula, as in `.xlsx`. A dynamic array also has its `cell_metadata`, even across a `BrtValueMeta` written in between. Clearing the other cells, and deciding from `fDynamic` whether to spill, is the consumer's call (gridcraft#3825).
- **Kept as values:** a formula that needs something the reader doesn't read keeps only its saved result, with a warning: user defined names (XLSB names aren't read yet), table references, other workbooks, `PY(index, …)` (code stored elsewhere in the package) and data tables. So do malformed tokens or records, and formulas past the allowance. A workbook whose links can't be read opens with values only, as before.

## Changed from the GridCraft PRs

- craft-xlsx's own function table, tree and printer, instead of GridCraft's formula crate and the table and helpers shared with its `.xls` decoder.
- A single-cell dynamic-array formula is read in commit 2, where #3818 left dynamic arrays to #3821: with `cell_metadata` there is nothing more to decide.
- The test of GridCraft's 8,192-character parser limit is replaced by one showing that a shared copy may be longer than its first cell. Tests that saved to `.xlsx` with GridCraft are dropped. The rest of the PRs' tests come along in `tests/xlsb.rs`, with craft-xlsx's model in the expectations.
- `Limits` gets a new public field, `binary_formula_text`. Code that builds `Limits` with `..Limits::default()` is unaffected.

## Checked against Excel

Excel saved the same workbook (`make_probe.ps1` in the GridCraft work: refs, 3-D refs, names, shared, CSE and dynamic arrays, newer functions, LET, a table, an external link, unions and intersections, array constants) as `.xlsb` and as `.xlsx`. Read through craft-xlsx, all 33 formulas this decodes from the `.xlsb` match the `.xlsx`: same text, kind, range and shared index, with the same cell metadata. The other 5 use tables, an external link or defined names, and keep their values with those warnings.

A second, wider round (commits 8 and 9) saved three more workbooks both ways:
- 487 functions Excel accepts on a worksheet: every name in the function table except 113 macro-sheet-only commands, and about 120 newer `_xlfn.` functions, LET, LAMBDA and its helpers, GROUPBY, the REGEX functions, `TRIMRANGE` and `A1:.A5`, IMAGE, RTD, CUBEVALUE and an add-in function;
- operators and precedence, unary and percent chains, unions, intersections, whole rows and columns, 3-D references and spans, quoted and cell-like sheet names, array constants, edge-magnitude numbers, a 255-character string, unicode, errors and missing arguments, legacy and dynamic-array `@`, references to a deleted sheet, column and row;
- 246 shared-formula cells, including relative references that wrap at the grid's edges, legacy array formulas, spills, spill references and a data table;
- 60 formulas typed with spaces and line breaks, and 62 number literals around Excel's switch to scientific notation.

Before commit 8, 725 formulas matched and 20 differed. Now 744 of the 745 match, with the same text, kind, range and cell metadata, and every non-first shared-formula cell matches Excel's own `Range.Formula`. The other one is a deleted column on a sheet named like a cell (`'S3'!#REF!`). That is #1330's: it prints the sheet before `#REF!`, which this PR doesn't. The spacing and number workbooks match in full. Two inline LAMBDA calls, an add-in call and a data table keep their saved results, as described above.

## Ran locally (own target directory)

- `cargo fmt -p craft-xlsx --check`: clean
- `cargo clippy -p craft-xlsx --all-targets --all-features -- -D warnings`: clean (and `cargo clippy --workspace --all-targets -- -D warnings` after commit 9)
- `cargo test -p craft-xlsx --all-features`: 74 pass, 0 fail (after commit 7); 76 after commit 9. No other workspace crate depends on craft-xlsx.
- The new tests fail without their commit's code: 5 of 6 for commit 3 (the sixth checks that unreadable shared formulas keep their values, which commit 2 already does), and 1 for commit 4. The tests for commits 8 and 9 fail without them.

## For the coordinator

Once this is merged, gridcraft#3810, #3818 and #3821 can be closed, each with a note like "Ported to craft-xlsx in storytold/craft-libs#1166; GridCraft gets it when it switches to craft-xlsx." GridCraft's remainder is the mapping. That means `from_file` and the implicit `@` (as for `.xlsx`), Python cells, clearing array cells, spilling dynamic arrays, the `xlsb-import.md` / `gaps.md` / `file-format-parity.md` notes and the engine's use of `read_xlsb_with`. I'll redo those against the switch.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

* **New Features**
  * Binary Excel workbooks now display decoded formula text in familiar spreadsheet notation, including shared and array formulas.
  * Shared-formula cells show formula text adjusted to their own cell references. Array-formula follower cells do not display separate formula text.
* **Bug Fixes**
  * When formulas cannot be decoded or exceed supported limits, cells retain their saved results; a warning may be provided.
* **Documentation**
  * Clarified formula display behavior and documented limits on formula text, token length, nesting depth, and array constants.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-15 12:11:39 +01:00
Dylan M. Taylor f29e82f1a7 Merge pull request #1467 from keithcsqueensu/rtf-writer-lookup-1466
craft-rtf: look up fonts, colours and authors through an index (#1466)
2026-11-15 06:03:00 -05:00
Dylan M. Taylor 390fe98b0d Merge pull request #1508 from storytold/codex/materialx-displacement-tiling
Preserve UV tiling from MaterialX displacement images
2026-11-15 06:02:13 -05:00
Dylan M. Taylor 7581345a22 craft-update: do not self-update extracted nextest test binaries (#1503)
* fix(update): treat extracted nextest archives as development builds

* fix(update): match only binaries listed by bounded nextest inventory
2026-11-15 05:35:32 -05:00
Dylan M. Taylor 0f5b23b4de Merge pull request #1455 from storytold/pr-mailcraft-smime
feat: add craft-smime (S/MIME CMS/X.509/PKCS#12)
2026-11-15 05:34:46 -05:00
Dylan M. Taylor 73149fe3e6 Preserve displacement image tiling in MaterialX 2026-11-15 10:20:02 +00:00
Dylan M. Taylor 2369c13f28 Merge remote-tracking branch 'origin/main' into HEAD 2026-11-15 10:12:11 +00:00
Dylan M. Taylor 0c144b7d98 Merge pull request #1496 from storytold/fix/webp-animation-export
craft-webp: expose animated writer and avoid duplicate frame buffer
2026-11-15 04:45:59 -05:00
Dylan M. Taylor d594b6763b fix(svg): use normalized data URL error accounting 2026-11-15 04:32:18 -05:00
Dylan M. Taylor 586041bb68 fix(svg): preserve omission accounting on malformed data 2026-11-15 04:30:16 -05:00
Dylan M. Taylor 4e9b3e9fdd fix(svg): account for invalid embedded image data 2026-11-15 04:11:43 -05:00
.CooCoo b63198042b craft-image-codecs: check the deep EXR corpus against pinned SHA-256 values (#1487)
The deep EXR corpus test read the five OpenEXR files in corpus/exr/
without a size bound or digest check, and a missing file failed with
advice to run `cargo xtask corpus --exr`, which craft-libs does not have.

- REAL_FILE_SHA256 pins the five files (OpenEXR commit 69b2604f). The
  test reads only those names, refuses a file that is missing, is not
  a regular file, is larger than 4 MiB or does not match its digest,
  and names the file and both digests.
- The failure points at the crate README, which says where the files
  come from and how to fill corpus/exr/, either from OpenEXR or with
  PhotoCraft's fetcher at the commit CI uses.
- Three tests that run without the corpus feature cover the pins, a
  changed byte, and missing, oversize and folder inputs.
2026-11-15 04:11:00 -05:00
Dylan M. Taylor 26a4f6c855 feat: add craft-proto (IMAP4rev1 + SMTP clients) (#1457)
* feat: add craft-proto (IMAP4rev1 + SMTP clients)

Port from mailcraft: IMAP4rev1 and SMTP protocol clients over any
Read + Write stream, no sockets or browser APIs.

## IMAP (RFC 3501, 4315, 4959, 6154, 6851)
- Mailbox operations: SELECT, EXAMINE, LIST, LSUB
- Message operations: FETCH, STORE, COPY, MOVE, EXPUNGE
- IDLE (RFC 2177): server-initiated change notifications
- AUTH: PLAIN, LOGIN, XOAUTH2 (RFC 4954)
- STATUS (RFC 3501 §6.3.10): mailbox stats without opening
- Modified UTF-7 encoding for mailbox names

## SMTP (RFC 5321, 3207, 4954)
- EHLO/HELO, MAIL FROM, RCPT TO, DATA
- STARTTLS (RFC 3207)
- AUTH: PLAIN, LOGIN, XOAUTH2

## Design
- No I/O of its own: works over any Read + Write stream
- Hostile-input bounded reading: lines, literals, counts capped
- Passwords and tokens never reach logs or errors (Secrets wiped on drop)
- Follows craft-libs safety pattern: no unwrap/expect/panic

* fix(proto): restore standalone protocol wiring and safe authentication helpers

* fix(proto): register dependencies and export message macro

* fix(proto): retain upstream private trace filtering and regression coverage

* fix(proto): redact uppercase private reply text and preserve error API

* fix(proto): satisfy hosted workspace lint gates

* fix(proto): bound IDLE polling and preserve mailbox updates safely

* test(proto): exercise BODY sections in FETCH context

* test(proto): distinguish fixed UID ranges from wildcard bounds
2026-11-15 04:02:49 -05:00
Dylan M. Taylorandbflatastic 979413980e Extract MIDI synchronization wire helpers from LoopCraft (#1478)
Preserve transport/SPP and non-drop MTC wire behavior; keep scheduling app-owned. Add allocation-free shared APIs and protocol regression tests.

Co-authored-by: bflatastic <bombayapirat@gmail.com>
2026-11-15 04:02:15 -05:00
Dylan M. Taylor 14e4ef8e5c fix(apng): enforce PNG integer ranges and satisfy chunk lint 2026-11-15 04:01:54 -05:00
.CooCoo cd8207b436 Merge pull request #1489 from dotCooCoo/fix/bounded-to-rgba-f32-a-tiff-strip-past-the-file
image-codecs: to_rgba_f32 checks its output against caller Limits
2026-11-15 04:01:40 -05:00
Tanish Manku 82fa80fca9 feat(image-codecs): extract bounded APNG container assembly
Preserve AniCraft animated PNG wire semantics while validating geometry, sequence numbers and allocation budget before writing.

Co-authored-by: Tanish Manku <tmanku.dev@gmail.com>
2026-11-15 03:58:22 -05:00
Dylan M. Taylor dd7c65ba32 Merge pull request #1479 from storytold/fix/wav-ixml-parity
craft-audio: preserve WAV iXML time-reference fallback
2026-11-15 03:53:25 -05:00
Dylan M. Taylor b7bc61e1a1 Merge pull request #1477 from storytold/fix/fcpxml-depth-parity
craft-timeline: preserve FCPXML depth and cache safeguards
2026-11-15 03:52:20 -05:00
Dylan M. Taylor 80d4e2395e test(webp): cover lossy alpha container and XMP framing 2026-11-15 03:26:23 -05:00
Dylan M. Taylor 502f056d6c xfl: expose neutral instance color effect view 2026-11-15 07:14:59 +00:00
Dylan M. Taylor ec53d5d399 fix(webp): expose animation writer and avoid full frame-list copy 2026-11-15 01:55:09 -05:00
.CooCoo 600fc71614 print: winprint 8ae097ef: from_dev_mode at its merge commit on main (#1474)
craft-print took PrintTicket::from_dev_mode (winprint.rs#24) at the pull request's earlier head
72951d74, which is not on winprint.rs's main. #24 merged as 8ae097ef, and the pin moves there.

The only code difference between the two revisions is in from_dev_mode's own size check: it now
accepts a DEVMODE whose public part reaches dmFields (76 bytes) instead of requiring 156 bytes.
craft-print passes it only settings that craft-winprint-gdi's DevMode::from_bytes accepted, which
still requires 156 bytes, so craft-print's behavior does not change. The API craft-print uses is
unchanged; only the rev and the lockfile move.
2026-11-15 00:12:37 -06:00
Dylan M. Taylor 0073b2674f Merge pull request #1480 from storytold/codex/xfl-library-schema
Add shared XFL library metadata views
2026-11-15 00:53:30 -05:00
Dylan M. Taylor bf48df0473 Merge pull request #1471 from storytold/codex/xfl-frame-sound
Add shared XFL frame sound schema views
2026-11-15 00:36:20 -05:00
Dylan M. Taylor ec2dd163b4 Merge pull request #1470 from storytold/codex/xfl-shape-hints
Add shared XFL shape hint and tween schema views
2026-11-15 00:22:00 -05:00
FredApps ad8faa1c13 Merge pull request #1300 from Terra854/codex/format-adapter-safety
Preserve native bundle saves and validate SVG/RAW decoding
2026-11-15 06:17:29 +01:00
Dylan M. Taylor 5542cd3d59 Merge pull request #1475 from storytold/codex/xfl-document-metadata
Generalize XFL document schema views
2026-11-15 00:02:26 -05:00
Dylan M. Taylor fc82610ef9 test(xfl): cover library metadata accessors 2026-11-15 04:51:19 +00:00
Dylan M. Taylor b8ac314286 Merge remote-tracking branch 'origin/codex/xfl-shape-hints' into codex/xfl-frame-sound
# Conflicts:
#	crates/xfl/src/lib.rs
2026-11-15 04:30:21 +00:00
Dylan M. Taylor c3b9c1f91a Merge remote-tracking branch 'origin/main' into codex/xfl-shape-hints
# Conflicts:
#	crates/xfl/src/lib.rs
2026-11-15 04:24:07 +00:00
Dylan M. Taylor ef6a3441c1 Merge pull request #1468 from storytold/codex/xfl-filter-schema
Add shared XFL filter schema views
2026-11-14 23:20:46 -05:00
Dylan M. Taylor 186005d279 Add shared XFL library metadata views 2026-11-15 04:18:13 +00:00
Dylan M. Taylor c4fb292825 Use canonical parity check in WAV iXML fixture 2026-11-14 23:02:09 -05:00
Dylan M. Taylor a44e1b7273 Reject cached FCPXML graphs that reach a rebuilding ancestor 2026-11-14 23:00:44 -05:00
Dylan M. Taylor 47036b5e63 craft-timeline: bound FCPXML traversal and retry truncated media 2026-11-14 22:57:47 -05:00
Dylan M. Taylor 44802bd54d craft-audio: preserve iXML WAV time-reference fallback 2026-11-14 22:57:45 -05:00
FredApps c29336211a Merge branch 'main' into codex/format-adapter-safety 2026-11-15 04:53:49 +01:00
Dylan M. Taylor 33f797781f Merge latest main into XFL schema views 2026-11-15 03:46:49 +00:00
Dylan M. Taylor c0ff073356 feat(xfl): expose shared color attribute parser 2026-11-15 03:29:34 +00:00
Dylan M. Taylor a9c1d06a0f Test XFL document views with app adapter 2026-11-15 03:24:35 +00:00
Dylan M. Taylor 7f3fdc0dfe Merge pull request #1464 from storytold/codex/xfl-text-schema
Add shared XFL text field schema views
2026-11-14 22:19:12 -05:00
Dylan M. Taylor 4f3507ffff Merge pull request #1454 from storytold/pr-mailcraft-ber
feat: add craft-ber (ASN.1 BER/DER reader)
2026-11-14 22:16:03 -05:00
Dylan M. Taylor 100b68ed48 Merge branch 'codex/xfl-shape-hints' of https://github.com/storytold/craft-libs into codex/xfl-frame-sound 2026-11-15 03:16:00 +00:00
Dylan M. Taylor 8d8c85e881 Merge branch 'codex/xfl-filter-schema' of https://github.com/storytold/craft-libs into codex/xfl-shape-hints 2026-11-15 03:15:37 +00:00
Dylan M. Taylor 85ca7a893c Merge remote-tracking branch 'origin/main' into codex/xfl-filter-schema 2026-11-15 03:15:12 +00:00
Dylan M. Taylor b304c0790f Generalize XFL document schema views 2026-11-15 03:13:43 +00:00