91 Commits
Author SHA1 Message Date
NetterSilverandClaude Haiku 5.5 55729a5128 Skip RTF header/footer left, right and first-page groups on import
Found while testing crates/textimport/src/rtf.rs:43-63 (SKIP list) and :201 (`SKIP.contains(&word.as_str())`)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 02:45:42 +01:00
NetterSilverandClaude Haiku 5.5 5da2393e28 Reject signed row numbers in XLSX cell references
Found while testing crates/textimport/src/xlsx.rs:15

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 18:14:57 +01:00
NetterSilverandClaude Haiku 5.5 8e5e40d76b Drop signed numeric character references in DOCX XML parser
Found while testing crates/textimport/src/docx.rs:132-133 (parse, Event::GeneralRef branch)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 08:32:16 +00:00
Dylan M. Taylor 36da81e03b fix(textimport): bound xlsx data sources and make docx and hyphenation parsing linear (#2213) 2026-11-03 22:23:32 -05:00
Dylan M. Taylorandaiongg 1d752d2f1f Text import: cap style names read from Tagged Text and DOCX files (#2258)
* textimport: cap style names read from files

* Apply review fixes

- crates/textimport/src/tagged.rs

---------

Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
2026-11-03 21:28:52 -05:00
NetterSilverandClaude Haiku 5.5 00717534cb Drop text variable markers in Text Only and RTF story export
Found while testing crates/textimport/src/export.rs:18 (plain_char) and :177 (rtf)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 20:55:44 +01:00
Brandon Thomas dfc0b19e7a Merge storytold/main into PR #1952
# Conflicts:
#	crates/textimport/src/tagged.rs
2026-10-29 21:53:38 -04:00
Brandon Thomas 3892c8ed1a Merge pull request #1871 from Hamza0101/fix/import-amplification
Import: a shared string and a footnote are not copied per reference (#1584)
2026-10-29 21:52:53 -04:00
Brandon Thomas fd207a0d7c Merge pull request #1998 from Silvertree2010/fix/h3964-tagged-text-import-trims-style-n
Keep leading and trailing spaces in Tagged Text style names
2026-10-29 21:52:46 -04:00
Brandon Thomas 1f20480194 Merge pull request #1948 from Silvertree2010/fix/h3527-placing-a-plain-text-file-keeps
Strip story marker characters from placed plain text files
2026-10-29 21:52:33 -04:00
Brandon Thomas e62583dfc4 Merge pull request #1905 from Silvertree2010/fix/h2977-rtf-import-a-short-or-invalid-hh
Fix RTF import swallowing characters after a short \'hh escape
2026-10-29 21:52:27 -04:00
Brandon Thomas 352d9c526d Merge pull request #1860 from Silvertree2010/fix/h2768-tagged-text-export-does-not-esca
Escape font family and typeface values in Tagged Text export
2026-10-29 21:52:16 -04:00
NetterSilverandClaude Haiku 5.5 e9d7a19994 Keep leading and trailing spaces in Tagged Text style names
Found while testing crates/textimport/src/tagged.rs:~190-205 (the ParaStyle and CharStyle branches of import: `unescape_value(value.trim())`)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 11:58:54 +00:00
NetterSilverandClaude Haiku 5.5 ae7e87ddd0 Skip signed hex codes in Tagged Text <0x…> tags
Found while testing crates/textimport/src/tagged.rs:~190 (the `n.starts_with("0x")` branch of import: `u32::from_str_radix(&n[2..], 16)`)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 17:22:28 +01:00
NetterSilverandClaude Haiku 5.5 79b871b94f Strip story marker characters from placed plain text files
Found while testing crates/textimport/src/lib.rs:140 (plain) and crates/engine/src/cmd/place_text.rs:372 (place_text)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 15:18:35 +00:00
Hamza 96b498ca70 Import: a shared string and a footnote are not copied per reference (#1584)
A footnote reference clones its note and walks it again, and an XLSX shared
string is copied per cell that uses it, so one 1 MB footnote referenced 100k
times meant around 100 GB. The text is kept at the first reference; later ones
get the mark and an empty note, and the report says so. The spreadsheet reader
stops at 64 MB of text and 400,000 collected cells.

Carries both review fixes:

- a footnote id is recorded only once its text has actually been kept, so a
  first reference arriving after the budget was spent no longer counts as
  "done" and tell the next reference the text is at a reference that has none;
- the report and the note budget are both bounded. The duplicate warning names
  the id, so 400 ids referenced twice made 400 warnings, each pushed after a
  Vec::contains scan of all the earlier ones; warnings go through warn() now,
  which dedups through a set and stops at 100 distinct messages. And the budget
  counted only `text`, which holds a table's anchor and not its cells, so 18
  notes of 600 KB of cell text retained 10,800,072 bytes against the 8 MB
  budget; note_bytes() walks the tables and their nested tables.

Rebased onto the post-split main: the implementation stays in docx.rs and the
tests moved to docx/tests.rs, so the test additions went there.
2026-10-28 18:10:41 +05:00
Dylan M. Taylor f1a5744e8c Refactor textimport for strict Clippy checks 2026-10-28 11:39:25 +00:00
NetterSilverandClaude Haiku 5.5 590ea661a4 Fix RTF import swallowing characters after a short \'hh escape
Found while testing crates/textimport/src/rtf.rs:~118-131 (the \' branch of import: `i += 4` and `u8::from_str_radix(&hex, 16)`)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 08:29:34 +00:00
aiongg 53b3144ab5 refactor(textimport): move the tests module of docx.rs to docx/tests.rs 2026-10-26 18:18:13 +00:00
NetterSilverandClaude Haiku 5.5 21acf88a80 Escape font family and typeface values in Tagged Text export
Found while testing crates/textimport/src/tagged.rs:~93-100 (tagged_text, the `<cFont:{v}>` and `<cTypeface:{v}>` writes)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 05:55:05 +00:00
Dylan M. TaylorandClaude Sonnet 5.5 8e77e8db2d refactor: split complex functions in swatch_refs, xlsx, render group drawing (Sonar S3776) (#1733)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 17:33:59 -04:00
Dylan M. Taylor 19bc14fe03 Merge pull request #1718 from storytold/sonar/s1612-b
chore: method paths for remaining closures (Sonar S1612)
2026-10-25 16:30:13 -04:00
NetterSilverandClaude Haiku 5.5 6a78e8f354 Fix spurious empty paragraph after tables in DOCX import
Found while testing crates/textimport/src/docx.rs:~272 (the "tbl" branch of Ctx::blocks: `st.insert_table(len, t); first = false;`)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 18:19:24 +00:00
Hamza 7808816e9a import: a value a file gives is one the engine can mean (#1593)
The importers each had their own spelling of "is this a number we can use", or
only checked that it was finite. Measured before the change:

    <cSize:1e300>  -> Some(1e300) pt           tagged text
    <cSize:-5>     -> Some(-5.0) pt
    <cSize:0>      -> Some(0.0) pt
    w:spacing before=1e300 -> 5e298 pt indent  DOCX

DC-12 asks for a shared clamp helper used by every importer, so there are two
in `doc::attrs`, beside the tab limits:

* `imported_type_size` drops a size the engine cannot mean, so the text keeps
  the size its style gives it -- which is what RTF and DOCX already did.
* `imported_measure` clamps an indent, spacing or offset to the widest page.
  Clamped rather than dropped on purpose: a file that says "indent everything"
  still indents as far as the engine goes, where dropping would quietly restore
  the style's own indent and hide what the file asked for.

Both are now used by tagged text, DOCX runs and paragraphs, DOCX numbering
levels and RTF -- the last two had the 0.1-1296 pt range as a literal, which is
the duplication that let tagged text drift.

Control: with the helpers back to a finite-only test, both new tests fail, the
DOCX one printing the 5e298 pt spacing in full.

Stale in the issue: its first location, `docx.rs` `w:sz`, is already clamped --
that came with #1574. DC-12's remaining holes are the ones fixed here.
2026-10-24 16:17:49 +05:00
Dylan M. TaylorandClaude Sonnet 5.5 9770002c5c chore: method paths for remaining trivial closures (Sonar S1612)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 08:32:21 +00:00
Dylan M. TaylorandClaude Sonnet 5.5 8299d26400 chore: replace trivial closures with method paths (Sonar S1612) (#1714)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 04:30:43 -04:00
Brandon Thomas 8ce126407e Merge pull request #1310 from Silvertree2010/fix/h572-docx-import-stores-out-of-range
Drop out-of-range font sizes on DOCX import
2026-10-23 17:30:10 -04:00
Brandon Thomas a77660b9fa Merge pull request #1319 from Silvertree2010/fix/h679-docx-xlsx-xml-parser-silently-dr
Keep CDATA text in the DOCX/XLSX XML parser
2026-10-23 17:30:07 -04:00
Brandon Thomas fc2e42738c Merge pull request #1397 from Silvertree2010/fix/h951-tagged-text-export-strips-from-s
Keep ':', '<', '>' and '\' in Tagged Text style names
2026-10-23 17:28:47 -04:00
Brandon Thomas a627c88e45 Merge pull request #1396 from Silvertree2010/fix/h957-rtf-import-wraps-out-of-range-un
Drop out-of-range RTF \uN values instead of wrapping them
2026-10-23 17:27:14 -04:00
Brandon Thomas e70fdb4f14 Merge pull request #1607 from Silvertree2010/fix/h1298-rtf-import-doubles-characters-an
Skip control words and symbols used as RTF \uN fallbacks
2026-10-23 17:27:11 -04:00
Brandon ThomasandClaude Opus 5.5 96a3290021 Tagged Text: decode only the escapes we write; other backslashes stay literal
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:16:26 -04:00
Brandon ThomasandClaude Opus 5.5 13d17e7a6d RTF: a group opening ends the \uN fallback skip
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:58:25 -04:00
NetterSilverandClaude Haiku 5.5 2ddcf6d2a1 Decode UTF-16 plain text files by BOM on import
Found while testing crates/textimport/src/lib.rs:117

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 20:08:56 +02:00
NetterSilverandClaude Haiku 5.5 bbb95930a2 Skip control words and symbols used as RTF \uN fallbacks
Found while testing crates/textimport/src/rtf.rs:197-243 (the \u handler at :235-242; control words never decrement `pending`)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 19:26:49 +02:00
NetterSilverandClaude Haiku 5.5 e1e9923471 Keep ':', '<', '>' and '\' in Tagged Text style names
Found while testing crates/textimport/src/tagged.rs:35 (also :52)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 18:02:07 +00:00
NetterSilverandClaude Haiku 5.5 6641e7c29f Drop out-of-range RTF \uN values instead of wrapping them
Found while testing crates/textimport/src/rtf.rs:237

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 17:44:49 +00:00
NetterSilverandClaude Haiku 5.5 177ac86741 Keep CDATA text in the DOCX/XLSX XML parser
Found while testing crates/textimport/src/docx.rs:134-135

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 17:41:15 +00:00
NetterSilverandClaude Haiku 5.5 a5ff17356d Drop out-of-range font sizes on DOCX import
Found while testing crates/textimport/src/docx.rs:183-185

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 11:56:51 +00:00
Brandon Thomas cbcf6185b7 Merge pull request #1177 from Silvertree2010/fix/h2234-xlsx-import-appends-phonetic-gui
Skip phonetic guide text when importing XLSX strings
2026-10-19 02:50:23 -04:00
Brandon Thomas 71ab5282d9 Merge pull request #1182 from Silvertree2010/fix/h2256-tagged-text-import-accepts-non-f
Ignore non-finite <cSize> values in Tagged Text import
2026-10-19 02:50:19 -04:00
Brandon Thomas 242dd4c08d Merge pull request #1187 from Silvertree2010/fix/h2354-rtf-import-stores-zero-negative
Ignore out-of-range font sizes in RTF import
2026-10-19 02:50:11 -04:00
NetterSilverandClaude Haiku 5.5 2cc21e19b2 Ignore non-finite w:sz, w:spacing and w:ind values in DOCX import
Found while testing crates/textimport/src/docx.rs:183 (run_attrs, w:sz) and :206-214 (para_attrs, w:spacing/w:ind via the `tw` closure)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 04:55:36 +02:00
NetterSilverandClaude Haiku 5.5 3823d7d942 Ignore out-of-range font sizes in RTF import
Found while testing crates/textimport/src/rtf.rs:219

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 01:22:17 +00:00
NetterSilverandClaude Haiku 5.5 61387304b6 Ignore non-finite <cSize> values in Tagged Text import
Found while testing crates/textimport/src/tagged.rs:173

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-16 23:26:08 +02:00
NetterSilverandClaude Haiku 5.5 81cc1404b8 Skip phonetic guide text when importing XLSX strings
Found while testing crates/textimport/src/xlsx.rs:62 and :112 (shared strings are read with El::text, which includes <rPh> children; inline strings use the same call, crates/textimport/src/docx.rs:37)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-16 20:22:58 +02:00
aiongg 3f9824906f Merge PR #1029: docx: an explicit off imports as off 2026-10-16 04:18:38 +00:00
aiongg 5d2c94cb57 Merge PR #985: End the RTF \uN fallback skip at the end of its group 2026-10-16 04:18:37 +00:00
aiongg 499f5f27c6 Merge PR #1001: Importers: XML element nesting and DTD group nesting are bounded (#712)
crates/engine/src/dtd.rs: keep the branch's version; #319 already bounds DTD group nesting (with the same limit of 64) along with the entity budget and matching fixes, so this PR's DTD part is dropped. The XML element limits are taken.
2026-10-16 04:03:53 +00:00
aiongg 3ea3cd97dd Merge PR #1054: Keep text of inline content controls in DOCX import
crates/textimport/src/docx.rs: keep the branch's page_break_ends_its_paragraph test (already present further down); add the PR's inline content control test.
2026-10-16 04:02:18 +00:00