Commit Graph
8 Commits
Author SHA1 Message Date
Dylan M. Taylor 52368e0016 Merge pull request #1913 from scks1/fix/latex-paragraph-breaks
craft-latex: paragraph breaks survive comments, control words and CR line ends (wordcraft#6350, #6365, #6366)
2026-11-16 19:46:09 -05:00
Dylan M. Taylor fe39ad9e27 Merge pull request #1916 from scks1/fix/latex-break-star
craft-latex: a starred line break shows no star
2026-11-16 19:42:26 -05:00
Simon Koch-Sultan dca3593452 craft-latex: skip the blanks after a control space
TeX skips the blanks after a control space, `\ ` or `\` and a tab, as it
does after `\` at a line end. The lexer left them to the whitespace branch,
which added a second space, so text taken as written, such as a reference
or an `\item` label, read `one  two`. The control-space branch now skips
them the way the line-end branch does, and a blank line after one still
ends the paragraph.
2026-11-16 14:06:20 -05:00
Simon Koch-Sultan b974484e03 craft-latex: paragraph breaks survive comments, control words and CR line ends (wordcraft#6350, #6365, #6366)
Three ways the lexer lost a blank line, so two paragraphs ran together:

- A comment took its line end with it, so a blank line right after it
  read as a single line end (wordcraft#6350).
- A control word took the spaces and one line end after it, so a blank
  line right after `\relax` at a line end read as a single line end
  (wordcraft#6366).
- Only LF was a line end. TeX ends a line at LF, CR or CRLF. With CR line
  ends every paragraph break was lost, and a comment or a verbatim
  environment ran to the end of the file and dropped the rest of it; with
  CR or CRLF a control word kept the line end after it as a space
  (wordcraft#6365, #6373, #6374, #6375). The lexer now reads every line
  end as LF.

Each symptom has its own test. The CR fix needs the control-word fix
before it: CRLF files read a blank line after a control word correctly
only because the CR stopped the control word from taking the line end.

Port of storytold/wordcraft#6387, whose files moved here with the LaTeX
reader.
2026-11-16 12:18:30 -05:00
Simon Koch-Sultan 9f73a01143 craft-latex: a backslash before a line end or a tab is a control space (wordcraft#6296)
The control symbol branch took the line end or tab after a `\` as the
symbol's name, and the parser drops unknown commands, so `mid\` at a line
end followed by `line` read as `midline`, and a blank line after it no
longer ended the paragraph. TeX reads `\` before a line end or a tab as a
control space, as for `\ `.

- lex.rs: `\` before a line end (LF or CRLF) or a tab gives `Tok::Cmd(" ")`
  and leaves the line end in place, so a following blank line still ends
  the paragraph and the line end adds no second space.
- parser.rs: an optional argument such as an `\item` label reads a
  control space as a space instead of writing `\ ` back as text.

This also keeps apart, on reopening, a paragraph that ends in two spaces:
export writes its last space as a control space at the line end.

Port of storytold/wordcraft#6298, whose files moved here with the LaTeX
reader.
2026-11-16 12:18:30 -05:00
Simon Koch-Sultan c3c19cc917 craft-latex: a starred line break shows no star
`\\*` is a line break that also forbids a page break after it. The lexer
read `\\` as the control symbol and left the `*` as text, so `a\\*b`
showed `a`, a line break, then `*b`, `\\*[3pt]` showed `*[3pt]`, and in
a tabular the next row's first cell started with `*`.

The lexer now takes the star with `\\`, so the parser and the tabular
reader see the line break they already handle, with its optional
spacing argument.
2026-11-16 11:54:29 -05:00
Dylan M. Taylor 7316e4e223 fix: clear workspace baseline gates on Rust 1.100 2026-11-13 03:13:53 +00:00
Dylan M. Taylorandshuff57 bdf06f1029 Add craft-latex: LaTeX documents read and written, on craft-flow (stacked) (#1038)
* Add craft-wordmodel: WordCraft's document model without its editing code

WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.

Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.

* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs

The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.

* craft-mime: detect, for bytes that name no character set

is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.

* craft-wordmodel: keep the document operations (edit.rs) with the model

The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.

base64 0.23, as WordCraft and craft-offcrypto use.

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (7a6ef6b9), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* Add craft-flow: the flow model the text formats map to, on craft-wordmodel

WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.

* Add craft-markdown: Markdown read and written, on craft-flow

WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.

* WIP: craft-latex sources from WordCraft (unmodified)

* Add craft-latex: LaTeX documents read and written, on craft-flow

WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.

* craft-math: latex_text, LaTeX math and the linear format converted as text

WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.

* craft-math: latex_text's group finders are public, for craft-latex's length parser

* craft-latex: math through craft_math::latex_text; label timing test by ratio

The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
2026-11-11 20:58:36 -05:00