Files
craft-libs/crates/latex/tests/hostile.rs
T
Dylan M. Taylorandshuff57 bdf06f1029 Add craft-latex: LaTeX documents read and written, on craft-flow (stacked) (#1038)
* Add craft-wordmodel: WordCraft's document model without its editing code

WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.

Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.

* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs

The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.

* craft-mime: detect, for bytes that name no character set

is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.

* craft-wordmodel: keep the document operations (edit.rs) with the model

The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.

base64 0.23, as WordCraft and craft-offcrypto use.

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (7a6ef6b9), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* Add craft-flow: the flow model the text formats map to, on craft-wordmodel

WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.

* Add craft-markdown: Markdown read and written, on craft-flow

WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.

* WIP: craft-latex sources from WordCraft (unmodified)

* Add craft-latex: LaTeX documents read and written, on craft-flow

WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.

* craft-math: latex_text, LaTeX math and the linear format converted as text

WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.

* craft-math: latex_text's group finders are public, for craft-latex's length parser

* craft-latex: math through craft_math::latex_text; label timing test by ratio

The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
2026-11-11 20:58:36 -05:00

104 lines
5.2 KiB
Rust

//! Hostile and generated input, from WordCraft's cross-format tests (`crates/formats/src/tests.rs`):
//! nesting, column specs and labels are bounded and linear, the export carries no TeX commands
//! from the document, and text survives a round trip.
use craft_latex::{export, import, linear_to_latex};
use craft_wordmodel::Document;
use std::time::{Duration, Instant};
fn text(tex: &str) -> String {
import(tex.as_bytes()).plain_text(Default::default())
}
#[test]
fn deep_nesting_is_bounded() {
assert!(text(&("{\\textbf{".repeat(50_000) + "x")).contains('x'));
assert!(text(&("\\begin{itemize}\\item ".repeat(10_000) + "deep")).contains("deep"));
let _ = import(("\\begin{tabular}{l}".repeat(2_000) + "cell").as_bytes());
let _ = import(format!("${}x{}$", "\\frac{".repeat(10_000), "}".repeat(10_000)).as_bytes());
// `*{n}{…}` column specs nested 100k deep, and repeats multiplying past what a table holds.
assert!(text(&format!("\\begin{{tabular}}{{{}l{}}}x\\end{{tabular}}", "*{1}{".repeat(100_000), "}".repeat(100_000))).contains('x'));
assert!(text(&format!("\\begin{{tabular}}{{{}p{{1cm}}{}}}x\\end{{tabular}}", "*{64}{".repeat(30), "}".repeat(30))).contains('x'));
// An equation of unmatched `(` and one nested 100k deep convert in linear time.
let started = Instant::now();
assert_eq!(linear_to_latex(&"(".repeat(200_000)).len(), 200_000);
let nested = format!("{}x{}", "(".repeat(100_000), ")".repeat(100_000));
assert_eq!(linear_to_latex(&nested).len(), 200_001);
assert!(started.elapsed() < Duration::from_secs(10), "{:?}", started.elapsed());
assert_eq!(linear_to_latex("(a)/(b)"), "\\frac{a}{b}");
assert_eq!(linear_to_latex("((a)/(b))/(c)"), "\\frac{\\frac{a}{b}}{c}");
// `\begin{` never closed.
let started = Instant::now();
let _ = import("\\begin{".repeat(300_000).as_bytes());
assert!(started.elapsed() < Duration::from_secs(10), "{:?}", started.elapsed());
}
/// What a document says in an equation or a bookmark name is not written to the exported file as
/// TeX commands: compiling the export must not read files or run programs the document names.
#[test]
fn latex_export_carries_no_commands_from_the_document() {
let tex = linear_to_latex("\\input secret.txt \\immediate\\write18\\bgroup id\\egroup \\csname x\\endcsname");
for cmd in ["\\input", "\\immediate", "\\write", "\\bgroup", "\\egroup", "\\csname", "\\endcsname"] {
assert!(!tex.contains(cmd), "{cmd} in {tex}");
}
assert!(tex.contains("\\backslash input"), "{tex}");
for linear in ["a^^5cinput secret.txt ", "a^^^^5cinput", "x^(^^5cinput)", "^^5cinput"] {
assert!(!linear_to_latex(linear).contains("^^"), "{linear}");
}
let deep = format!("{}^^5cinput x{}", "x^(".repeat(40), ")".repeat(40));
assert!(!linear_to_latex(&deep).contains("^^"));
assert_eq!(linear_to_latex("\\alpha+\\sin x+\\mathbb R"), "\\alpha+\\sin x+\\mathbb R");
assert_eq!(linear_to_latex("x^2^3"), "x^2^3");
let d = import("Text $\\input secret.txt $ and \\hypertarget{^^5cinput secret.txt ^^5c}{here}.".as_bytes());
let out = export(&d);
assert!(!out.contains("\\input") && !out.contains("^^"), "{out}");
}
/// The fastest of three reads of `src`.
fn fastest(src: &str) -> Duration {
(0..3)
.map(|_| {
let started = Instant::now();
assert!(import(src.as_bytes()).body.len() <= 2);
started.elapsed()
})
.min()
.unwrap_or_default()
}
/// Labels in a row are read in linear time: four times as many take well under the sixteen times
/// as long a quadratic read would (a ratio, so a busy machine does not fail it).
#[test]
fn latex_labels_in_a_row_are_read_in_linear_time() {
let cases: [fn(usize) -> String; 3] = [|n| "\\label{a} ".repeat(n) + "x", |n| "\\label{a}".repeat(n) + "x", |n| "\\hypertarget{a}{} ".repeat(n)];
for make in cases {
let (small, large) = (fastest(&make(25_000)), fastest(&make(100_000)));
assert!(large < small * 8 + Duration::from_millis(50), "{small:?} for 25,000, {large:?} for 100,000");
}
let d = import("\\label{one}First \\label{two}second \\label{three}third.".as_bytes());
let p = d.para_at(&craft_wordmodel::Pos::body(0, 0)).unwrap();
let names: Vec<&str> = p
.objects
.iter()
.filter_map(|o| if let craft_wordmodel::InlineObject::BookmarkStart { name } = o { Some(name.as_str()) } else { None })
.collect();
assert_eq!(names, ["one", "two", "three"]);
assert_eq!(d.plain_text(craft_wordmodel::StoryRef::Body).replace('\u{FFFC}', ""), "First second third.");
}
proptest::proptest! {
#[test]
fn fuzz_latex(s in "(\\\\(begin|end)\\{(itemize|tabular|verbatim|quote|center|x)\\}|\\\\[a-zA-Z]{1,8}\\*?|\\\\[^a-zA-Z]|[{}\\[\\]$&%~^_#]|\n\n|[a-z ]){0,150}") {
let d = import(s.as_bytes());
let _ = export(&d);
}
#[test]
fn latex_text_round_trips(words in proptest::collection::vec("[a-zA-Z0-9{}$&%#_~^\\\\<>|`'.!-]{1,8}", 1..8)) {
let text = words.join(" ");
let tex = export(&Document::from_text(&text));
let got = import(tex.as_bytes()).plain_text(Default::default());
proptest::prop_assert_eq!(got.trim(), text.trim(), "tex: {}", tex);
}
}