mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-17 15:55:14 +00:00
* Add craft-wordmodel: WordCraft's document model without its editing code
WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.
Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.
* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs
The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.
* craft-mime: detect, for bytes that name no character set
is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.
* craft-wordmodel: keep the document operations (edit.rs) with the model
The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.
base64 0.23, as WordCraft and craft-offcrypto use.
* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath
The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.
mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.
* craft-math: read MathLive's and other spellings of LaTeX math commands
\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).
* craft-math: equaSHion's equation corpus as test data
corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (7a6ef6b9), unmodified.
* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README
The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.
* Add craft-flow: the flow model the text formats map to, on craft-wordmodel
WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.
* Add craft-markdown: Markdown read and written, on craft-flow
WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.
* WIP: craft-latex sources from WordCraft (unmodified)
* Add craft-latex: LaTeX documents read and written, on craft-flow
WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.
* craft-math: latex_text, LaTeX math and the linear format converted as text
WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.
* craft-math: latex_text's group finders are public, for craft-latex's length parser
* craft-latex: math through craft_math::latex_text; label timing test by ratio
The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.
---------
Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
229 lines
8.5 KiB
Rust
229 lines
8.5 KiB
Rust
use super::lex::{Tok, text_symbol};
|
|
use super::limits::MAX_NEST;
|
|
use super::parser::Parser;
|
|
use super::units::{parse_color, spec_widths};
|
|
use super::{Cell, FTable};
|
|
|
|
/// Plain text of tokens (arguments that are names, URLs, lengths).
|
|
pub(super) fn raw_text(toks: &[Tok]) -> String {
|
|
let mut s = String::new();
|
|
for t in toks {
|
|
match t {
|
|
Tok::Text(x) => s.push_str(x),
|
|
Tok::Tie => s.push('~'),
|
|
Tok::Cmd(c) => match text_symbol(c) {
|
|
Some(sym) => s.push_str(sym),
|
|
None if c == "\\" => s.push(' '),
|
|
None => {
|
|
s.push('\\');
|
|
s.push_str(c);
|
|
}
|
|
},
|
|
Tok::Verb(v) | Tok::Math(v, _) => s.push_str(v),
|
|
Tok::Amp => s.push('&'),
|
|
_ => {}
|
|
}
|
|
}
|
|
s.trim().to_string()
|
|
}
|
|
|
|
/// Tokens back as LaTeX source (comments and layout whitespace normalised).
|
|
pub(super) fn source_text(toks: &[Tok]) -> String {
|
|
let mut s = String::new();
|
|
for t in toks {
|
|
match t {
|
|
Tok::Text(x) => s.push_str(x),
|
|
Tok::Open => s.push('{'),
|
|
Tok::Close => s.push('}'),
|
|
Tok::Tie => s.push('~'),
|
|
Tok::Amp => s.push('&'),
|
|
Tok::Par => s.push(' '),
|
|
Tok::Cmd(c) => {
|
|
s.push('\\');
|
|
s.push_str(c);
|
|
if c.chars().all(|c| c.is_ascii_alphabetic()) {
|
|
s.push(' ');
|
|
}
|
|
}
|
|
Tok::Math(m, _) => {
|
|
s.push('$');
|
|
s.push_str(m);
|
|
s.push('$');
|
|
}
|
|
Tok::Verb(v) | Tok::Verbatim(v) => s.push_str(v),
|
|
Tok::Begin(n) => s.push_str(&format!("\\begin{{{n}}}")),
|
|
Tok::End(n) => s.push_str(&format!("\\end{{{n}}}")),
|
|
}
|
|
}
|
|
s.trim().to_string()
|
|
}
|
|
|
|
/// A `tabular` body as a table.
|
|
pub(super) fn table(toks: &[Tok], spec: &str, depth: usize) -> FTable {
|
|
let (spec_cols, widths) = spec_widths(spec);
|
|
let mut has_rules = spec.contains('|');
|
|
// Split rows at `\\` and cells at `&`, outside braces and nested environments.
|
|
let mut rows: Vec<Vec<&[Tok]>> = Vec::new();
|
|
let mut row: Vec<&[Tok]> = Vec::new();
|
|
let mut start = 0usize;
|
|
let mut brace = 0usize;
|
|
let mut env = 0usize;
|
|
for (i, t) in toks.iter().enumerate() {
|
|
match t {
|
|
Tok::Open => brace += 1,
|
|
Tok::Close => brace = brace.saturating_sub(1),
|
|
Tok::Begin(_) => env += 1,
|
|
Tok::End(_) => env = env.saturating_sub(1),
|
|
Tok::Cmd(c) if brace == 0 && env == 0 && matches!(c.as_str(), "hline" | "cline" | "toprule" | "midrule" | "bottomrule") => {
|
|
has_rules = true;
|
|
}
|
|
Tok::Amp if brace == 0 && env == 0 => {
|
|
row.push(toks.get(start..i).unwrap_or_default());
|
|
start = i + 1;
|
|
}
|
|
Tok::Cmd(c) if brace == 0 && env == 0 && (c == "\\" || c == "tabularnewline") => {
|
|
row.push(toks.get(start..i).unwrap_or_default());
|
|
rows.push(std::mem::take(&mut row));
|
|
start = i + 1;
|
|
}
|
|
_ => {}
|
|
}
|
|
if rows.len() >= craft_wordmodel::table::MAX_ROWS {
|
|
break;
|
|
}
|
|
}
|
|
if start < toks.len() {
|
|
row.push(toks.get(start..).unwrap_or_default());
|
|
}
|
|
if !row.is_empty() {
|
|
rows.push(row);
|
|
}
|
|
let mut out = FTable { rows: Vec::new(), widths: Vec::new(), borderless: !has_rules, extra: None, kept: None, wrap: None };
|
|
// Rows still covered by a `\multirow` above, per grid column.
|
|
let mut covering: Vec<u32> = Vec::new();
|
|
for r in rows {
|
|
let mut cells = Vec::new();
|
|
let mut g = 0usize;
|
|
// A document row keeps MAX_COLS cells; the rest would only widen `covering`, which is
|
|
// walked once per row.
|
|
for cell_toks in r.into_iter().take(craft_wordmodel::table::MAX_COLS) {
|
|
let mut cell = Cell::default();
|
|
let mut content = trim_cell(cell_toks);
|
|
// `\multicolumn{n}{spec}{content}`, `\multirow{n}{width}{content}`. Each wrapper
|
|
// rescans its content, so only MAX_NEST of them are unwrapped (a cell nests one or
|
|
// two); further ones stay in the content.
|
|
let mut sub = Parser::new(content, depth + 1);
|
|
let mut wrappers = 0usize;
|
|
loop {
|
|
sub.skip_spaces();
|
|
match sub.t.get(sub.pos) {
|
|
Some(Tok::Cmd(c)) if c == "multicolumn" && wrappers < MAX_NEST => {
|
|
wrappers += 1;
|
|
sub.pos += 1;
|
|
cell.colspan = sub.arg_raw().trim().parse::<u32>().unwrap_or(1).clamp(1, 63);
|
|
let _ = sub.arg_raw();
|
|
content = sub.arg_tokens();
|
|
sub = Parser::new(content, depth + 1);
|
|
}
|
|
Some(Tok::Cmd(c)) if (c == "multirow" || c == "multirow*") && wrappers < MAX_NEST => {
|
|
wrappers += 1;
|
|
sub.pos += 1;
|
|
let _ = sub.opt_arg();
|
|
cell.rowspan = sub.arg_raw().trim().parse::<i64>().map(|n| n.unsigned_abs().clamp(1, 1000) as u32).unwrap_or(1);
|
|
let _ = sub.opt_arg();
|
|
let _ = sub.arg_raw();
|
|
let _ = sub.opt_arg();
|
|
content = sub.arg_tokens();
|
|
sub = Parser::new(content, depth + 1);
|
|
}
|
|
Some(Tok::Cmd(c)) if c == "cellcolor" => {
|
|
sub.pos += 1;
|
|
let model = sub.opt_arg();
|
|
let spec = sub.arg_raw();
|
|
cell.shading = parse_color(model.as_deref(), &spec);
|
|
}
|
|
_ => break,
|
|
}
|
|
}
|
|
if covering.get(g).copied().unwrap_or(0) > 0 {
|
|
cell.covered = true;
|
|
}
|
|
if !cell.covered && depth < MAX_NEST {
|
|
sub.run(None);
|
|
let (blocks, _) = sub.finish();
|
|
cell.blocks = blocks;
|
|
}
|
|
let span = cell.colspan as usize;
|
|
if covering.len() < g + span {
|
|
covering.resize(g + span, 0);
|
|
}
|
|
for k in g..g + span {
|
|
if let Some(c) = covering.get_mut(k)
|
|
&& cell.rowspan > 1
|
|
&& !cell.covered
|
|
{
|
|
*c = cell.rowspan;
|
|
}
|
|
}
|
|
g += span;
|
|
cells.push(cell);
|
|
}
|
|
for c in covering.iter_mut() {
|
|
*c = c.saturating_sub(1);
|
|
}
|
|
// `\hline` alone after the last `\\` is not a row.
|
|
let empty = cells.iter().all(|c| !c.covered && c.blocks.is_empty());
|
|
if empty && cells.len() <= 1 {
|
|
continue;
|
|
}
|
|
out.rows.push(cells);
|
|
}
|
|
let cols = out.cols();
|
|
if !widths.is_empty() && widths.len() == cols && spec_cols == cols {
|
|
out.widths = widths;
|
|
}
|
|
out
|
|
}
|
|
|
|
/// A cell's tokens without the rules (`\hline`…) and spaces around them.
|
|
fn trim_cell(toks: &[Tok]) -> &[Tok] {
|
|
let skip = |t: &Tok| match t {
|
|
Tok::Cmd(c) => {
|
|
matches!(c.as_str(), "hline" | "toprule" | "midrule" | "bottomrule" | "endhead" | "endfirsthead" | "endfoot" | "endlastfoot" | "noalign")
|
|
}
|
|
Tok::Text(s) => s.trim().is_empty(),
|
|
_ => false,
|
|
};
|
|
let mut a = 0;
|
|
while let Some(t) = toks.get(a) {
|
|
if skip(t) {
|
|
a += 1;
|
|
} else if matches!(t, Tok::Cmd(c) if c == "cline" || c == "cmidrule" || c == "addlinespace") {
|
|
// `\cline{2-3}`, `\cmidrule(lr){1-2}`: the command and its argument.
|
|
a += 1;
|
|
while let Some(Tok::Text(s)) = toks.get(a) {
|
|
if s.trim().is_empty() || s.trim_start().starts_with('(') || s.trim_start().starts_with('[') {
|
|
a += 1;
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
if toks.get(a) == Some(&Tok::Open) {
|
|
while let Some(t) = toks.get(a) {
|
|
a += 1;
|
|
if *t == Tok::Close {
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
let mut b = toks.len();
|
|
while b > a && toks.get(b - 1).is_some_and(skip) {
|
|
b -= 1;
|
|
}
|
|
toks.get(a..b).unwrap_or_default()
|
|
}
|