From fdbc3322b31c955612bf88de797dbce4e44c65ff Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 13:24:09 +0000 Subject: [PATCH] ODT: unedited tables, pages, sections and properties written as read; no-edit loss gate More of what WordCraft doesn't change is written back as it was read: tables; the document properties it doesn't model (template, statistics, editing time, printing); the declarations opening the text and the document's scripts; page layouts and master pages while the page setup is unchanged, each section starting with the master page it started with; named sections around unedited paragraphs. Package files and folders kept markup names (a style's fill picture, a header's embedded object) are written with it, with their manifest entries. Files are saved as ODF 1.3; markup kept from an older file is brought to 1.3's meaning (a gradient's or hatch's unitless angle was tenths of a degree). WordCraft's own drawings state style:flow-with-text, which the standard leaves open. odt::tests_more::lossless saves seven samples of our own without edits and fails on anything lost (nothing is allowed), and checks that edits keep the saved parts well-formed. Over the 133-file corpus the round-trip difference falls from 191k to 51k pixels; 95 files come back pixel-identical (62 before). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01GBVQLyDnrG8Tn1yDQcUabC --- ROADMAP.md | 1 + crates/doc/src/kept.rs | 113 +++++- crates/formats/src/html/builder.rs | 2 +- crates/formats/src/latex/table.rs | 2 +- crates/formats/src/latex/tests.rs | 4 +- crates/formats/src/model.rs | 2 + crates/formats/src/model/from_doc.rs | 9 +- crates/formats/src/model/to_doc.rs | 24 +- crates/formats/src/model/types.rs | 6 + crates/formats/src/odt/body.rs | 101 +++++ crates/formats/src/odt/body/tables.rs | 2 +- crates/formats/src/odt/import.rs | 22 +- crates/formats/src/odt/kept.rs | 351 ++++++++++++++---- crates/formats/src/odt/package.rs | 77 +++- crates/formats/src/odt/tests.rs | 4 +- crates/formats/src/odt/tests_more.rs | 1 + .../src/odt/tests_more/cell_formulas.rs | 10 +- crates/formats/src/odt/tests_more/fields.rs | 2 +- .../formats/src/odt/tests_more/flow_breaks.rs | 14 +- crates/formats/src/odt/tests_more/headers.rs | 2 +- crates/formats/src/odt/tests_more/kept.rs | 5 + crates/formats/src/odt/tests_more/lossless.rs | 275 ++++++++++++++ crates/formats/src/odt/tests_more/objects.rs | 11 +- crates/formats/src/odt/writer.rs | 73 +++- crates/formats/src/odt/writer/tables.rs | 9 + crates/formats/src/rtf/reader.rs | 2 +- docs/file-format-parity.md | 3 +- docs/odf-interpretation.md | 26 +- 28 files changed, 1021 insertions(+), 132 deletions(-) create mode 100644 crates/formats/src/odt/tests_more/lossless.rs diff --git a/ROADMAP.md b/ROADMAP.md index 6a85352735..99b72433f3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -140,6 +140,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm ## Progress log +- 2026-11-03: ODT saves without edits lose (almost) nothing (gap 19): tables, page layouts and master pages, named sections, document properties and declarations WordCraft doesn't model are written back as read, and a test saves seven samples of our own and finds nothing missing. Over the 133-file corpus 95 files come back pixel-identical (62 before) and the round-trip difference falls to 51k pixels (191k) - 2026-11-03: ODT saves what WordCraft doesn't model (gap 19): a paragraph nobody edited is written back with its own markup, and while no style changed, the file's own style definitions are written, so unknown elements, attributes, unused list styles and note settings survive a save that doesn't touch them. Over the 133-file corpus (LibreOffice, Google Docs and ONLYOFFICE files) the round-trip difference falls by 64% and 62 files come back pixel-identical (46 before) - 2026-11-03: ODT checked the other way round too (gap 19): 42 documents made in WordCraft (its templates, themes, and one per feature) are saved as ODT and opened in LibreOffice; every one opens with the same page count and passes the ODF 1.3 schema. Found and fixed: table styles were left out of the saved file (now written into each cell), ODT tables were given Word's Table Grid style on opening (which changed their spacing), section columns now say whether they end balanced, and SmartArt diagrams are saved as a group of their shapes and text. The LibreOffice round-trip corpus is at its best yet (40k differing pixels over 125 documents, 42 identical). - 2026-11-02: ODT files follow their standard wherever it says something, and a setting wherever it doesn't (gap 19). File › Options › Advanced › OpenDocument files (and `file.options` `odf`) chooses WordCraft's own behaviour (the default), LibreOffice's, or custom values for each of twelve cases the standard leaves open: header heights and spacing, column gaps, shape colours and hairlines, where tab stops are measured from, frame relations the standard doesn't define, and lines of only pictures. A page layout that states no margins follows the setting too: 0 for WordCraft (XSL's initial value), 2 cm for LibreOffice. Every case is listed in docs/odf-interpretation.md; Google Docs and ONLYOFFICE files checked, and neither needs a preset of its own yet. diff --git a/crates/doc/src/kept.rs b/crates/doc/src/kept.rs index 07739927fc..cae60f2c2d 100644 --- a/crates/doc/src/kept.rs +++ b/crates/doc/src/kept.rs @@ -27,6 +27,16 @@ pub struct KeptXml { /// Paragraph [`fingerprint`] → the paragraph's markup, in the order read (identical /// paragraphs keep the first's). pub paras: HashMap>, + /// The same for the body's tables ([`table_fingerprint`]). + pub tables: HashMap>, + /// While reading: the markup of the body's `n`th table, until [`Kept::seal`]. + pub pending_tables: Vec<(usize, Arc)>, + /// Wrappers kept as read around paragraphs (sections): for a paragraph's fingerprint, the + /// start tags opening before it and how many close after it. Only paragraphs that were the + /// only one of their kind have them (a copy can't open a section twice). + pub wraps: HashMap>, u32)>, + /// While reading: the same by paragraph `rev`, until [`Kept::seal`]. + pub pending_wraps: HashMap>, u32)>, /// Named definitions the kept markup refers to (styles, font faces…), by a key the format /// chooses (`"auto:P12"`, `"style:Heading_20_1"`), as read. pub defs: BTreeMap>, @@ -42,6 +52,12 @@ pub struct KeptXml { /// The document's style definitions' [`styles_fingerprint`] when read: while it is the /// same, the file's own style definitions can be written as they were. pub styles_fp: Option, + /// The document's page setup's [`pages_fingerprint`] when read: while it is the same, the + /// file's own page layouts and master pages can be written as they were. + pub pages_fp: Option, + /// Each section's master page, as the file named it where its section starts (`None`: the + /// file's default), for a writer naming the file's own master pages. + pub section_masters: Vec>, } impl PartialEq for Kept { @@ -92,6 +108,35 @@ impl KeptXml { } self.paras.get(&fingerprint(p)) } + /// The markup kept for table `t`, while it still is what was read. + pub fn table(&self, t: &crate::Table) -> Option<&Arc> { + if self.tables.is_empty() { + return None; + } + self.tables.get(&table_fingerprint(t)) + } + /// The wrappers kept around paragraph `p`, while it still is what was read. + pub fn wrap(&self, p: &Paragraph) -> Option<&(Vec>, u32)> { + if self.wraps.is_empty() { + return None; + } + self.wraps.get(&fingerprint(p)) + } + /// Keep wrapper markup for the paragraph whose `rev` is `rev`, within the budget. + pub fn keep_pending_wrap(&mut self, rev: u64, wrap: (Vec>, u32)) { + let n: usize = wrap.0.iter().map(|t| t.len()).sum(); + if self.bytes.saturating_add(n) <= MAX_KEPT_BYTES { + self.bytes += n; + self.pending_wraps.insert(rev, wrap); + } + } + /// Keep `xml` for the body's `n`th table, within the budget. + pub fn keep_pending_table(&mut self, n: usize, xml: Arc) { + if self.bytes.saturating_add(xml.len()) <= MAX_KEPT_BYTES { + self.bytes += xml.len(); + self.pending_tables.push((n, xml)); + } + } /// Keep `xml` for the paragraph whose `rev` is `rev`, within the budget. pub fn keep_pending(&mut self, rev: u64, xml: Arc) { if self.bytes.saturating_add(xml.len()) <= MAX_KEPT_BYTES { @@ -114,6 +159,10 @@ impl KeptXml { self.files.insert(path, Arc::new(bytes)); } } + /// Whether `doc`'s page setup is still as it was read. + pub fn pages_as_read(&self, doc: &crate::Document) -> bool { + self.pages_fp.is_some_and(|fp| fp == pages_fingerprint(doc)) + } /// Whether `doc`'s style definitions are still as they were read. pub fn styles_as_read(&self, doc: &crate::Document) -> bool { self.styles_fp.is_some_and(|fp| fp == styles_fingerprint(doc)) @@ -130,6 +179,40 @@ pub fn styles_fingerprint(doc: &crate::Document) -> u64 { h.finish() } +/// What `doc`'s page setup is: each section's setup (where it starts, its size, margins, +/// columns, headers and footers), the headers' and footers' content, the page colour and +/// picture, and the notes' numbering. +pub fn pages_fingerprint(doc: &crate::Document) -> u64 { + let mut sections: Vec<&crate::SectionProps> = doc.body.iter().filter_map(|b| b.as_para()).filter_map(|p| p.section.as_deref()).collect(); + sections.push(&doc.last_section); + let parts: Vec<(&u32, &crate::Part)> = + doc.parts.iter().filter(|(_, p)| matches!(p.kind, crate::PartKind::Header | crate::PartKind::Footer)).collect(); + let s = &doc.settings; + let mut h = std::collections::hash_map::DefaultHasher::new(); + match serde_json::to_vec(&( + sections, + parts, + &s.page_color, + &s.page_picture, + (&s.footnote_format, &s.endnote_format, &s.footnote_pr, &s.endnote_pr), + (&doc.footnote_separators, &doc.endnote_separators), + )) { + Ok(v) => v.hash(&mut h), + Err(_) => u64::MAX.hash(&mut h), + } + h.finish() +} + +/// What table `t` is in the model. +pub fn table_fingerprint(t: &crate::Table) -> u64 { + let mut h = std::collections::hash_map::DefaultHasher::new(); + match serde_json::to_vec(t) { + Ok(v) => v.hash(&mut h), + Err(_) => u64::MAX.hash(&mut h), + } + h.finish() +} + /// What paragraph `p` is in the model (its text, runs, objects and properties, not its `rev`): /// equal for a paragraph nobody changed. pub fn fingerprint(p: &Paragraph) -> u64 { @@ -165,7 +248,17 @@ impl crate::Document { /// After reading: the markup kept for paragraphs still as the reader made them is filed /// under their fingerprints (see [`Kept::seal`]); the rest is dropped. pub fn seal_kept(&mut self) { - if self.kept.get().is_none_or(|k| k.pending.is_empty()) { + // The body's tables kept, by their place among its tables. + if let Some(k) = self.kept.get_mut() { + let pending = std::mem::take(&mut k.pending_tables); + let tables: Vec<&crate::Table> = self.body.iter().filter_map(|b| b.as_table()).collect(); + for (n, xml) in pending { + if let Some(t) = tables.get(n) { + k.tables.entry(table_fingerprint(t)).or_insert(xml); + } + } + } + if self.kept.get().is_none_or(|k| k.pending.is_empty() && k.pending_wraps.is_empty()) { return; } let mut all = Vec::new(); @@ -174,6 +267,24 @@ impl crate::Document { paras(&p.blocks, 0, &mut all); } let mut kept = std::mem::take(&mut self.kept); + // Wrappers, for paragraphs that are the only one of their kind. + if let Some(k) = kept.get_mut() + && !k.pending_wraps.is_empty() + { + let mut count: HashMap = HashMap::new(); + let fps: Vec<(u64, u64)> = all.iter().map(|p| (p.rev, fingerprint(p))).collect(); + for (_, fp) in &fps { + *count.entry(*fp).or_default() += 1; + } + for (rev, fp) in fps { + if let Some(w) = k.pending_wraps.remove(&rev) + && count.get(&fp) == Some(&1) + { + k.wraps.insert(fp, w); + } + } + k.pending_wraps.clear(); + } kept.seal(&mut all.into_iter()); self.kept = kept; } diff --git a/crates/formats/src/html/builder.rs b/crates/formats/src/html/builder.rs index 0c8d090960..b0a39088df 100644 --- a/crates/formats/src/html/builder.rs +++ b/crates/formats/src/html/builder.rs @@ -479,7 +479,7 @@ impl<'r> Builder<'r> { // Keep at most MAX_ROWS rows (the model drops the rest anyway): bounds the // O(rows × rowspan) work of `insert_covered` on a table of millions of ``. let rows: Vec<_> = t.rows.into_iter().filter(|r| !r.is_empty()).take(wordcraft_doc::table::MAX_ROWS).collect(); - let mut ft = FTable { rows, widths: Vec::new(), borderless, extra: None }; + let mut ft = FTable { rows, widths: Vec::new(), borderless, extra: None, kept: None }; if !ft.rows.is_empty() { ft.insert_covered(); self.container().push(FBlock::Table(ft)); diff --git a/crates/formats/src/latex/table.rs b/crates/formats/src/latex/table.rs index 30380c3130..3f145ec1d9 100644 --- a/crates/formats/src/latex/table.rs +++ b/crates/formats/src/latex/table.rs @@ -98,7 +98,7 @@ pub(super) fn table(toks: &[Tok], spec: &str, depth: usize) -> FTable { if !row.is_empty() { rows.push(row); } - let mut out = FTable { rows: Vec::new(), widths: Vec::new(), borderless: !has_rules, extra: None }; + let mut out = FTable { rows: Vec::new(), widths: Vec::new(), borderless: !has_rules, extra: None, kept: None }; // Rows still covered by a `\multirow` above, per grid column. let mut covering: Vec = Vec::new(); for r in rows { diff --git a/crates/formats/src/latex/tests.rs b/crates/formats/src/latex/tests.rs index be59be3649..0c55dfa7ca 100644 --- a/crates/formats/src/latex/tests.rs +++ b/crates/formats/src/latex/tests.rs @@ -324,7 +324,7 @@ fn special_characters_round_trip() { #[test] fn exported_tables_size_their_columns() { - let mut t = FTable { rows: Vec::new(), widths: vec![100.0, 200.0], borderless: false, extra: None }; + let mut t = FTable { rows: Vec::new(), widths: vec![100.0, 200.0], borderless: false, extra: None, kept: None }; let cell = |s: &str| Cell { blocks: vec![FBlock::Para({ let mut p = Para::default(); @@ -359,7 +359,7 @@ fn urls_preserve_tildes() { #[test] fn exported_borderless_tables_omit_rules() { - let mut t = FTable { rows: Vec::new(), widths: vec![100.0, 200.0], borderless: true, extra: None }; + let mut t = FTable { rows: Vec::new(), widths: vec![100.0, 200.0], borderless: true, extra: None, kept: None }; let cell = |s: &str| Cell { blocks: vec![FBlock::Para({ let mut p = Para::default(); diff --git a/crates/formats/src/model.rs b/crates/formats/src/model.rs index 5ecd528855..ce955cfc7b 100644 --- a/crates/formats/src/model.rs +++ b/crates/formats/src/model.rs @@ -62,6 +62,7 @@ mod tests { widths: Vec::new(), borderless: false, extra: None, + kept: None, }; t.insert_covered(); assert!(t.rows.iter().all(|r| r.len() <= 64), "a row grew past the column cap"); @@ -113,6 +114,7 @@ mod tests { widths: vec![], borderless: false, extra: None, + kept: None, }; t.insert_covered(); assert_eq!(t.rows[1].len(), 2); diff --git a/crates/formats/src/model/from_doc.rs b/crates/formats/src/model/from_doc.rs index 989c044e33..4117983571 100644 --- a/crates/formats/src/model/from_doc.rs +++ b/crates/formats/src/model/from_doc.rs @@ -462,6 +462,7 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, opts: FlowOpts) -> Vec { extra: None, mark: None, kept: None, + wrap: None, }; if opts.paras { out.extra = para_extra(doc, &p.props); @@ -501,6 +502,7 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, opts: FlowOpts) -> Vec { extra: None, mark: None, kept: None, + wrap: None, }; let prev = std::mem::replace(&mut out, next); if prev.inlines.is_empty() { @@ -620,6 +622,10 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, opts: FlowOpts) -> Vec { if opts.paras && done.is_empty() && p.props.page_break_before != Some(true) { out.kept = doc.kept.of("odt").and_then(|k| k.para(p)).cloned(); } + // The sections kept as read that it opens or closes. + if opts.paras { + out.wrap = doc.kept.of("odt").and_then(|k| k.wrap(p)).map(|w| Box::new(w.clone())); + } done.push(out); done } @@ -1057,7 +1063,8 @@ fn flow_table(doc: &Document, t: &Table, depth: usize, tracked: FlowOpts) -> FTa }); // Kept whole for formats that ask for it: its header rows count even without the rest. let extra = extra.map(Box::new); - FTable { rows, widths: t.grid.clone(), borderless, extra } + let kept = (tracked.paras && depth == 0).then(|| doc.kept.of("odt").and_then(|k| k.table(t)).cloned()).flatten(); + FTable { rows, widths: t.grid.clone(), borderless, extra, kept } } /// The document body as a flow. diff --git a/crates/formats/src/model/to_doc.rs b/crates/formats/src/model/to_doc.rs index 8be9a84891..84c5369bad 100644 --- a/crates/formats/src/model/to_doc.rs +++ b/crates/formats/src/model/to_doc.rs @@ -210,6 +210,8 @@ struct Builder<'a> { /// boxes), linked once all are made ([`Builder::link_boxes`]). boxes: std::collections::HashMap, chains: Vec<(u32, String)>, + /// Tables made at the top of the flow so far (see `wordcraft_doc::kept`). + top_tables: usize, } fn rev_key(kind: RevisionKind) -> u8 { @@ -241,6 +243,7 @@ impl<'a> Builder<'a> { ended: Default::default(), boxes: Default::default(), chains: Vec::new(), + top_tables: 0, } } @@ -622,12 +625,17 @@ impl<'a> Builder<'a> { out.touch(); // Its markup kept for the writer, as long as nothing changes the paragraph from here on // (see `wordcraft_doc::kept`: the reader seals it once the document is done). - if let Some(xml) = &p.kept { + if p.kept.is_some() || p.wrap.is_some() { if self.doc.kept.get().is_none() { self.doc.kept = wordcraft_doc::kept::Kept::new(Default::default()); } if let Some(k) = self.doc.kept.get_mut() { - k.keep_pending(out.rev, xml.clone()); + if let Some(xml) = &p.kept { + k.keep_pending(out.rev, xml.clone()); + } + if let Some(w) = &p.wrap { + k.keep_pending_wrap(out.rev, (**w).clone()); + } } } out @@ -850,6 +858,18 @@ impl<'a> Builder<'a> { } tb.rows.push(row); } + // Its markup kept for the writer, by its place among the top's tables. + if depth == 0 { + if let Some(xml) = &t.kept { + if self.doc.kept.get().is_none() { + self.doc.kept = wordcraft_doc::kept::Kept::new(Default::default()); + } + if let Some(k) = self.doc.kept.get_mut() { + k.keep_pending_table(self.top_tables, xml.clone()); + } + } + self.top_tables += 1; + } Some(tb) } } diff --git a/crates/formats/src/model/types.rs b/crates/formats/src/model/types.rs index 4f53fca825..0b25014f08 100644 --- a/crates/formats/src/model/types.rs +++ b/crates/formats/src/model/types.rs @@ -670,6 +670,9 @@ pub struct Para { /// The paragraph's markup in the file it came from, while it still is what was read /// ([`wordcraft_doc::kept`]): a format that keeps it (ODT) writes it as it was. pub kept: Option>, + /// Sections kept as read around paragraphs ([`wordcraft_doc::kept`]): the start tags of + /// those opening before this paragraph (outermost first), and how many close after it. + pub wrap: Option>, u32)>>, } impl Para { @@ -870,6 +873,9 @@ pub struct FTable { pub borderless: bool, /// Its alignment, width and row heights, for formats that keep them (ODT). pub extra: Option>, + /// The table's markup in the file it came from, while it still is what was read + /// ([`wordcraft_doc::kept`]): a format that keeps it (ODT) writes it as it was. + pub kept: Option>, } /// A table's placement and row heights ([`FTable::extra`]). diff --git a/crates/formats/src/odt/body.rs b/crates/formats/src/odt/body.rs index b915845fc7..7c1da2a8f9 100644 --- a/crates/formats/src/odt/body.rs +++ b/crates/formats/src/odt/body.rs @@ -90,6 +90,15 @@ pub(super) struct Body<'a> { pub(super) raw: &'a [u8], /// The paragraph whose markup is being kept (see [`KeptOpen`]). kept_open: Option, + /// The body table whose markup is being kept (its `done`: the container's length before it). + kept_table: Option, + /// Body sections whose wrapper is kept as read: start tag, depth, the body's length before + /// it; `None` for one read as a columned section (kept as a section break). + kept_sections: Vec, usize, usize)>>, + /// Each section's master page, as the paragraph or table starting it named it (`None`: as + /// the file's default, or a columned section), in order; and the one just named. + pub(super) section_masters: Vec>, + pending_master: Option, /// Paragraphs finished so far (a kept paragraph must come out of its element whole). paras_done: usize, /// `(width, height, picture bytes, alt text, equation read from an embedded `draw:object`)`. @@ -210,6 +219,10 @@ impl<'a> Body<'a> { depth: 0, raw: &[], kept_open: None, + kept_table: None, + kept_sections: Vec::new(), + section_masters: vec![None], + pending_master: None, paras_done: 0, frame: None, in_desc: false, @@ -287,6 +300,86 @@ impl<'a> Body<'a> { /// whole and refers to nothing outside itself keeps its markup (`Para::kept`), so a save /// that doesn't change it writes it as it was (see `wordcraft_doc::kept`). pub(super) fn feed_at(&mut self, ev: Ev, at: (usize, usize)) { + // A body section: its start tag kept, put around its paragraphs at its end. + match &ev { + Ev::Start(q, _) if q == "text:section" && self.kept_sections.len() < 64 => { + let body = !self.raw.is_empty() + && self.in_text + && self.skip == 0 + && self.containers.len() == 1 + && self.tables.is_empty() + && self.frame.is_none() + && self.note.is_none() + && self.del.is_none() + && self.lists.is_empty(); + let tag = self.raw.get(at.0..at.1).and_then(|b| std::str::from_utf8(b).ok()).filter(|t| !t.ends_with("/>")).map(std::sync::Arc::from); + let depth = self.depth + 1; + let cols = self.cols_at; + self.feed(ev); + // Read as a columned section (a section break): no wrapper besides. + let columned = self.cols_at != cols; + let len = self.container().len(); + self.kept_sections.push(tag.filter(|_| body && !columned).map(|t| (t, depth, len))); + return; + } + Ev::End(q) if q == "text:section" && !self.kept_sections.is_empty() => { + let depth = self.depth; + self.feed(ev); + if let Some(Some((tag, _, from))) = self.kept_sections.pop().filter(|s| s.as_ref().is_none_or(|s| s.1 == depth)) + && self.containers.len() == 1 + { + let blocks = self.container(); + let plain = |b: Option<&FBlock>| matches!(b, Some(FBlock::Para(p)) if p.list.is_none()); + if blocks.len() > from && plain(blocks.get(from)) && plain(blocks.last()) { + if let Some(FBlock::Para(p)) = blocks.get_mut(from) { + p.wrap.get_or_insert_with(Default::default).0.insert(0, tag); + } + if let Some(FBlock::Para(p)) = blocks.last_mut() { + let w = p.wrap.get_or_insert_with(Default::default); + w.1 = w.1.saturating_add(1); + } + } + } + return; + } + _ => {} + } + // A body table, kept whole as the paragraphs are. + match &ev { + Ev::Start(q, _) => match &mut self.kept_table { + Some(k) => k.clean &= keepable(q), + None if q == "table:table" + && self.tables.is_empty() + && !self.raw.is_empty() + && self.in_text + && self.skip == 0 + && self.note.is_none() + && self.frame.is_none() + && self.del.is_none() + && self.field.is_none() + && self.loose.is_empty() => + { + // Before it starts: what ends the paragraph open before it. + self.end_para(); + self.kept_table = Some(KeptOpen { from: at.0, depth: self.depth + 1, clean: true, done: self.container().len() }); + } + None => {} + }, + Ev::End(q) if q == "table:table" && self.kept_table.as_ref().is_some_and(|k| k.depth == self.depth) => { + let k = self.kept_table.take(); + self.feed(ev); + if let Some(k) = k + && k.clean + && let Some(xml) = self.raw.get(k.from..at.1).and_then(|b| std::str::from_utf8(b).ok()) + && self.container().len() == k.done + 1 + && let Some(FBlock::Table(t)) = self.container().last_mut() + { + t.kept = Some(std::sync::Arc::from(xml)); + } + return; + } + _ => {} + } match &ev { Ev::Start(q, _) => match &mut self.kept_open { Some(k) => k.clean &= keepable(q), @@ -474,6 +567,12 @@ impl Body<'_> { } let Some(body) = self.containers.first_mut() else { return false }; if body.len() <= self.sec_from { + // The open section, empty, takes its place: its master page too. + if let Some(m) = self.pending_master.take() + && let Some(last) = self.section_masters.last_mut() + { + *last = Some(m); + } let restart = self.sec.page_num_start; self.sec = next; self.sec.page_num_start = self.sec.page_num_start.or(restart); @@ -491,6 +590,7 @@ impl Body<'_> { } self.sec_from = body.len(); self.sections += 1; + self.section_masters.push(self.pending_master.take()); true } @@ -507,6 +607,7 @@ impl Body<'_> { } let (master, number) = self.styles.page_start(name); let restart = number.flatten(); + self.pending_master = master.map(str::to_string); let page = master.and_then(|m| self.pages.masters.get(m))?.clone(); self.page = page.clone(); Some(SectionProps { start: SectionStart::NextPage, page_num_start: restart, ..page }) diff --git a/crates/formats/src/odt/body/tables.rs b/crates/formats/src/odt/body/tables.rs index 894bd45760..37201a4d58 100644 --- a/crates/formats/src/odt/body/tables.rs +++ b/crates/formats/src/odt/body/tables.rs @@ -88,6 +88,6 @@ impl TableB { let borderless = stated && rows.iter().flatten().all(|c| !c.borders.is_some_and(|b| b.any_visible())); let extra = self.extra; let kept = extra.fixed || extra.align.is_some() || extra.width_pct.is_some() || extra.heights.iter().any(Option::is_some); - FTable { rows, widths, borderless, extra: kept.then(|| Box::new(extra)) } + FTable { rows, widths, borderless, extra: kept.then(|| Box::new(extra)), kept: None } } } diff --git a/crates/formats/src/odt/import.rs b/crates/formats/src/odt/import.rs index b1ca4266aa..bff43f4a57 100644 --- a/crates/formats/src/odt/import.rs +++ b/crates/formats/src/odt/import.rs @@ -435,6 +435,8 @@ struct Parsed { kept_defs: Vec<(String, std::sync::Arc)>, /// The package files those definitions refer to. kept_files: Vec<(String, Vec)>, + /// Each section's master page as the file named it. + section_masters: Vec>, /// The space between paragraphs is their spacings' sum (else the larger). sum_spacing: bool, /// The last section's properties. @@ -593,6 +595,7 @@ fn parse_tracked(bytes: &[u8], paper: (f32, f32), profile: &OdfProfile) -> Resul // The file's own definitions: what kept paragraphs refer to, and the style definitions // written as read while they stay unchanged. let mut kept_defs = super::kept::defs(&content, true); + kept_defs.extend(meta_xml.as_deref().map(super::kept::meta_defs).unwrap_or_default()); kept_defs.extend(styles_xml.as_deref().map(|x| super::kept::defs(x, false)).unwrap_or_default()); let hidden_controls = body.take_hidden_controls(); let (mut named_odt, mut named) = custom_styles(&mut blocks, &styles, Vec::new()); @@ -623,7 +626,17 @@ fn parse_tracked(bytes: &[u8], paper: (f32, f32), profile: &OdfProfile) -> Resul let hyphenate = hyphenation(&mut blocks); let track = body.track; let sources = std::mem::take(&mut body.sources); - let kept_files = super::kept::files(&kept_defs, &mut fetch); + let section_masters = std::mem::take(&mut body.section_masters); + let kept_files = super::kept::files(&kept_defs, &entries, &mut fetch); + // Their manifest entries, and their folders' (an object's says what it is). + for (path, _) in &kept_files { + let dir = path.rsplit_once('/').map(|(d, _)| format!("{d}/")); + for p in std::iter::once(path.clone()).chain(dir) { + if let Some(mime) = manifest.get(&p).or_else(|| manifest.get(p.trim_end_matches('/'))) { + kept_defs.push((format!("manifest:{p}"), std::sync::Arc::from(mime.as_str()))); + } + } + } Ok(Parsed { flow: Flow { blocks, meta }, last, @@ -656,6 +669,7 @@ fn parse_tracked(bytes: &[u8], paper: (f32, f32), profile: &OdfProfile) -> Resul changes, kept_defs, kept_files, + section_masters, }) } @@ -706,6 +720,7 @@ pub fn import_with(bytes: &[u8], page: &SectionProps, profile: &OdfProfile) -> R changes, kept_defs, kept_files, + section_masters, } = parse_tracked(bytes, (page.page_w, page.page_h), profile)?; let mut doc = model::to_doc(&flow); kept.store(&mut doc); @@ -781,11 +796,12 @@ pub fn import_with(bytes: &[u8], page: &SectionProps, profile: &OdfProfile) -> R for (path, bytes) in kept_files { k.keep_file(path, bytes); } + k.section_masters = section_masters; } doc.seal_kept(); - let fp = wordcraft_doc::kept::styles_fingerprint(&doc); + let (fp, pages) = (wordcraft_doc::kept::styles_fingerprint(&doc), wordcraft_doc::kept::pages_fingerprint(&doc)); if let Some(k) = doc.kept.get_mut() { - k.styles_fp = Some(fp); + (k.styles_fp, k.pages_fp) = (Some(fp), Some(pages)); } Ok(doc) } diff --git a/crates/formats/src/odt/kept.rs b/crates/formats/src/odt/kept.rs index 8cbf93e65a..2485f2ae11 100644 --- a/crates/formats/src/odt/kept.rs +++ b/crates/formats/src/odt/kept.rs @@ -15,20 +15,26 @@ use super::xml::{Ev, events_at, get}; /// Most definitions kept from one part (hostile input). const MAX_DEFS: usize = 100_000; -/// The definitions in `xml` (a `content.xml` or `styles.xml`): each child of -/// `office:automatic-styles` as `auto:` (only for `content.xml`, `content` true), of -/// `office:styles` as `common:`, of `office:font-face-decls` as `font:`; and the -/// namespaces its root declares, as `ns:` (the URI). +/// The definitions in `xml` (a `content.xml` or `styles.xml`), each by the key it is kept +/// under: a child of `office:automatic-styles` as `auto:` in `content.xml` (`content` +/// true), `sauto:` in `styles.xml`; of `office:styles` as `common:`; of +/// `office:font-face-decls` as `font:`; of `office:master-styles` as `master:`; the +/// namespaces the root declares as `ns:` (the URI); and in `content.xml`, the document's +/// `office:scripts` as `top:office:scripts` and the declarations opening the text +/// (`text:variable-decls`…) as `decls:`. pub(super) fn defs(xml: &[u8], content: bool) -> Vec<(String, Arc)> { let mut out = Vec::new(); - let mut depth = 0usize; - let mut group: Option<&'static str> = None; - let mut open: Option<(String, usize)> = None; + let mut path: Vec = Vec::new(); + let mut open: Option<(String, usize, usize)> = None; + let mut before_1_3 = false; events_at(xml, |ev, at| match ev { Ev::Start(q, a) => { - depth += 1; + path.push(q.clone()); + let depth = path.len(); // The namespaces the root declares, which kept markup may use. if depth == 1 { + // ODF 1.2 and before measured some angles in tenths of a degree (see `to_1_3`). + before_1_3 = get(&a, "office:version").is_none_or(|v| v.trim() < "1.3"); for (k, v) in &a { if let Some(p) = k.strip_prefix("xmlns:").filter(|p| !p.is_empty() && p.len() <= 64) && out.len() < MAX_DEFS @@ -36,67 +42,112 @@ pub(super) fn defs(xml: &[u8], content: bool) -> Vec<(String, Arc)> { out.push((format!("ns:{p}"), Arc::from(v.as_str()))); } } - } else if depth == 2 { - group = match q.as_str() { - "office:automatic-styles" if content => Some("auto"), - "office:styles" => Some("common"), - "office:font-face-decls" => Some("font"), - _ => None, - }; - } else if depth == 3 - && let Some(g) = group - && let Some(key) = child_key(g, &q, &a) - && out.len() < MAX_DEFS - { - open = Some((key, at.0)); + return; + } + if open.is_some() || out.len() >= MAX_DEFS { + return; + } + let parent = path.get(depth.wrapping_sub(2)).map(String::as_str).unwrap_or(""); + let key = match (depth, parent) { + (2, _) if content && q == "office:scripts" => Some("top:office:scripts".to_string()), + (3, "office:automatic-styles") => child_key(if content { "auto" } else { "sauto" }, &q, &a), + (3, "office:styles") => child_key("common", &q, &a), + (3, "office:font-face-decls") => child_key("font", &q, &a), + (3, "office:master-styles") => child_key("master", &q, &a), + (4, "office:text") if content && q.starts_with("text:") && q.ends_with("-decls") => Some(format!("decls:{q}")), + _ => None, + }; + if let Some(key) = key { + open = Some((key, at.0, depth)); } } Ev::End(_) => { - if depth == 3 - && let Some((key, from)) = open.take() - && let Some(s) = xml.get(from..at.1).and_then(|b| std::str::from_utf8(b).ok()) + if let Some((_, from, d)) = &open + && *d == path.len() { - out.push((key, Arc::from(s))); + let from = *from; + if let Some((key, ..)) = open.take() + && let Some(s) = xml.get(from..at.1).and_then(|b| std::str::from_utf8(b).ok()) + { + let s = if before_1_3 { to_1_3(s) } else { s.to_string() }; + out.push((key, Arc::from(s))); + } } - if depth == 2 { - group = None; - } - depth = depth.saturating_sub(1); + path.pop(); } Ev::Text(_) => {} }); out } +/// Markup kept from an ODF 1.2 (or older) file, written as ODF 1.3: a gradient's, +/// transparency gradient's or hatch's angle without a unit was tenths of a degree there and is +/// degrees in 1.3 (§19.112 `draw:angle`, §19.209 `draw:rotation`), so it is given its unit. +fn to_1_3(xml: &str) -> String { + if !(xml.contains("').map_or(tail.len(), |e| e + 1); + let (tag, after) = tail.split_at(end); + if tag.starts_with("().ok()).filter(|v| v.is_finite()) + { + t.replace_range(a..a + len, &format!("{}deg", v / 10.0)); + } + } + out.push_str(&t); + } else { + out.push_str(tag); + } + rest = after; + } + out.push_str(rest); + out +} + /// The key a definition (element `q` with attributes `a`, a child of group `g`) is kept under: -/// `:` (its `style:name`, or a drawing definition's `draw:name`); an unnamed child of `office:styles` (a default style, the outline style, a note -/// or line numbering configuration) as `~~`. +/// `:` (its `style:name`, or a drawing definition's `draw:name`); an unnamed one (a +/// default style, the outline style, a note or line numbering configuration, a layer set) as +/// `~~`. fn child_key(g: &str, q: &str, a: &super::xml::Attrs) -> Option { // Drawing definitions (dashes, gradients, arrowheads…) are named by `draw:name`. match get(a, "style:name").or_else(|| get(a, "draw:name")).filter(|n| !n.is_empty()) { Some(name) => Some(format!("{g}:{name}")), - None if g == "common" => { + None => { let which = get(a, "style:family").or_else(|| get(a, "text:note-class")).unwrap_or(""); Some(format!("{g}~{q}~{which}")) } - None => None, } } -/// The children of `office:styles` in `xml` (a `styles.xml` WordCraft wrote): each one's key -/// (as [`defs`] gives it) and its markup's byte range. -pub(super) fn common_children(xml: &str) -> Vec<(String, (usize, usize))> { +/// The children of `container` (a child of the root) in `xml` (a part WordCraft wrote): each +/// one's key in `group` (as [`defs`] gives it) and its markup's byte range; and the range of +/// the container's content (`None` when there is no such container). +fn children(xml: &str, container: &str, group: &str) -> (Vec<(String, (usize, usize))>, Option<(usize, usize)>) { let mut out = Vec::new(); let (mut depth, mut inside) = (0usize, false); let mut open: Option<(String, usize)> = None; + let mut content: Option<(usize, usize)> = None; events_at(xml.as_bytes(), |ev, at| match ev { Ev::Start(q, a) => { depth += 1; if depth == 2 { - inside = q == "office:styles"; + inside = q == container; + if inside { + content = Some((at.1, at.1)); + } } else if depth == 3 && inside - && let Some(key) = child_key("common", &q, &a) + && let Some(key) = child_key(group, &q, &a) { open = Some((key, at.0)); } @@ -107,11 +158,50 @@ pub(super) fn common_children(xml: &str) -> Vec<(String, (usize, usize))> { { out.push((key, (from, at.1))); } + if depth == 2 && inside { + // An empty element (``) has no content to replace: none. + content = content.filter(|c| c.0 < at.1).map(|c| (c.0, at.0.max(c.0))); + inside = false; + } depth = depth.saturating_sub(1); } Ev::Text(_) => {} }); - out + (out, content) +} + +/// `xml` (a part WordCraft wrote) with `container`'s children as read: the file's (those of +/// `group` in `defs`, in `order`; `pick` may choose WordCraft's instead, or neither), then +/// WordCraft's own the file had none of by that key. +fn with_as_read( + xml: &str, + container: &str, + group: &str, + defs: &BTreeMap>, + order: &[String], + pick: &dyn Fn(&str, &str, Option<&str>) -> Option, +) -> String { + let (kids, content) = children(xml, container, group); + let Some((from, to)) = content else { return xml.to_string() }; + let ours = |k: &str| kids.iter().find(|(key, _)| key == k).and_then(|(_, (a, b))| xml.get(*a..*b)); + let prefix = (format!("{group}:"), format!("{group}~")); + let mut body = String::new(); + for k in order.iter().filter(|k| k.starts_with(&prefix.0) || k.starts_with(&prefix.1)) { + if let Some(d) = defs.get(k) + && let Some(chosen) = pick(k, d, ours(k)) + { + body.push_str(&chosen); + } + } + for (k, (a, b)) in &kids { + if !defs.contains_key(k) + && let Some(x) = xml.get(*a..*b) + { + body.push_str(x); + } + } + let (Some(head), Some(tail)) = (xml.get(..from), xml.get(to..)) else { return xml.to_string() }; + format!("{head}{body}{tail}") } /// Whether a definition (its markup) is a drawing's: a graphic style or default style, or a @@ -123,34 +213,24 @@ fn drawing_def(xml: &str) -> bool { } /// `styles` (a `styles.xml` WordCraft wrote) with its `office:styles` as read: every kept -/// child of the file's (`order`, from `defs`), then WordCraft's own children the file had no -/// such of. A drawing's definition both have stays WordCraft's (see [`drawing_def`]). For a -/// document whose style definitions are as read. +/// child of the file's, then WordCraft's own children the file had no such of. A drawing's +/// definition both have stays WordCraft's (see [`drawing_def`]). For a document whose style +/// definitions are as read. pub(super) fn with_common_as_read(styles: &str, defs: &BTreeMap>, order: &[String]) -> String { - let kids = common_children(styles); - let (Some(first), Some(last)) = (kids.first(), kids.last()) else { return styles.to_string() }; - let (from, to) = (first.1.0, last.1.1); - let ours = |k: &str| kids.iter().find(|(key, _)| key == k).and_then(|(_, (a, b))| styles.get(*a..*b)); - let mut body = String::new(); - for k in order.iter().filter(|k| k.starts_with("common")) { - match (defs.get(k), ours(k)) { - (Some(_), Some(o)) if drawing_def(o) => body.push_str(o), - // The default graphic style applies to every drawing, which WordCraft writes from the - // model under its own (none: the standard's). - (Some(_), None) if k == "common~style:default-style~graphic" => {} - (Some(d), _) => body.push_str(d), - (None, _) => {} - } - } - for (k, (a, b)) in &kids { - if !defs.contains_key(k) - && let Some(x) = styles.get(*a..*b) - { - body.push_str(x); - } - } - let (Some(head), Some(tail)) = (styles.get(..from), styles.get(to..)) else { return styles.to_string() }; - format!("{head}{body}{tail}") + with_as_read(styles, "office:styles", "common", defs, order, &|_, d, ours| match ours { + Some(o) if drawing_def(o) => Some(o.to_string()), + // (The default graphic style is the file's: WordCraft's drawings say what they need.) + _ => Some(d.to_string()), + }) +} + +/// `styles` (a `styles.xml` WordCraft wrote) with its page layouts, master pages and their +/// automatic styles as read, WordCraft's own added where the file had none by that name. For a +/// document whose page setup is as read. +pub(super) fn with_pages_as_read(styles: &str, defs: &BTreeMap>, order: &[String]) -> String { + let keep_file = |_: &str, d: &str, _: Option<&str>| Some(d.to_string()); + let s = with_as_read(styles, "office:automatic-styles", "sauto", defs, order, &keep_file); + with_as_read(&s, "office:master-styles", "master", defs, order, &keep_file) } /// Attribute names whose values name a definition kept markup may refer to. @@ -177,8 +257,9 @@ pub(super) fn names(xml: &str) -> BTreeSet { const MAX_FILES: usize = 1000; /// The package files `defs` refer to (`xlink:href` to a path inside the package, as a style's -/// fill picture), read with `fetch`: a definition written as read needs them in the package. -pub(super) fn files(defs: &[(String, Arc)], fetch: &mut dyn FnMut(&str) -> Option>) -> Vec<(String, Vec)> { +/// fill picture; to a folder, as an embedded object in a header: the files in it), read with +/// `fetch` (`entries`: the package's paths): a definition written as read needs them. +pub(super) fn files(defs: &[(String, Arc)], entries: &[String], fetch: &mut dyn FnMut(&str) -> Option>) -> Vec<(String, Vec)> { let mut paths = BTreeSet::new(); for (_, xml) in defs.iter().filter(|(_, x)| x.contains("xlink:href")) { super::xml::events(xml.as_bytes(), |ev| { @@ -191,7 +272,19 @@ pub(super) fn files(defs: &[(String, Arc)], fetch: &mut dyn FnMut(&str) -> } }); } - paths.into_iter().filter_map(|p| fetch(&p).map(|b| (p, b))).collect() + let mut out = Vec::new(); + for p in paths { + if entries.contains(&p) { + out.extend(fetch(&p).map(|b| (p, b))); + } else { + // A folder (an object): the files in it. + let dir = format!("{}/", p.trim_end_matches('/')); + for e in entries.iter().filter(|e| e.starts_with(&dir) && !e.ends_with('/')).take(MAX_FILES) { + out.extend(fetch(e).map(|b| (e.clone(), b))); + } + } + } + out } /// Whether link `h` names a file inside the package (a relative path, not a URL or a fragment). @@ -199,6 +292,104 @@ fn inside_package(h: &str) -> bool { !h.is_empty() && !h.contains(':') && !h.starts_with(['#', '/']) && !h.contains("..") } +/// The document properties WordCraft writes from its model (§4.3); the file's others are kept. +const META_MODELLED: [&str; 11] = [ + "meta:generator", + "dc:title", + "dc:subject", + "dc:description", + "meta:initial-creator", + "dc:creator", + "dc:date", + "meta:editing-cycles", + "meta:keyword", + "meta:creation-date", + "meta:user-defined", +]; + +/// The children of `meta.xml`'s `office:meta` WordCraft doesn't model (the template, the +/// statistics, the editing time, the printing…), as read, in order. +pub(super) fn meta_defs(xml: &[u8]) -> Vec<(String, Arc)> { + let mut out = Vec::new(); + let mut depth = 0usize; + let mut open: Option<(String, usize)> = None; + events_at(xml, |ev, at| match ev { + Ev::Start(q, _) => { + depth += 1; + if depth == 3 && open.is_none() && !META_MODELLED.contains(&q.as_str()) && out.len() < MAX_DEFS { + open = Some((format!("meta~{}~{q}", out.len()), at.0)); + } + } + Ev::End(_) => { + if depth == 3 + && let Some((key, from)) = open.take() + && let Some(s) = xml.get(from..at.1).and_then(|b| std::str::from_utf8(b).ok()) + { + out.push((key, Arc::from(s))); + } + depth = depth.saturating_sub(1); + } + Ev::Text(_) => {} + }); + out +} + +/// `meta` (a `meta.xml` WordCraft wrote) with the file's document properties WordCraft doesn't +/// model (`defs`' `meta~` ones) added, in their order. +pub(super) fn with_kept_meta(meta: String, defs: &BTreeMap>, order: &[String]) -> String { + let kept: String = order.iter().filter(|k| k.starts_with("meta~")).filter_map(|k| defs.get(k)).map(|d| &**d).collect(); + match meta.rfind("") { + Some(at) if !kept.is_empty() => format!("{}{kept}{}", meta.get(..at).unwrap_or(""), meta.get(at..).unwrap_or("")), + _ => meta, + } +} + +/// A list of declarations as the file had it (`orig`, a `text:variable-decls` or the like), +/// with those of WordCraft's (`ours`, the same list) it doesn't have by name; `ours` alone when +/// the file had none. +pub(super) fn merge_decls(orig: Option<&str>, ours: String) -> String { + let Some(orig) = orig else { return ours }; + let named = |xml: &str| -> Vec<(String, (usize, usize))> { + let mut out = Vec::new(); + let mut depth = 0usize; + events_at(xml.as_bytes(), |ev, at| match ev { + Ev::Start(_, a) => { + depth += 1; + if depth == 2 + && let Some(n) = get(&a, "text:name") + { + out.push((n.to_string(), at)); + } + } + Ev::End(_) => { + if depth == 2 + && let Some(last) = out.last_mut() + { + last.1.1 = at.1; + } + depth = depth.saturating_sub(1); + } + Ev::Text(_) => {} + }); + out + }; + let have: BTreeSet = named(orig).into_iter().map(|(n, _)| n).collect(); + let extra: String = named(&ours).into_iter().filter(|(n, _)| !have.contains(n)).filter_map(|(_, (a, b))| ours.get(a..b)).collect(); + if extra.is_empty() { + return orig.to_string(); + } + match orig.rfind("') && !orig.trim_end().ends_with("/>") => { + format!("{}{extra}{}", orig.get(..at).unwrap_or(""), orig.get(at..).unwrap_or("")) + } + // ``: opened and closed around them. + _ => { + let el = orig.trim_start_matches('<').split([' ', '/', '>']).next().unwrap_or(""); + format!("{}>{extra}", orig.trim_end().trim_end_matches("/>")) + } + } +} + /// The sequences (§7.4.13 `text:sequence`) `xml` counts, which the document declares. pub(super) fn sequences(xml: &str) -> BTreeSet { let mut out = BTreeSet::new(); @@ -221,7 +412,7 @@ pub(super) fn sequences(xml: &str) -> BTreeSet { /// written from the model instead. pub(super) type Needed = (BTreeMap>, BTreeMap>, BTreeMap>); -pub(super) fn needed(defs: &BTreeMap>, start: BTreeSet) -> Option { +pub(super) fn needed(defs: &BTreeMap>, start: BTreeSet, masters: bool) -> Option { let (mut auto, mut common, mut fonts) = (BTreeMap::new(), BTreeMap::new(), BTreeMap::new()); let mut todo: Vec = start.into_iter().collect(); let mut seen = BTreeSet::new(); @@ -238,7 +429,7 @@ pub(super) fn needed(defs: &BTreeMap>, start: BTreeSet) } else { continue; }; - if def.contains("style:master-page-name=\"") && !def.contains("style:master-page-name=\"\"") { + if !masters && def.contains("style:master-page-name=\"") && !def.contains("style:master-page-name=\"\"") { return None; } into.insert(n, def.clone()); @@ -258,13 +449,25 @@ mod tests { let d: BTreeMap> = defs(CONTENT.as_bytes(), true).into_iter().collect(); assert_eq!(d.keys().cloned().collect::>(), ["auto:L1", "auto:P1", "auto:P2", "font:Liberation Serif"]); assert!(d["auto:P1"].starts_with("")); - let (auto, common, fonts) = needed(&d, names(r#"x"#)).unwrap(); + let (auto, common, fonts) = needed(&d, names(r#"x"#), false).unwrap(); assert_eq!(auto.keys().cloned().collect::>(), ["L1", "P1"]); // "Body" and "Bullets" weren't kept here (they're common styles in styles.xml). assert!(common.is_empty()); assert_eq!(fonts.keys().cloned().collect::>(), ["Liberation Serif"]); // A paragraph starting a page isn't written as kept. - assert!(needed(&d, names(r#""#)).is_none()); + assert!(needed(&d, names(r#""#), false).is_none()); + } + + #[test] + fn angles_from_an_older_file_get_their_unit() { + let xml = r#""#; + let d: BTreeMap> = defs(xml.as_bytes(), false).into_iter().collect(); + assert!(d["common:G"].contains(r#"draw:angle="30deg""#), "{}", d["common:G"]); + assert!(d["common:H"].contains(r#"draw:rotation="45deg""#)); + assert!(d["common:D"].contains(r#"draw:angle="45deg""#)); + // From a 1.3 file: as it is. + let d: BTreeMap> = defs(xml.replace("1.2", "1.3").as_bytes(), false).into_iter().collect(); + assert!(d["common:G"].contains(r#"draw:angle="300""#)); } #[test] @@ -274,6 +477,6 @@ mod tests { // A definition naming itself is followed once. let d: BTreeMap> = [("auto:A".to_string(), Arc::from(r#""#))].into(); - assert_eq!(needed(&d, ["A".to_string()].into()).unwrap().0.len(), 1); + assert_eq!(needed(&d, ["A".to_string()].into(), false).unwrap().0.len(), 1); } } diff --git a/crates/formats/src/odt/package.rs b/crates/formats/src/odt/package.rs index a672555631..f00bae7582 100644 --- a/crates/formats/src/odt/package.rs +++ b/crates/formats/src/odt/package.rs @@ -323,6 +323,8 @@ fn styles_xml( part_styles: &str, named: &std::collections::BTreeSet, ) -> String { + // WordCraft's page layouts take another prefix next to the file's own (see `super::kept`). + let pm = if doc.kept.of("odt").is_some() { "wcpm" } else { "pm" }; // The document defaults: Normal's font, spacing and indents, pair kerning and tab interval. let body_font = doc.styles.default_chr.font.clone().unwrap_or_else(|| wordcraft_doc::styles::BODY_FONT.to_string()); let normal = doc.styles.resolve_para(&Default::default()); @@ -411,7 +413,7 @@ fn styles_xml( String::new() }; s.push_str(&format!( - "{cols}{header}{footer}", + "{cols}{header}{footer}", i + 1, pt(sec.page_w), pt(sec.page_h), @@ -426,7 +428,7 @@ fn styles_xml( s.push_str(""); for (i, (name, _)) in pages.iter().enumerate() { let xml = parts.get(i).map(|p| p.xml.as_str()).unwrap_or(""); - s.push_str(&format!("{xml}", x(name), i + 1)); + s.push_str(&format!("{xml}", x(name), i + 1)); } s.push_str("\n"); s @@ -777,14 +779,23 @@ fn with_kept_files( manifest: &mut String, ) { let Some(k) = kept else { return }; + let entry = |manifest: &mut String, path: &str, mime: &str| { + let e = format!("manifest:full-path=\"{}\"", x(path)); + if !manifest.contains(&e) { + manifest.push_str(&format!("\n", x(mime))); + } + }; for (path, bytes) in &k.files { if !taken(path) && !files.iter().any(|(p, _)| p == path) { + // As the file's manifest had it (the folder too), else by its extension. + if let Some((dir, _)) = path.rsplit_once('/') + && let Some(mime) = k.defs.get(&format!("manifest:{dir}/")) + { + entry(manifest, &format!("{dir}/"), mime); + } let ext = path.rsplit('.').next().unwrap_or(""); - manifest.push_str(&format!( - "\n", - x(path), - x(model::mime_of(ext)) - )); + let mime = k.defs.get(&format!("manifest:{path}")).map_or_else(|| model::mime_of(ext).to_string(), |m| m.to_string()); + entry(manifest, path, &mime); files.push((path.clone(), bytes.clone())); } } @@ -872,6 +883,8 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { let kept = doc.kept.of("odt"); let mut w = if kept.is_some() { Writer::with_prefix(KEPT_PREFIX) } else { Writer::default() }; w.kept_defs = std::sync::Arc::new(kept.map(|k| k.defs.clone()).unwrap_or_default()); + // The file's master pages are written as read: what was kept may start them. + w.kept_pages = kept.is_some_and(|k| k.pages_as_read(doc)); w.accents.clone_from(&accents); w.conn_kinds = conn_kinds(&flow.blocks); w.sources = doc.sources.iter().map(|s| (s.tag.clone(), s.clone())).collect(); @@ -889,11 +902,20 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { for (k, (sec, range)) in sections.iter().enumerate() { let new_page = k == 0 || sec.start != SectionStart::Continuous; if new_page { - let name = if k == 0 { "Standard".to_string() } else { format!("Page{}", pages.len() + 1) }; + // The file's own master page while its pages are as read. + let own = kept + .filter(|kx| kx.pages_as_read(doc) && kx.section_masters.len() == sections.len()) + .and_then(|kx| kx.section_masters.get(k).cloned().flatten()); + let name = match &own { + Some(m) => m.clone(), + None if k == 0 => "Standard".to_string(), + None => format!("{}Page{}", w.prefix, pages.len() + 1), + }; pages.push((name.clone(), sec.clone())); page_cols = sec.columns.clone(); - // The first section needs its paragraph to name the page only to restart numbering. - if k > 0 || sec.page_num_start.is_some() { + // The first section needs its paragraph to name the page only to restart numbering + // (or when it is the file's own other than Standard). + if k > 0 || sec.page_num_start.is_some() || own.as_deref().is_some_and(|m| m != "Standard") { w.pending_start = Some((name, sec.page_num_start)); } } @@ -908,7 +930,7 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { w.continue_list = k > 0 && list_at(flow.blocks.get(range.start)) && list_at(range.start.checked_sub(1).and_then(|i| flow.blocks.get(i))); if wrapped { w.close_ins(&mut body); - let n = w.section_styles.len() + 1; + let n = format!("{}{}", w.prefix, w.section_styles.len() + 1); // Whether its columns end balanced, said so every reader does the same: as WordCraft // lays it out (balanced when text follows on the page, or when the section says). let next_continuous = sections.get(k + 1).is_some_and(|(s, _)| s.start == SectionStart::Continuous); @@ -924,6 +946,8 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { body.push_str(&format!("")); } w.blocks(flow.blocks.get(range.clone()).unwrap_or(&[]), &mut body, 0); + // A kept section doesn't cross a page or column section's edge. + w.close_sections(&mut body); w.pending_start = None; w.continue_list = false; if wrapped { @@ -953,12 +977,29 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { format!("{d}") }; let user_decls = user_field_decls(&doc.custom_props); + // The file's own declarations, with WordCraft's added to them; its scripts. + let kept_decl = |el: &str, ours: String| super::kept::merge_decls(kept.and_then(|k| k.defs.get(&format!("decls:{el}"))).map(|d| &**d), ours); + let var_decls = kept_decl("text:variable-decls", var_decls); + let user_decls = kept_decl("text:user-field-decls", user_decls); + let decls = kept_decl("text:sequence-decls", decls); + let other_decls: String = kept + .map(|k| { + k.defs + .iter() + .filter(|(key, _)| { + key.starts_with("decls:") && !["variable", "user-field", "sequence"].iter().any(|d| key.contains(&format!("text:{d}-decls"))) + }) + .map(|(_, d)| &**d) + .collect() + }) + .unwrap_or_default(); + let scripts = kept.and_then(|k| k.defs.get("top:office:scripts")).map(|d| d.to_string()).unwrap_or_default(); let content = format!( - "\n{}\n{}{tracked}{var_decls}{user_decls}{decls}{body}\n", + "\n{scripts}{}\n{}{tracked}{var_decls}{user_decls}{decls}{other_decls}{body}\n", w.automatic_styles(), super::forms::forms_xml(&w.forms, &hidden_controls(doc)) ); - let mut hw = Writer::with_prefix("M"); + let mut hw = Writer::with_prefix(if kept.is_some() { "wcM" } else { "M" }); hw.accents = accents; hw.variables.clone_from(&w.variables); let parts: Vec = pages.iter().map(|(_, sec)| page_parts(doc, sec, &mut hw)).collect(); @@ -996,6 +1037,11 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { w.objects.append(&mut hw.objects); w.object_dirs.append(&mut hw.object_dirs); let meta = meta_xml(&flow.meta, &doc.core, &doc.custom_props); + // With the file's properties WordCraft doesn't model. + let meta = match kept { + Some(k) => super::kept::with_kept_meta(meta, &k.defs, &k.order), + None => meta, + }; let mut manifest = format!( "\n\n\n\n\n\n\n", ); @@ -1041,6 +1087,11 @@ fn export_as(doc: &Document, mime: &str) -> Result, String> { // What the paragraphs written as read need, as read (see `super::kept`). // Style definitions as read while nobody changed them: the file's own `office:styles`. let styles = styles_as_read(doc, kept, styles, &mut w.kept_used); + // Page setup as read while nobody changed it: the file's own page layouts and master pages. + let styles = match kept.filter(|k| k.pages_as_read(doc)) { + Some(k) => super::kept::with_pages_as_read(&styles, &k.defs, &k.order), + None => styles, + }; let (content, styles) = with_kept_defs(content, styles, &w.kept_used, &w.kept_defs); for (name, data) in [ ("META-INF/manifest.xml", manifest.as_bytes()), diff --git a/crates/formats/src/odt/tests.rs b/crates/formats/src/odt/tests.rs index fcfc37ca97..3a57bdc3e3 100644 --- a/crates/formats/src/odt/tests.rs +++ b/crates/formats/src/odt/tests.rs @@ -688,14 +688,14 @@ fn export_keeps_a_page_number_restart_on_the_first_section() { #[test] fn export_of_a_one_section_document_stays_simple() { let doc = import(§ions_package()).unwrap(); - let one = import(&export(&doc).unwrap()).unwrap(); + let one = import(&crate::odt::tests::export_model(&doc).unwrap()).unwrap(); let mut single = one.clone(); for b in single.body.iter_mut() { if let wordcraft_doc::Block::Para(p) = std::sync::Arc::make_mut(b) { p.section = None; } } - let bytes = export(&single).unwrap(); + let bytes = crate::odt::tests::export_model(&single).unwrap(); let mut z = archive(&bytes); let styles = String::from_utf8(read_entry(&mut z, "styles.xml").unwrap()).unwrap(); let content = String::from_utf8(read_entry(&mut z, "content.xml").unwrap()).unwrap(); diff --git a/crates/formats/src/odt/tests_more.rs b/crates/formats/src/odt/tests_more.rs index c4bf99c930..32649e89c9 100644 --- a/crates/formats/src/odt/tests_more.rs +++ b/crates/formats/src/odt/tests_more.rs @@ -23,6 +23,7 @@ mod headers; mod indexes; mod kept; mod lists; +mod lossless; mod named_styles; mod notes; mod numbering_fields; diff --git a/crates/formats/src/odt/tests_more/cell_formulas.rs b/crates/formats/src/odt/tests_more/cell_formulas.rs index 1708e893f4..24e9006a5c 100644 --- a/crates/formats/src/odt/tests_more/cell_formulas.rs +++ b/crates/formats/src/odt/tests_more/cell_formulas.rs @@ -4,8 +4,8 @@ //! fields in cells are written back as cell formulas. use crate::odt::cell_formula::{odf_of, word_of}; +use crate::odt::import; use crate::odt::tests::{NS_T, content_of, package}; -use crate::odt::{export, import}; use wordcraft_doc::InlineObject; #[test] @@ -96,22 +96,22 @@ fn a_cell_formula_opens_as_a_formula_field_showing_its_value() { #[test] fn formula_fields_in_cells_are_written_as_cell_formulas() { let d = open(); - let c = content_of(&export(&d).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&d).unwrap()); assert!(c.contains(r#"table:formula="ooow:(sum <A1:A2>)" office:value-type="float" office:value="4""#), "{c}"); // Read back, the same field. - let again = import(&export(&d).unwrap()).unwrap(); + let again = import(&crate::odt::tests::export_model(&d).unwrap()).unwrap(); assert_eq!(fields(&again), fields(&d)); } #[test] fn cells_holding_numbers_are_written_as_numbers() { - let c = content_of(&export(&open()).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&open()).unwrap()); assert!(c.contains(r#"office:value-type="float" office:value="2.5">x1,5-1.2.3 -7 "# ); - let c = content_of(&export(&import(&package(&content, None)).unwrap()).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&import(&package(&content, None)).unwrap()).unwrap()); assert_eq!(c.matches(r#"office:value-type="float""#).count(), 1, "{c}"); assert!(c.contains(r#"office:value="-7""#), "{c}"); } diff --git a/crates/formats/src/odt/tests_more/fields.rs b/crates/formats/src/odt/tests_more/fields.rs index 1cd7efafd2..2e180144e2 100644 --- a/crates/formats/src/odt/tests_more/fields.rs +++ b/crates/formats/src/odt/tests_more/fields.rs @@ -329,7 +329,7 @@ fn user_fields_stay_user_fields() { ); let d = import(&package(&content, None)).unwrap(); assert!(d.custom_props.iter().all(|p| p.odf_user_field), "{:?}", d.custom_props); - let saved = export(&d).unwrap(); + let saved = crate::odt::tests::export_model(&d).unwrap(); let c = crate::odt::tests::content_of(&saved); assert!(c.contains(r#""#), "{c}"); assert!(c.contains(r#"office:value-type="float" office:value="3""#), "{c}"); diff --git a/crates/formats/src/odt/tests_more/flow_breaks.rs b/crates/formats/src/odt/tests_more/flow_breaks.rs index 8ca3a3ea52..8168fcc90b 100644 --- a/crates/formats/src/odt/tests_more/flow_breaks.rs +++ b/crates/formats/src/odt/tests_more/flow_breaks.rs @@ -78,7 +78,7 @@ fn a_tables_alignment_width_and_row_heights_are_kept() { assert_eq!((t.props.align, t.props.width_pct), (Some(Align::Center), Some(50.0))); let heights: Vec<(Option, HeightRule)> = t.rows.iter().map(|r| (r.props.height, r.props.height_rule)).collect(); assert_eq!(heights, [(Some(36.0), HeightRule::Exact), (Some(18.0), HeightRule::AtLeast), (None, HeightRule::Auto)]); - let c = content_of(&export(&d).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&d).unwrap()); for want in [r#"style:rel-width="50%" table:align="center""#, r#"style:row-height="36pt""#, r#"style:min-row-height="18pt""#] { assert!(c.contains(want), "{want} in {c}"); } @@ -86,7 +86,7 @@ fn a_tables_alignment_width_and_row_heights_are_kept() { let plain = doc("", r#"x"#); let pt = plain.body.iter().find_map(|b| b.as_table()).unwrap(); assert_eq!((pt.props.align, pt.props.width_pct), (None, Some(100.0))); - assert!(content_of(&export(&plain).unwrap()).contains(r#"table:align="margins""#)); + assert!(content_of(&crate::odt::tests::export_model(&plain).unwrap()).contains(r#"table:align="margins""#)); } #[test] @@ -98,7 +98,7 @@ fn columns_are_written_with_their_share_of_the_table() { styles, r#"ab"#, ); - let c = content_of(&export(&d).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&d).unwrap()); assert!(c.contains(r#"style:column-width="72pt" style:rel-column-width="7200*""#) && c.contains(r#"style:rel-column-width="21600*""#), "{c}"); // An aligned table keeps its widths: LibreOffice lays one out oddly with shares. let left = r#""#; @@ -106,7 +106,7 @@ fn columns_are_written_with_their_share_of_the_table() { &format!("{styles}{left}"), r#"ab"#, ); - let c = content_of(&export(&d).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&d).unwrap()); assert!(c.contains(r#"table:align="left""#) && !c.contains("rel-column-width"), "{c}"); } @@ -138,9 +138,9 @@ fn a_tables_own_space_above_and_below_is_kept() { (t.props.space_before.map(|v| (v * 100.0).round() / 100.0), t.props.space_after.map(|v| (v * 100.0).round() / 100.0)) }; assert_eq!(space(&d), (Some(7.2), Some(8.5))); - let out = content_of(&export(&d).unwrap()); + let out = content_of(&crate::odt::tests::export_model(&d).unwrap()); assert!(out.contains(r#"fo:margin-top="7.2pt" fo:margin-bottom="8.5"#), "{out}"); - assert_eq!(space(&import(&export(&d).unwrap()).unwrap()), (Some(7.2), Some(8.5))); + assert_eq!(space(&import(&crate::odt::tests::export_model(&d).unwrap()).unwrap()), (Some(7.2), Some(8.5))); } #[test] @@ -156,6 +156,6 @@ fn a_table_starting_a_page_starts_it() { assert_eq!(first.props.page_break_before, Some(true)); // Nothing else starts a page. assert!(paras(&d).iter().all(|p| p.page_break_before != Some(true))); - let c = content_of(&export(&d).unwrap()); + let c = content_of(&crate::odt::tests::export_model(&d).unwrap()); assert_eq!(c.matches(r#"fo:break-before="page""#).count(), 2, "{c}"); } diff --git a/crates/formats/src/odt/tests_more/headers.rs b/crates/formats/src/odt/tests_more/headers.rs index a0a1158b83..4b166008be 100644 --- a/crates/formats/src/odt/tests_more/headers.rs +++ b/crates/formats/src/odt/tests_more/headers.rs @@ -244,7 +244,7 @@ fn a_first_page_master_followed_by_another_is_a_different_first_page() { // The first page's own size and margins. assert_eq!(sec.margin_top, 144.0); // Written back: an empty first-page header. - let saved = export(&d).unwrap(); + let saved = crate::odt::tests::export_model(&d).unwrap(); let s = crate::odt::tests::entry(&saved, "styles.xml"); let first = s.split("").nth(1).and_then(|r| r.split("").next()).unwrap_or("missing"); assert!(first.trim().starts_with("ab"#, r#"xy"#, r#"12"#, + // Sections left open, closed twice, inside a list, empty. + r#"x"#, + r#"yz"#, + r#"w"#, + r#"v"#, ] { let content = format!( r#"{body}"# diff --git a/crates/formats/src/odt/tests_more/lossless.rs b/crates/formats/src/odt/tests_more/lossless.rs new file mode 100644 index 0000000000..4a6af8f91c --- /dev/null +++ b/crates/formats/src/odt/tests_more/lossless.rs @@ -0,0 +1,275 @@ +//! The no-edit loss gate: a document opened and saved without changes loses nothing. Each +//! sample below is saved without edits and compared, part by part, with what was read: every +//! element, every attribute, every formatting property's value and every word must still be +//! there. What may go is listed in [`ALLOWED`], each with its reason. +//! +//! The samples are written here, by us, for the purpose (see `kept`'s module docs): never a file +//! or text taken from someone else. + +use std::collections::BTreeMap; + +use crate::odt::xml::{Ev, events}; +use crate::odt::{export, import}; + +/// What a save may lose, and why. Matched against the start of a loss (see [`inventory`]). +const ALLOWED: &[(&str, &str)] = &[]; + +const NS: &str = r#"xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" xmlns:style="urn:oasis:names:tc:opendocument:xmlns:style:1.0" xmlns:table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" xmlns:fo="urn:oasis:names:tc:opendocument:xmlns:xsl-fo-compatible:1.0" xmlns:svg="urn:oasis:names:tc:opendocument:xmlns:svg-compatible:1.0" xmlns:draw="urn:oasis:names:tc:opendocument:xmlns:drawing:1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:meta="urn:oasis:names:tc:opendocument:xmlns:meta:1.0" xmlns:number="urn:oasis:names:tc:opendocument:xmlns:datastyle:1.0" xmlns:loext="urn:org:documentfoundation:names:experimental:office:xmlns:loext:1.0" xmlns:wcx="urn:example:wordcraft-test" office:version="1.3""#; + +/// A sample package: `content.xml`, `styles.xml` and `meta.xml` bodies (inside their roots). +struct Sample { + name: &'static str, + content: &'static str, + styles: &'static str, + meta: &'static str, +} + +fn package(s: &Sample) -> Vec { + let parts = [ + ("mimetype", "application/vnd.oasis.opendocument.text".to_string()), + ("content.xml", format!("{}", s.content)), + ("styles.xml", format!("{}", s.styles)), + ("meta.xml", format!("{}", s.meta)), + ]; + let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); + let o = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored); + for (name, data) in parts { + z.start_file(name, o).unwrap(); + std::io::Write::write_all(&mut z, data.as_bytes()).unwrap(); + } + z.finish().unwrap().into_inner() +} + +/// A length's value in points to a tenth, else the value as written. +fn norm(v: &str) -> String { + let v = v.trim(); + let units = [("cm", 72.0 / 2.54), ("mm", 72.0 / 25.4), ("in", 72.0), ("pt", 1.0), ("pc", 12.0), ("px", 0.75)]; + for (u, f) in units { + if let Some(n) = v.strip_suffix(u).and_then(|n| n.parse::().ok()) { + return format!("{:.1}pt", n * f); + } + } + v.to_lowercase() +} + +/// Attributes whose values are names or places (their presence counts, not their value). +fn nameish(a: &str) -> bool { + ["name", "id", "ref", "href", "class", "index", "parent", "master", "list-style", "data-style"] + .iter() + .any(|n| a.ends_with(n) || a.contains("style-name")) +} + +/// What a package holds, as counts: `el `, `attr @`, `val @=` (the +/// values of formatting properties), and `word `. +fn inventory(bytes: &[u8]) -> BTreeMap { + let mut z = zip::ZipArchive::new(std::io::Cursor::new(bytes.to_vec())).unwrap(); + let mut inv = BTreeMap::new(); + let mut add = |k: String| *inv.entry(k).or_insert(0usize) += 1; + for part in ["content.xml", "styles.xml", "meta.xml"] { + let mut xml = Vec::new(); + let Ok(mut f) = z.by_name(part) else { continue }; + std::io::Read::read_to_end(&mut f, &mut xml).unwrap(); + let mut text = String::new(); + events(&xml, |ev| match ev { + Ev::Start(q, a) => { + add(format!("el {q}")); + let props = q.ends_with("-properties"); + for (k, v) in a { + if k.starts_with("xmlns") { + continue; + } + if props && !nameish(&k) { + add(format!("val {q}@{k}={}", norm(&v))); + } else { + add(format!("attr {q}@{k}")); + } + } + if matches!(q.as_str(), "text:p" | "text:h" | "text:tab" | "text:s" | "text:line-break") { + text.push(' '); + } + } + Ev::End(_) => {} + Ev::Text(t) => text.push_str(&t), + }); + if part == "content.xml" { + for w in text.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) { + add(format!("word {w}")); + } + } + } + inv +} + +/// What `saved` lacks that `orig` has (none of it allowed). +fn losses(orig: &[u8], saved: &[u8]) -> Vec { + let (a, b) = (inventory(orig), inventory(saved)); + a.into_iter() + .filter(|(k, n)| b.get(k).copied().unwrap_or(0) < *n && (k.starts_with("word ") || !b.contains_key(k))) + .map(|(k, _)| k) + .filter(|k| !ALLOWED.iter().any(|(p, _)| k.starts_with(p))) + .collect() +} + +fn check(s: &Sample) { + let orig = package(s); + let d = import(&orig).unwrap(); + let saved = export(&d).unwrap(); + let lost = losses(&orig, &saved); + assert!(lost.is_empty(), "{}: lost on a save without edits:\n {}", s.name, lost.join("\n ")); + // Saved again: nothing more goes. + let again = export(&import(&saved).unwrap()).unwrap(); + let lost = losses(&saved, &again); + assert!(lost.is_empty(), "{}: lost on a second save:\n {}", s.name, lost.join("\n ")); +} + +/// Paragraphs and text WordCraft partly models: unknown elements, attributes and namespaces. +const TEXT: Sample = Sample { + name: "text", + content: r##"Orchard notesPears ripen late in the year, and apples early.Second paragraph with a 1Footnote words. note."##, + styles: r##""##, + meta: "", +}; + +/// Style definitions WordCraft doesn't use: an unused list style, note and line numbering +/// settings, an outline style, unknown style attributes. +const STYLES: Sample = Sample { + name: "styles", + content: r##"Only text."##, + styles: r##""##, + meta: "", +}; + +#[test] +fn a_save_without_edits_loses_nothing() { + for s in SAMPLES { + check(&s); + } +} + +#[test] +fn the_gate_sees_a_loss() { + let with = Sample { + name: "with", + content: r##"Kept words"##, + styles: "", + meta: "", + }; + let without = + Sample { name: "without", content: r##"Kept"##, styles: "", meta: "" }; + let lost = losses(&package(&with), &package(&without)); + assert_eq!(lost, ["attr text:p@wcx:a", "el wcx:b", "word words"]); +} + +/// Document properties WordCraft doesn't model. +const META: Sample = Sample { + name: "meta", + content: r##"Body."##, + styles: "", + meta: r##"Sample WriterLossless sampleen-GBPT1H2M3S2026-01-02T03:04:05Sample Printertrue"##, +}; + +/// Declarations at the start of the text and the document's scripts. +const DECLS: Sample = Sample { + name: "decls", + content: r##"Decls."##, + styles: "", + meta: "", +}; + +/// Page layouts and master pages WordCraft partly models: the footnote separator, header +/// properties, a layout grid, a header with its own paragraph. +const PAGES: Sample = Sample { + name: "pages", + content: r##"Page."##, + styles: r##"Running words"##, + meta: "", +}; + +/// A table WordCraft partly models: unknown table, row and cell attributes. +const TABLE: Sample = Sample { + name: "table", + content: r##"OneTwoAfter."##, + styles: "", + meta: "", +}; + +/// Named sections, one protected. +const SECTIONS: Sample = Sample { + name: "sections", + content: r##"Before.Inside the terms.Inside the lock.After."##, + styles: "", + meta: "", +}; + +/// Every sample. +const SAMPLES: [Sample; 7] = [TEXT, STYLES, META, DECLS, TABLE, PAGES, SECTIONS]; + +/// Whether `xml` is well-formed: every element closed, in order, once. +fn well_formed(xml: &[u8]) -> bool { + let mut r = quick_xml::Reader::from_reader(xml); + r.config_mut().check_end_names = true; + let mut buf = Vec::new(); + let mut depth = 0i64; + loop { + buf.clear(); + match r.read_event_into(&mut buf) { + Ok(quick_xml::events::Event::Start(_)) => depth += 1, + Ok(quick_xml::events::Event::End(_)) => depth -= 1, + Ok(quick_xml::events::Event::Eof) => return depth == 0, + Ok(_) => {} + Err(_) => return false, + } + if depth < 0 { + return false; + } + } +} + +fn parts_well_formed(bytes: &[u8]) -> Result<(), String> { + let mut z = zip::ZipArchive::new(std::io::Cursor::new(bytes.to_vec())).map_err(|e| e.to_string())?; + for part in ["content.xml", "styles.xml", "meta.xml", "settings.xml", "META-INF/manifest.xml"] { + let mut xml = Vec::new(); + let mut f = z.by_name(part).map_err(|e| format!("{part}: {e}"))?; + std::io::Read::read_to_end(&mut f, &mut xml).map_err(|e| e.to_string())?; + if !well_formed(&xml) { + return Err(format!("{part} isn't well-formed:\n{}", String::from_utf8_lossy(&xml))); + } + } + Ok(()) +} + +#[test] +fn edits_leave_kept_markup_well_formed() { + // Whatever is edited (a paragraph that opens a section deleted, text typed into each + // paragraph, a style changed), what is written stays well-formed and reads back. + use wordcraft_doc::Pos; + for s in SAMPLES { + let d = import(&package(&s)).unwrap(); + let mut edits: Vec = Vec::new(); + for i in 0..d.body.len() { + let mut e = d.clone(); + if e.body.get(i).is_some_and(|b| b.as_para().is_some()) { + e.insert_text(&Pos::body(i, 0), "Edited ", &Default::default()).unwrap(); + } + edits.push(e); + if d.body.len() > 1 { + let mut e = d.clone(); + e.body.remove(i); + edits.push(e); + } + } + let mut styled = d.clone(); + if let Some(st) = styled.styles.styles.iter_mut().find(|s| s.id == "Normal") { + st.chr.italic = Some(true); + } + edits.push(styled); + for e in edits { + let saved = export(&e).unwrap(); + if let Err(m) = parts_well_formed(&saved) { + panic!("{}: {m}", s.name); + } + let again = export(&import(&saved).unwrap()).unwrap(); + assert!(parts_well_formed(&again).is_ok(), "{}", s.name); + } + } +} diff --git a/crates/formats/src/odt/tests_more/objects.rs b/crates/formats/src/odt/tests_more/objects.rs index 0bf89285f0..f57db5c093 100644 --- a/crates/formats/src/odt/tests_more/objects.rs +++ b/crates/formats/src/odt/tests_more/objects.rs @@ -148,7 +148,16 @@ fn an_object_in_a_header_is_kept_too() { ("Object 1/content.xml", SHEET.as_bytes()), ("ObjectReplacements/Object 1", METAFILE), ]); - let saved = export(&import(&pkg).unwrap()).unwrap(); + // Unedited: the header as read, and the object's files it names with it. + let d = import(&pkg).unwrap(); + let saved = export(&d).unwrap(); + let z = zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap(); + let names: Vec<&str> = z.file_names().collect(); + assert!(names.contains(&"Object 1/content.xml") && names.contains(&"ObjectReplacements/Object 1"), "{names:?}"); + assert!(entry(&saved, "styles.xml").contains(">()); assert!(entry(&saved, "styles.xml").contains(">>, pub(super) kept_used: super::kept::Needed, + /// The file's own master pages are written (its page setup is as read): kept markup may + /// start one. + pub(super) kept_pages: bool, + /// Sections kept as read that are open (see `Para::wrap`). + pub(super) sections_open: usize, /// Cell styles by (shading, vertical-alignment). cell_styles: Vec, /// Graphic styles of floating pictures: their `style:graphic-properties` attributes. @@ -555,22 +560,50 @@ impl Writer { /// Paragraph `p`, or with `chain` the paragraphs joined at deleted paragraph marks (`p` /// is the first). + /// Close the sections kept as read that are still open (at the end of what holds them). + pub(super) fn close_sections(&mut self, out: &mut String) { + if self.sections_open > 0 { + self.close_ins(out); + } + while self.sections_open > 0 { + out.push_str(""); + self.sections_open -= 1; + } + } + /// Paragraph `p` as it was read, when it still is and what it refers to was kept too (see /// `super::kept`); false when it is written from the model. fn kept_para(&mut self, p: &Para, out: &mut String) -> bool { let Some(xml) = &p.kept else { return false }; - let Some((auto, common, fonts)) = super::kept::needed(&self.kept_defs, super::kept::names(xml)) else { return false }; + self.kept_markup(xml, out) + } + + /// Markup kept as read (a paragraph's or a table's), with the definitions it names; false + /// when one of them can't be written as read (see `super::kept::needed`). + pub(in crate::odt) fn kept_markup(&mut self, xml: &str, out: &mut String) -> bool { + if !self.kept_defs_of(xml) { + return false; + } self.close_ins(out); - self.kept_used.0.extend(auto); - self.kept_used.1.extend(common); - self.kept_used.2.extend(fonts); - self.sequences.extend(super::kept::sequences(xml)); out.push_str(xml); true } + /// The definitions markup kept as read names, added to those written; false when one of + /// them can't be written as read. + fn kept_defs_of(&mut self, xml: &str) -> bool { + let Some((auto, common, fonts)) = super::kept::needed(&self.kept_defs, super::kept::names(xml), self.kept_pages) else { return false }; + self.kept_used.0.extend(auto); + self.kept_used.1.extend(common); + self.kept_used.2.extend(fonts); + self.sequences.extend(super::kept::sequences(xml)); + true + } + fn para_in(&mut self, p: &Para, chain: Option<&Vec>, out: &mut String) { - if chain.is_none() && self.master.is_none() && self.kept_para(p, out) { + if chain.is_none() && (self.master.is_none() || self.kept_pages) && self.kept_para(p, out) { + // Its own page start, if any, is in its markup. + self.master = None; return; } let paras: &[Para] = chain.map(Vec::as_slice).unwrap_or(std::slice::from_ref(p)); @@ -615,6 +648,27 @@ impl Writer { self.chain.clear(); } + /// Paragraph `p` (`chain`: the paragraphs a tracked deletion joins it to) with the sections + /// kept as read that open before it and close after it, at the top only. + fn wrapped_para(&mut self, p: &Para, chain: Option<&Vec>, out: &mut String, depth: usize) { + let wrap = p.wrap.as_deref().filter(|_| depth == 0); + for tag in wrap.map(|w| w.0.as_slice()).unwrap_or(&[]) { + self.close_ins(out); + // With the section style it names. + self.kept_defs_of(tag); + out.push_str(tag); + self.sections_open += 1; + } + self.para_in(p, chain, out); + for _ in 0..wrap.map_or(0, |w| w.1) { + if self.sections_open > 0 { + self.close_ins(out); + out.push_str(""); + self.sections_open -= 1; + } + } + } + pub(super) fn blocks(&mut self, blocks: &[FBlock], out: &mut String, depth: usize) { let joined = join_deleted_marks(blocks); let (blocks, chains) = match &joined { @@ -732,7 +786,7 @@ impl Writer { out.push('\n'); } FBlock::Para(p) => { - self.para_in(p, chain(i), out); + self.wrapped_para(p, chain(i), out, depth); out.push('\n'); i += 1; } @@ -943,8 +997,11 @@ impl Writer { } else { ("", props.as_str()) }; + // Laid out where WordCraft places it, not held in the text's area: said, as the + // standard gives no default (§20.267) and a file's default style may say otherwise. + let flow = if props.contains("style:flow-with-text") { "" } else { " style:flow-with-text=\"false\"" }; s.push_str(&format!( - "", + "", self.prefix, i + 1 )); diff --git a/crates/formats/src/odt/writer/tables.rs b/crates/formats/src/odt/writer/tables.rs index 0e7f4fba05..2410a0803c 100644 --- a/crates/formats/src/odt/writer/tables.rs +++ b/crates/formats/src/odt/writer/tables.rs @@ -123,6 +123,15 @@ impl Writer { } pub(in crate::odt) fn table(&mut self, t: &FTable, out: &mut String, depth: usize) { + // As read, while it is what was read. + if let Some(xml) = &t.kept + && (self.master.is_none() || self.kept_pages) + && self.kept_markup(xml, out) + { + self.master = None; + out.push('\n'); + return; + } self.close_ins(out); let cols = t.cols().max(1); let widths: Vec = if t.widths.len() == cols && t.widths.iter().all(|w| w.is_finite() && *w > 0.0) { diff --git a/crates/formats/src/rtf/reader.rs b/crates/formats/src/rtf/reader.rs index c17fa057bf..e190a6ff41 100644 --- a/crates/formats/src/rtf/reader.rs +++ b/crates/formats/src/rtf/reader.rs @@ -409,7 +409,7 @@ impl Reader { let rights = std::mem::take(&mut self.table_rights); let widths = column_grid(&mut rows, &rights); // Word draws only the borders the cells name: no grid style, each cell its own. - self.body.push(FBlock::Table(FTable { rows, widths, borderless: true, extra: None })); + self.body.push(FBlock::Table(FTable { rows, widths, borderless: true, extra: None, kept: None })); } } } diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 1cd6a15ecf..75eb31434f 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity with Microsoft Word -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-11-03 · **Change:** minor (ODT: unedited paragraphs and style definitions written as read); previously trivial (ODT author fields said not fixed); previously trivial (ODT default text colour; covered cells' content); previously trivial (ODT widow control profile case; tabs from the indent kept); previously trivial (ODT span kerning; note reference formatting; notes starting with a tab); previously trivial (ODT fixed and sender fields); previously trivial (ODT: table styles saved into their cells; ODT tables carry no Word table style; section column balance); previously trivial (ODT interpretation profile: the standard is law, what it leaves open (including unstated page margins) follows WordCraft, LibreOffice or custom settings); previously trivial (ODT form controls keep their place on the line and text areas their lines; custom shapes keep their geometry); previously trivial (ODT in-line frames placed against the text; in-line pictures saved standing on the baseline); previously trivial (ODT charts written back as they were while unchanged; frames' drawn outline; default header height); previously trivial (ODT header rows only where they repeat); previously trivial (ODT padding without a border; unequal columns; column breaks); previously trivial (ODT lines of only pictures laid out as LibreOffice does; default page without a layout); previously trivial (ODT pictures alone in a paragraph keep its size; in-line frames raised under an inherited paragraph relation); previously trivial (ODT tables fit the margins and keep their space above and below; header spacing kept); previously trivial (ODT frames anchored and placed by their style); previously trivial (ODT default font size saved; page-anchored frames placed from the page; text box paragraph styles kept; empty paragraphs keep their size); previously trivial (ODT default font size; locale-ordered dates); previously trivial (ODT page background pictures); previously trivial (ODT tables fixed layout; tables starting a page); previously trivial (emphasis marks: model, layout, DOCX `w:em`, ODT `style:text-emphasize`, `format.emphasisMark`); previously trivial (ODT variables as SET/REF fields; character styles keep what they turn off); previously trivial (ODT styles keep what they turn off; WMF/EMF pictures kept; pictures in text boxes shown by LibreOffice); previously trivial (ODT unstated page margins are LibreOffice's 2 cm; drawings outside a paragraph join the next one); previously trivial (ODT placeholders as content controls, template dates, user name fields, unsized pages, style-only templates, found with LibreOffice's own templates); previously trivial (ODT hidden form controls kept; `.ott` templates opened and saved); previously trivial (ODT fidelity re-estimated at ~90%; a kept form control made another kind is saved as that kind); previously trivial (ODT form control sizes and buttons kept); previously trivial (ODT turned text in cells and text boxes); previously trivial (ODT macros and unknown package parts kept); previously trivial (ODT table cell formulas as formula fields); previously trivial (ODT custom shapes drawn from their own geometry); previously trivial (ODT bibliography marks as citations of sources); previously trivial (ODT linked text boxes); previously trivial (ODT picture crops and contour wrapping); previously trivial (ODT fidelity re-estimated at ~85%); previously trivial (ODT custom document properties and user fields as DOCPROPERTY); previously trivial (ODT connectors read and written glued); previously trivial (ODT borders around text read and written); previously trivial (ODT index marks read and written as XE fields; an index held around its entries updates); previously trivial (ODT ruby read and written); previously trivial (ODT page breaks after a paragraph, kept lines, widow control, hyphenation, table alignment and width and row heights); previously trivial (in-line drawings raised or lowered by their run's position; ODT in-line frames keep their place on the baseline; leading's space after the last line; text box borders shown by LibreOffice); previously trivial (ODT form controls opened and saved as content controls); previously trivial (password-protected ODT opened and saved); previously trivial (ODT headers and footers keep their shapes, text boxes, charts and embedded objects on save); previously trivial (ODT embedded spreadsheets and OLE objects kept whole instead of lost; flat ODT sub-documents valid `office:document`s); previously trivial (ODT charts in drawing groups drawn on open); previously trivial (ODT charts opened and saved as editable charts; flat ODT sub-documents no longer end the body); previously trivial (ODT leading read as an at-least spacing); previously trivial (ODT tracked formatting changes; changed regions named by `xml:id`); previously trivial (ODT drop caps; tab stops no longer drop a paragraph's background); previously trivial (flat ODT `.fodt` opened and saved; ODT fidelity re-estimated at ~80%); previously trivial (ODT page settings; a footnote no longer takes a footer's text); previously trivial (ODT dates in their formats, sequences, chapters and references); previously trivial (ODT bookmarks over text); previously trivial (ODT tables of contents and indexes; ODT fidelity re-estimated); previously trivial (ODT polygons, polylines and paths); previously trivial (ODT arrowheads, dashes and gradients); previously trivial (ODT groups of shapes and pictures); previously trivial (ODT rotated and mirrored shapes and pictures); previously trivial (ODT drawing shapes read and written); previously trivial (ODT text box frames keep their line, fill, padding and fit); previously trivial (ODT Asian and complex-script fonts); previously trivial (ODT right-to-left paragraphs); previously trivial (ODT text boxes kept); previously trivial (ODT unnumbered headings detached from the outline); previously trivial (ODT contextual spacing and layout settings); previously trivial (ODT custom character styles kept); previously trivial (ODT list level positions); previously trivial (ODT custom paragraph styles kept); previously trivial (ODT equations written as formula objects); previously trivial (ODT numbered headings written as headings); previously trivial (ODT floating pictures); previously trivial (ODT paragraph borders and background); previously trivial (ODT heading numbering); previously trivial (ODT font-face names); previously trivial (ODT table cell borders and padding); previously trivial (ODT heading, title, quote and preformatted styles keep their look); previously trivial (ODT tab stops read and written); previously trivial (ODT character formatting read and written); previously trivial (ODT lists keep their number formats, bullets and identity); previously trivial (ODT comments read and written); previously trivial (ODT footnotes and endnotes read and written); previously trivial (ODT headers, footers and text fields written; fields read); previously trivial (ODT paragraph spacing, indents and line spacing read and written); previously trivial (a paragraph naming an undefined style is formatted as Normal, as in Word, #5077); previously trivial (a hyperlinked TOC writes its page numbers as `PAGEREF _Toc… \h` fields, as Word does, #5097); previously trivial (HTML import ends the empty comments and at once, #5040); previously trivial (chart and SmartArt text the font lacks is drawn in a fallback face, so Chinese labels no longer show boxes, #4720); previously trivial (bibliography sources keep a corporate author (`b:Corporate`): read, written, cited and listed whole, #4848); previously trivial (custom-property bookmark-link preservation, #4767); previously trivial (note conversion preserves content with an ordered reference/run walk, #4750); previously trivial (DOCX comment-ID save lookups); previously trivial (built-in heading styles have their outline level without `w:outlineLvl`, #4696); previously trivial (DOCX comment ranges inside footnotes and endnotes are kept on read, #4548); previously trivial (RTF import and export keep Windows and Enhanced Metafile pictures (`\wmetafile`, `\emfblip`), #4259); previously trivial (#4658 bounds nested Markdown reference-label work); previously trivial (HTML import keeps preformatted lines with CR or CRLF input newlines, #4810); previously trivial (DOCX parts stored under another casing keep their embedded fonts through a save, and a private save of a kept glossary escapes the attribute values it rewrites; previously trivial: the `.doc` reader logs what its copy budget leaves out, once a kind per document; importers bound hostile expansion: font names, links, ODS repeats, LaTeX wrappers, Markdown delimiters, #3692/#3713; previously trivial: repeated DOCX style IDs: save/reopen regression; previously trivial: Markdown code spans keep line-broken text literal, #3738; previously trivial: Arabic list labels: `arabicAlpha` and `arabicAbjad` draw the 28 Arabic letters (alphabetical and abjad order), doubling past the last letter; Hebrew, Hindi, Thai and Vietnamese still show decimal (`doc::section::tests::arabic_letters_count_in_both_orders`); previously trivial: localized TOC style ids read as the TOC styles, and a TOC in columns keeps its layout through Update Table, #3889; previously trivial: cu20we: outline caps; previously trivial: a relationship the markup names without the prefix is read as it wrote: an unqualified `id` on an element that carries a relationship names that relationship, and the writer keeps the attribute with the id the new package holds the part under; an ordinary `id` — a VML shape's, a drawing's — names no part; previously trivial: the reader takes the relationship ids the markup refers to from the relationship attributes of the main document it has already parsed, so a document with many relationships opens in the time its part takes, #3411; previously trivial: a table cell span from a `.json` file stays within 63 columns through Insert Column and Merge Cells, #3637; previously trivial: the file commands that change the document are checked on a protected document, the protection of a protected document needs an explicit stop, the session author can't change through `file.setAuthor` under protection (the CLI's `--author` uses the same protection check), and the post-check compares the settings and the metadata; previously trivial: a document with many content controls saves in the time the controls take, #13; previously trivial: `majorBidi`/`minorBidi` theme fonts follow the settings' theme font language, as measured in Word; previously trivial: Word signature lines are kept on save, still signable; previously trivial: Times New Roman keeps Liberation Serif under the roman-font rule; previously trivial: shape gradients and transparency, #1500; previously trivial: a folder that keeps no Unix modes, such as a NAS share mounted by GVFS, still takes a save, #3003; previously trivial: a text frame in the paragraph defaults frames nothing, #2360; previously trivial: table style band sizes written, a style without them unbanded as in Word, #3415; previously trivial: HTML export writes ruby (furigana, phonetic guides) and HTML import reads it back, #1681; previously trivial: RTF import decodes the East Asian multibyte code pages, #1653; previously trivial: PDF export: lam-alef drawn as two glyphs at one x extracts in reading order, #2310; previously trivial: native .json stores pictures and kept parts as base64, #586 follow-up; previously trivial: ODT headers and footers read from each master page, #863; previously trivial: RTF import compares heading and code formatting with the file's stylesheet; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: Word XML and web pages saved as `.doc` open by what they hold, #2853; previously trivial: a VML shape in a header cell is always placed on the page, user-checked in Word for Mac, #1775; previously trivial: `w:sym` characters keep their symbol font when the text's font changes and are written back as `w:sym`, #2036; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: two lines in one and horizontal in vertical (`w:eastAsianLayout`) kept on save; previously trivial: PDF outline targets follow leading page breaks; previously trivial: positional tabs `w:ptab` read and kept on save, #2367; previously trivial: DOCX: mixed right-to-left runs split and marked `w:rtl` with their language, #1543; previously trivial: `w:doNotTrackMoves` round-trips and turns move tracking off; previously trivial: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: DOCX glossary documents (building blocks) read, kept and written; previously trivial: cached field results keep their visible run formatting; previously trivial: cached REF results retain hyperlink destinations; previously trivial: CLI conversions can explicitly refresh fields; previously trivial: a copied equation also goes in the clipboard's MathML formats, #1297; previously trivial: a pasted HTML `
` is an indented paragraph, not the Quote style, #1270); previously trivial: every table style part round-trips, #714; previously trivial: VML freeforms read and kept on save; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: `flowChartOr` draws its circle and cross spokes, as in Word; previously trivial: RTF: Mac OS Roman and DOS text, #1653; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: PDF export draws horizontal and vertical lines inside groups, #1474; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-11-03 · **Change:** minor (ODT: unedited tables, pages, sections, properties and declarations written as read; no-edit loss gate); previously minor (ODT: unedited paragraphs and style definitions written as read); previously trivial (ODT author fields said not fixed); previously trivial (ODT default text colour; covered cells' content); previously trivial (ODT widow control profile case; tabs from the indent kept); previously trivial (ODT span kerning; note reference formatting; notes starting with a tab); previously trivial (ODT fixed and sender fields); previously trivial (ODT: table styles saved into their cells; ODT tables carry no Word table style; section column balance); previously trivial (ODT interpretation profile: the standard is law, what it leaves open (including unstated page margins) follows WordCraft, LibreOffice or custom settings); previously trivial (ODT form controls keep their place on the line and text areas their lines; custom shapes keep their geometry); previously trivial (ODT in-line frames placed against the text; in-line pictures saved standing on the baseline); previously trivial (ODT charts written back as they were while unchanged; frames' drawn outline; default header height); previously trivial (ODT header rows only where they repeat); previously trivial (ODT padding without a border; unequal columns; column breaks); previously trivial (ODT lines of only pictures laid out as LibreOffice does; default page without a layout); previously trivial (ODT pictures alone in a paragraph keep its size; in-line frames raised under an inherited paragraph relation); previously trivial (ODT tables fit the margins and keep their space above and below; header spacing kept); previously trivial (ODT frames anchored and placed by their style); previously trivial (ODT default font size saved; page-anchored frames placed from the page; text box paragraph styles kept; empty paragraphs keep their size); previously trivial (ODT default font size; locale-ordered dates); previously trivial (ODT page background pictures); previously trivial (ODT tables fixed layout; tables starting a page); previously trivial (emphasis marks: model, layout, DOCX `w:em`, ODT `style:text-emphasize`, `format.emphasisMark`); previously trivial (ODT variables as SET/REF fields; character styles keep what they turn off); previously trivial (ODT styles keep what they turn off; WMF/EMF pictures kept; pictures in text boxes shown by LibreOffice); previously trivial (ODT unstated page margins are LibreOffice's 2 cm; drawings outside a paragraph join the next one); previously trivial (ODT placeholders as content controls, template dates, user name fields, unsized pages, style-only templates, found with LibreOffice's own templates); previously trivial (ODT hidden form controls kept; `.ott` templates opened and saved); previously trivial (ODT fidelity re-estimated at ~90%; a kept form control made another kind is saved as that kind); previously trivial (ODT form control sizes and buttons kept); previously trivial (ODT turned text in cells and text boxes); previously trivial (ODT macros and unknown package parts kept); previously trivial (ODT table cell formulas as formula fields); previously trivial (ODT custom shapes drawn from their own geometry); previously trivial (ODT bibliography marks as citations of sources); previously trivial (ODT linked text boxes); previously trivial (ODT picture crops and contour wrapping); previously trivial (ODT fidelity re-estimated at ~85%); previously trivial (ODT custom document properties and user fields as DOCPROPERTY); previously trivial (ODT connectors read and written glued); previously trivial (ODT borders around text read and written); previously trivial (ODT index marks read and written as XE fields; an index held around its entries updates); previously trivial (ODT ruby read and written); previously trivial (ODT page breaks after a paragraph, kept lines, widow control, hyphenation, table alignment and width and row heights); previously trivial (in-line drawings raised or lowered by their run's position; ODT in-line frames keep their place on the baseline; leading's space after the last line; text box borders shown by LibreOffice); previously trivial (ODT form controls opened and saved as content controls); previously trivial (password-protected ODT opened and saved); previously trivial (ODT headers and footers keep their shapes, text boxes, charts and embedded objects on save); previously trivial (ODT embedded spreadsheets and OLE objects kept whole instead of lost; flat ODT sub-documents valid `office:document`s); previously trivial (ODT charts in drawing groups drawn on open); previously trivial (ODT charts opened and saved as editable charts; flat ODT sub-documents no longer end the body); previously trivial (ODT leading read as an at-least spacing); previously trivial (ODT tracked formatting changes; changed regions named by `xml:id`); previously trivial (ODT drop caps; tab stops no longer drop a paragraph's background); previously trivial (flat ODT `.fodt` opened and saved; ODT fidelity re-estimated at ~80%); previously trivial (ODT page settings; a footnote no longer takes a footer's text); previously trivial (ODT dates in their formats, sequences, chapters and references); previously trivial (ODT bookmarks over text); previously trivial (ODT tables of contents and indexes; ODT fidelity re-estimated); previously trivial (ODT polygons, polylines and paths); previously trivial (ODT arrowheads, dashes and gradients); previously trivial (ODT groups of shapes and pictures); previously trivial (ODT rotated and mirrored shapes and pictures); previously trivial (ODT drawing shapes read and written); previously trivial (ODT text box frames keep their line, fill, padding and fit); previously trivial (ODT Asian and complex-script fonts); previously trivial (ODT right-to-left paragraphs); previously trivial (ODT text boxes kept); previously trivial (ODT unnumbered headings detached from the outline); previously trivial (ODT contextual spacing and layout settings); previously trivial (ODT custom character styles kept); previously trivial (ODT list level positions); previously trivial (ODT custom paragraph styles kept); previously trivial (ODT equations written as formula objects); previously trivial (ODT numbered headings written as headings); previously trivial (ODT floating pictures); previously trivial (ODT paragraph borders and background); previously trivial (ODT heading numbering); previously trivial (ODT font-face names); previously trivial (ODT table cell borders and padding); previously trivial (ODT heading, title, quote and preformatted styles keep their look); previously trivial (ODT tab stops read and written); previously trivial (ODT character formatting read and written); previously trivial (ODT lists keep their number formats, bullets and identity); previously trivial (ODT comments read and written); previously trivial (ODT footnotes and endnotes read and written); previously trivial (ODT headers, footers and text fields written; fields read); previously trivial (ODT paragraph spacing, indents and line spacing read and written); previously trivial (a paragraph naming an undefined style is formatted as Normal, as in Word, #5077); previously trivial (a hyperlinked TOC writes its page numbers as `PAGEREF _Toc… \h` fields, as Word does, #5097); previously trivial (HTML import ends the empty comments and at once, #5040); previously trivial (chart and SmartArt text the font lacks is drawn in a fallback face, so Chinese labels no longer show boxes, #4720); previously trivial (bibliography sources keep a corporate author (`b:Corporate`): read, written, cited and listed whole, #4848); previously trivial (custom-property bookmark-link preservation, #4767); previously trivial (note conversion preserves content with an ordered reference/run walk, #4750); previously trivial (DOCX comment-ID save lookups); previously trivial (built-in heading styles have their outline level without `w:outlineLvl`, #4696); previously trivial (DOCX comment ranges inside footnotes and endnotes are kept on read, #4548); previously trivial (RTF import and export keep Windows and Enhanced Metafile pictures (`\wmetafile`, `\emfblip`), #4259); previously trivial (#4658 bounds nested Markdown reference-label work); previously trivial (HTML import keeps preformatted lines with CR or CRLF input newlines, #4810); previously trivial (DOCX parts stored under another casing keep their embedded fonts through a save, and a private save of a kept glossary escapes the attribute values it rewrites; previously trivial: the `.doc` reader logs what its copy budget leaves out, once a kind per document; importers bound hostile expansion: font names, links, ODS repeats, LaTeX wrappers, Markdown delimiters, #3692/#3713; previously trivial: repeated DOCX style IDs: save/reopen regression; previously trivial: Markdown code spans keep line-broken text literal, #3738; previously trivial: Arabic list labels: `arabicAlpha` and `arabicAbjad` draw the 28 Arabic letters (alphabetical and abjad order), doubling past the last letter; Hebrew, Hindi, Thai and Vietnamese still show decimal (`doc::section::tests::arabic_letters_count_in_both_orders`); previously trivial: localized TOC style ids read as the TOC styles, and a TOC in columns keeps its layout through Update Table, #3889; previously trivial: cu20we: outline caps; previously trivial: a relationship the markup names without the prefix is read as it wrote: an unqualified `id` on an element that carries a relationship names that relationship, and the writer keeps the attribute with the id the new package holds the part under; an ordinary `id` — a VML shape's, a drawing's — names no part; previously trivial: the reader takes the relationship ids the markup refers to from the relationship attributes of the main document it has already parsed, so a document with many relationships opens in the time its part takes, #3411; previously trivial: a table cell span from a `.json` file stays within 63 columns through Insert Column and Merge Cells, #3637; previously trivial: the file commands that change the document are checked on a protected document, the protection of a protected document needs an explicit stop, the session author can't change through `file.setAuthor` under protection (the CLI's `--author` uses the same protection check), and the post-check compares the settings and the metadata; previously trivial: a document with many content controls saves in the time the controls take, #13; previously trivial: `majorBidi`/`minorBidi` theme fonts follow the settings' theme font language, as measured in Word; previously trivial: Word signature lines are kept on save, still signable; previously trivial: Times New Roman keeps Liberation Serif under the roman-font rule; previously trivial: shape gradients and transparency, #1500; previously trivial: a folder that keeps no Unix modes, such as a NAS share mounted by GVFS, still takes a save, #3003; previously trivial: a text frame in the paragraph defaults frames nothing, #2360; previously trivial: table style band sizes written, a style without them unbanded as in Word, #3415; previously trivial: HTML export writes ruby (furigana, phonetic guides) and HTML import reads it back, #1681; previously trivial: RTF import decodes the East Asian multibyte code pages, #1653; previously trivial: PDF export: lam-alef drawn as two glyphs at one x extracts in reading order, #2310; previously trivial: native .json stores pictures and kept parts as base64, #586 follow-up; previously trivial: ODT headers and footers read from each master page, #863; previously trivial: RTF import compares heading and code formatting with the file's stylesheet; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: Word XML and web pages saved as `.doc` open by what they hold, #2853; previously trivial: a VML shape in a header cell is always placed on the page, user-checked in Word for Mac, #1775; previously trivial: `w:sym` characters keep their symbol font when the text's font changes and are written back as `w:sym`, #2036; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: two lines in one and horizontal in vertical (`w:eastAsianLayout`) kept on save; previously trivial: PDF outline targets follow leading page breaks; previously trivial: positional tabs `w:ptab` read and kept on save, #2367; previously trivial: DOCX: mixed right-to-left runs split and marked `w:rtl` with their language, #1543; previously trivial: `w:doNotTrackMoves` round-trips and turns move tracking off; previously trivial: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: DOCX glossary documents (building blocks) read, kept and written; previously trivial: cached field results keep their visible run formatting; previously trivial: cached REF results retain hyperlink destinations; previously trivial: CLI conversions can explicitly refresh fields; previously trivial: a copied equation also goes in the clipboard's MathML formats, #1297; previously trivial: a pasted HTML `
` is an indented paragraph, not the Quote style, #1270); previously trivial: every table style part round-trips, #714; previously trivial: VML freeforms read and kept on save; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: `flowChartOr` draws its circle and cross spokes, as in Word; previously trivial: RTF: Mac OS Roman and DOS text, #1653; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: PDF export draws horizontal and vertical lines inside groups, #1474; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats Every format Word opens or saves, what WordCraft does with it, and how it's tested. Fidelity is the share of a typical real-world file's content and formatting that survives (read: shows correctly; @@ -285,6 +285,7 @@ Custom document properties retain their bookmark link targets through DOCX saves | 2026-11-02 | trivial | ODT: a chart lost the look WordCraft doesn't model (its fonts and sizes, axis and grid lines) on save, written anew from WordCraft's chart. The chart object as read is kept (through drawing it again when it opens or is resized), and written back whole while the chart reads the same; an edited chart is written from WordCraft's chart as before. The chart documents round-trip pixel-identical (`charts::an_unchanged_chart_is_written_back_as_it_was_and_an_edited_one_anew`, `engine::tests::redrawing_a_chart_keeps_the_chart_as_read`). Found with LibreOffice's serif business letter: a text frame outlined by its drawn stroke (`draw:stroke`, no `fo:border`) lost its outline; it's the frame's line. A header or footer whose style states no height (or that has none) was 0 tall; it's LibreOffice's 0.5 cm (measured), so the text no longer moves up. The letter round-trips to 116 differing pixels (from 4,223) (`text_boxes::a_frames_drawn_stroke_is_its_line_when_it_has_no_border`, `headers::a_header_without_a_stated_height_is_half_a_centimetre`) | | 2026-11-02 | trivial | ODT, found with LibreOffice's agendas: saving wrote every table's first row as a header row (`table:table-header-rows`), repeated at the top of each page, because a table style's header row (on by default in Word's table look) was taken for Repeat as header row; a table crossing a page showed its first row twice. Only rows set to repeat are written so. The agendas round-trip to about 145 differing pixels (from 5,269 and 5,086), their text the same (`tables::only_rows_set_to_repeat_are_written_as_header_rows`) | | 2026-11-02 | trivial | ODT, found with LibreOffice's report templates and one-page résumé. A paragraph's padding on a side without a border (measured: LibreOffice gives it room) was dropped; it's kept, laid out (see layout-parity) and written back. Unequal columns (`style:column` `style:rel-width` and indents, on pages and sections) were made equal; they keep their widths, written back as shares, without the gap LibreOffice ignores them beside. Columns with no stated gap have none (it had been Word's 0.5 in). A paragraph starting a column (`fo:break-before="column"`) didn't; it starts with Word's column break, written back on the paragraph. The report templates round-trip to 0–11 differing pixels (from 9,600), the résumé to 24 (from 6,286) (`para_borders::padding_without_a_border_keeps_its_room`, `odt::tests::unequal_columns_keep_their_widths_and_columns_without_a_gap_have_none`, `odt::tests::a_paragraph_starting_a_column_starts_with_a_column_break`) | +| 2026-11-03 | minor | ODT: more is written back as read when unedited — tables, the document's properties WordCraft doesn't model (template, statistics, editing time, printing), the declarations opening the text and the document's scripts, page layouts and master pages while the page setup is unchanged (with the master page each section started with), and named sections around unedited paragraphs (start tag and section style as read). A file's package files and folders that kept markup names (a style's fill picture, a header's embedded object) are written with it, with their manifest entries. Files are saved as ODF 1.3; markup kept from an older file is brought to 1.3's meaning (a gradient's or hatch's unitless angle was tenths of a degree). WordCraft's own drawings state `style:flow-with-text`, which the standard leaves open. A test now saves seven samples of our own without edits and finds nothing lost (`odt::tests_more::lossless`). Over the 133-file corpus: round-trip difference 191k → 51k pixels, pixel-identical files 62 → 95, lost element kinds 65 → 38. | | 2026-11-03 | minor | ODT: what isn't edited is written back as it was read. A paragraph nothing changed (by a fingerprint of how it was modelled) is written with its own markup, unknown elements and attributes included, with the automatic styles, common styles, font faces, namespaces and package files it names; while no style or list definition changed, `styles.xml`'s styles are the file's own (unused list styles, note and line numbering configuration, attributes WordCraft doesn't read). Paragraphs with drawings, comments, tracked changes, ranged marks, or that start a page are written from the model; the default graphic style and drawing definitions are WordCraft's, as drawings are written from the model. Kept markup stays in memory only (`wordcraft_doc::kept`, 64 MB at most). Round-trip differences over the 133-file corpus 528k → 191k pixels, pixel-identical files 46 → 62. | | 2026-11-03 | trivial | ODT: an author field that follows the document is written with `text:fixed="false"`, as LibreOffice writes it; LibreOffice reads one without the attribute as fixed (§19.807 says not fixed), so an empty author field stayed empty instead of showing the author. | | 2026-11-03 | trivial | ODT, found with Google Docs' and ONLYOFFICE's files: the default paragraph style's text colour (`fo:color`, Google's grey résumé) is read into the document's default and written back (it came back black; round trip 9.2k → 3.4k differing pixels); what a covered table cell holds (§9.1.5: not shown) is skipped instead of landing before the table as empty paragraphs, which pushed ONLYOFFICE's table down the page. | diff --git a/docs/odf-interpretation.md b/docs/odf-interpretation.md index 1710b9f055..fdf2bea860 100644 --- a/docs/odf-interpretation.md +++ b/docs/odf-interpretation.md @@ -80,12 +80,24 @@ These are documented so they aren't mistaken for law: ## How WordCraft writes ODF -What nobody edited is written back as it was read: a paragraph WordCraft hasn't changed keeps -its own markup (with what it names: its automatic styles, common styles, font faces and package -files), and while no style or list definition changed, the file's own style definitions are -written. So something WordCraft doesn't model survives a save that doesn't touch it. Paragraphs -holding drawings, comments, tracked changes or ranged marks, or starting a page, are written -from the model, as is everything edited. +What nobody edited is written back as it was read, so something WordCraft doesn't model survives +a save that doesn't touch it: + +- a paragraph or table WordCraft hasn't changed keeps its own markup, with what it names (its + automatic styles, common styles, font faces, and the package files and folders they refer to); +- while no style or list definition changed, the file's own style definitions are written; +- while the page setup didn't change, its page layouts and master pages, and each section starts + with the master page it started with; +- named sections around unedited paragraphs, the document properties WordCraft doesn't model, + the declarations opening the text and the document's scripts. + +Paragraphs and tables holding drawings, comments, tracked changes or ranged marks are written from +the model, as is everything edited. `odt::tests_more::lossless` saves samples without edits and +fails on anything lost. + +Files are saved as ODF 1.3, whatever version they were read as (as LibreOffice does). Markup kept +from an older file is brought to 1.3's meaning where it changed: a gradient's, transparency +gradient's or hatch's angle without a unit was tenths of a degree before 1.3 (§19.112, §19.209). What WordCraft writes from its model, it writes so that every reader takes it the same way. It states what the standard leaves open instead of relying on any application's default: @@ -93,6 +105,7 @@ states what the standard leaves open instead of relying on any application's def - header and footer heights, with `style:dynamic-spacing="false"` when there is spacing; - a frame's placement as an explicit position against the page or paragraph; - in-line pictures standing on the baseline; +- drawings not held in the text's area (`style:flow-with-text="false"`, §20.267 gives no default); - tab stops measured as the file it was opened from measured them (`TabsRelativeToIndent` in `settings.xml`), from the margin for WordCraft's own documents; - widow and orphan control, on or off, in the default paragraph style. @@ -144,6 +157,7 @@ one more `OdfPreset` with its values, and the cases above stay the same. | Date | Change | |---|---| +| 2026-11-03 | More written as read (tables, pages, sections, properties, declarations); files saved as ODF 1.3 with older markup brought to 1.3's meaning; drawings state `style:flow-with-text`. | | 2026-11-03 | Unedited content written as read (see How WordCraft writes ODF). | | 2026-11-03 | Tabs measured from the indent: default stops count from the indent too (measured in LibreOffice), in WordCraft's layout, and a file measuring tabs so is written back so (it was written from the margin, moving LibreOffice's default stops). | | 2026-11-03 | Widow and orphan control a file doesn't state (`widowControl`): WordCraft as XSL's initial value, LibreOffice off. WordCraft writes it on its default style either way. |