From ca39b5ab9f4966d593118f3ac85f2422d63088f5 Mon Sep 17 00:00:00 2001 From: neilcawse Date: Tue, 10 Nov 2026 06:43:07 -0500 Subject: [PATCH 1/6] XLSX: legacy formulas (no dynamic-array `cm`) open with Excel's implicit `@` and calculate like Excel (#417) * XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel; implied @ saved back without cm Co-Authored-By: Claude Opus 5.5 * XLSX: long operator chains analysed in full and unfinished legacy analysis leaves the formula unchanged (never saved as SINGLE); @ of a union keeps its parentheses; IF's condition intersected inside SUMPRODUCT * XLSX: a legacy union already in parentheses saves back without a second pair (never as SINGLE); a name's definition analysed as an array throughout, inserting no @ * XLSX: drop the unreachable keep-unions retry; an @ of a bare union saves as a dynamic formula * XLSX: an @ keeps parentheses the reader would not put back (redundant ones, a union's) * XLSX: a redundant @ keeps a legacy formula legacy; IFERROR and IFNA intersected inside SUMPRODUCT; file parentheses and unfinished analyses saved as read An edit could leave an implied @ on a single cell: deleting rows 2:3 under the legacy SQRT(A1:A3), read as SQRT(@A1:A3), gives SQRT(@A1:A1), and a name redefined to one cell does the same to SQRT(@Rng). The writer kept an @ only when re-analysing the formula without it put the same @ back, which it no longer does for one cell, so the formula was saved as SQRT(_xlfn.SINGLE(A1:A1)) with cm, readable only by dynamic-array Excel. The writer now analyses the formula with its own @ in place: an @ is accepted when the analysis implies it at that position or when its operand is a single value there (such an @ changes nothing), and any @ inserted elsewhere or any other @ still makes the formula dynamic. SQRT(@A1:A1) saves as SQRT(A1:A1), SQRT(@A1) as SQRT(A1) and @SUM(A1:A3) as SUM(A1:A3), all without cm; SUMPRODUCT(@SQRT(A1:A3)) stays dynamic because SQRT(A1:A3) is an array there. Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT unless the formula is entered with Ctrl+Shift+Enter, like IF's condition: =SUMPRODUCT(IFERROR(VALUE(C19:C21),0)) returns one row's value or #VALUE!, and the accepted fix is CSE or N(+range) (https://stackoverflow.com/questions/59228707). They now get the same single-value parameter class as IF's condition (Param::Single), so the legacy SUMPRODUCT(IFERROR(A1:A3/A1:A3,0)) reads as SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0)), and a typed SUMPRODUCT(IFERROR(VALUE(A1:A3),0)), which dynamic-array Excel evaluates as an array, is saved with cm. IF's result arguments are left as arrays: in https://stackoverflow.com/questions/29682328, a non-CSE =SUMPRODUCT(--(B4=IF(NOT($C$4:$C$11),$B$4:$B$11))) in row 4 returns 2 (both "apple" rows of B4:B11 counted, so the result branch was not intersected), 0 in a row where C is TRUE and #VALUE! outside rows 4:11 (the condition was intersected). The reader's @ in front of a whole-formula array result took no extra parentheses when the file already had them, so ({1,2}+1) and {1,2}+1 both read as @({1,2}+1), and the writer dropped the file's parentheses. An operand already in parentheses now gets a second pair, so ({1,2}+1) saves as written. A legacy formula the analysis could not finish (node budget, nesting) was read unchanged, evaluated as a dynamic-array formula and saved with cm, so Excel spilled it. Names were analysed again at every use, so a 399-term name used six times exhausted the 4096-node budget; their kinds are now remembered per formula, and the budget is 16384 nodes. The nesting limit is 128, so Excel's 64 levels of nested functions are analysed. A formula that still exceeds the limits is saved as it was read, without cm. * XLSX: an implied @ is dropped and any other @ of a single value kept as _xlfn.SINGLE in a legacy formula (as Excel writes it); a legacy formula the analysis cannot finish or whose semantics it does not know opens with a whole-formula @ and never spills; analysis depth follows the parser's limit * XLSX: a legacy union given to a function's single-value parameter keeps its implied @, so it calculates to #VALUE! instead of spilling and saves as read; checked on a calculated workbook * XLSX: only a one-cell range an @ applies to counts as one an edit shrank; a reading past the parser's nesting limit leaves the formula as read, and the writer saves the text a formula it cannot analyse was read from * XLSX: a formula nested past the parser's limit for its reading is left as read (documented: it may spill); test covers it * XLSX: a name's definition counts its one-cell ranges as collapsed only when it is itself a reference, so CHOOSE's index or IF's condition in it keeps the name legacy * XLSX: an @ on a name or one-cell range counts as implied when ranges an edit may have shrunk, anywhere the name returns them (CHOOSE's choices, parentheses), make it so; deleting rows keeps such formulas legacy * XLSX: the collapsed-range retry of an @ runs only when the first pass reached a one-cell range written as a range (tracked through cached names) * docs: describe unfinished formula export fallback * docs: the unfinished-analysis fallback differs for formulas with an explicit @ * Preserve main formula analyzer fixes * Order file-format revision history --------- Co-authored-by: neilcawse <8085959+neilcawse@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 Co-authored-by: Local contributor --- crates/calc/src/tests.rs | 37 +- crates/engine/src/io.rs | 22 +- crates/engine/tests/legacy_array_functions.rs | 335 +++++++ crates/formula/src/parser.rs | 6 +- crates/formula/src/tests.rs | 9 + crates/xlsx/src/array_formula.rs | 883 ++++++++++++++---- crates/xlsx/src/fmla.rs | 19 +- crates/xlsx/src/lib.rs | 95 +- crates/xlsx/src/read.rs | 15 +- crates/xlsx/src/sheet_read.rs | 24 +- crates/xlsx/src/sheet_write.rs | 25 +- crates/xlsx/src/tests/array_export.rs | 19 +- crates/xlsx/src/tests/fixtures.rs | 86 ++ crates/xlsx/src/tests/roundtrip.rs | 255 ++++- crates/xlsx/src/write.rs | 15 +- docs/decisions.md | 4 +- docs/file-format-parity.md | 2 + docs/function-parity.md | 3 + 18 files changed, 1635 insertions(+), 219 deletions(-) create mode 100644 crates/engine/tests/legacy_array_functions.rs diff --git a/crates/calc/src/tests.rs b/crates/calc/src/tests.rs index 083a5ff25..b4f820f0c 100644 --- a/crates/calc/src/tests.rs +++ b/crates/calc/src/tests.rs @@ -1922,6 +1922,41 @@ fn offset_extreme_arguments_return_ref() { } } +/// `@` intersects a reference with the formula's row (a column range) or column (a row range); +/// a 2D range or one the formula does not line up with gives #VALUE!. An array gives its +/// top-left value. Legacy formulas read from files use it where Excel intersects implicitly. +#[test] +fn implicit_intersection_operator() { + let mut t = T::new(); + for (at, v) in [("A1", "1"), ("A2", "2"), ("A3", "3"), ("A5", "4"), ("B5", "5"), ("C5", "6")] { + t.set(at, v); + } + t.set("B2", "=TANH(@A1:A3)"); + t.set("B7", "=TANH(@A5:C5)"); + t.set("B3", "=@A1:C3"); + t.set("B9", "=@A1:A3"); + t.set("E9", "=@A5:C5"); + t.set("D1", "=@{7,8;9,10}"); + t.set("D2", "=@A2"); + t.set("D3", "=SUM(@A1:A3)"); + assert_eq!(t.num("B2"), 2f64.tanh()); + assert_eq!(t.num("B7"), 5f64.tanh()); + for at in ["B3", "B9", "E9"] { + assert_eq!(t.get(at), Value::Error(CellError::Value), "{at}"); + } + assert_eq!(t.num("D1"), 7.0); + assert_eq!(t.num("D2"), 2.0); + assert_eq!(t.num("D3"), 3.0); + // An array result shows its top-left value (T of a reference reads its top-left cell). + t.set("A11", "x"); + t.set("C11", "y"); + t.set("E1", "=@T(A11:C11)"); + assert_eq!(t.get("E1"), Value::text("x")); + // Without `@` (a dynamic-array formula) the same formula spills. + t.set("F1", "=TANH(A1:A3)"); + assert_eq!((t.num("F1"), t.num("F2"), t.num("F3")), (1f64.tanh(), 2f64.tanh(), 3f64.tanh())); +} + /// "Set precision as displayed": results are rounded to their cell's format before they are /// stored and before other formulas read them. Values observed in Excel (COM, /// `Workbook.PrecisionAsDisplayed = True`) with the same synthetic formulas. @@ -1987,7 +2022,6 @@ fn precision_as_displayed_rounds_results_by_cell_format() { t.calc.recalc_all(&mut t.wb); assert_eq!(t.num("A2"), (-130.0 + 657.41) * 10.0); } - #[test] fn analysis_toolpak_value_params_reject_multi_cell_references() { // Excel 365 (observed): EOMONTH, WORKDAY, COUPDAYS, BIN2DEC... give #VALUE! for a reference to more @@ -2072,7 +2106,6 @@ fn analysis_toolpak_value_params_reject_multi_cell_references() { t.set("H5", ""); } } - /// A formula reading one further down (or on a later sheet) is calculated on demand, inside the /// formula that reads it. Chains of thousands of such formulas, common in financial models, used /// to overflow the stack (or give `#CIRC!` past 2,000 levels); they now calculate in full. diff --git a/crates/engine/src/io.rs b/crates/engine/src/io.rs index 6cd39f839..22c236e61 100644 --- a/crates/engine/src/io.rs +++ b/crates/engine/src/io.rs @@ -9,6 +9,13 @@ use crate::{EngineError, Result}; mod sylk; +/// The function library's answers to what the XLSX reader and writer need to know about functions +/// (which names are built in, which parameters take a single value). +pub const XLSX_FUNCTIONS: gridcraft_xlsx::FunctionInfo = gridcraft_xlsx::FunctionInfo { + is_builtin: |name| gridcraft_functions::lookup(name).is_some(), + is_scalar_arg: |name, i| gridcraft_functions::lookup(name).is_none_or(|s| s.is_scalar_arg(i)), +}; + /// File formats GridCraft reads or writes. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum FileKind { @@ -289,12 +296,13 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, // Text named `.xlsx` is read as text. if sniffed == gridcraft_xlsx::Format::Xlsx || (kind == Some(FileKind::Xlsx) && sniffed != gridcraft_xlsx::Format::Csv) { // Google Sheets' export wrapper is read as the formula inside when GridCraft calculates it (#2871). - let (wb, report) = gridcraft_xlsx::read_xlsx_with(bytes, &gridcraft_calc::is_known_function).map_err(|e| { - EngineError::Other(match (&e, gridcraft_xlsx::oversized_part(bytes)) { - (gridcraft_xlsx::IoError::TooLarge(_), Some((part, size))) => part_too_large_message(name, &part, size), - _ => format!("We can't open '{name}': {e}"), - }) - })?; + let (wb, report) = gridcraft_xlsx::read_xlsx_with(bytes, (XLSX_FUNCTIONS, &gridcraft_calc::is_known_function as &dyn Fn(&str) -> bool)) + .map_err(|e| { + EngineError::Other(match (&e, gridcraft_xlsx::oversized_part(bytes)) { + (gridcraft_xlsx::IoError::TooLarge(_), Some((part, size))) => part_too_large_message(name, &part, size), + _ => format!("We can't open '{name}': {e}"), + }) + })?; return Ok((wb, report.warnings)); } if let Some(read) = read_html_table(name, kind, bytes, csv_as_text) { @@ -502,7 +510,7 @@ pub fn save_bytes_as(wb: &Workbook, path: &str, kind: FileKind) -> Result named, }; let book = std::path::Path::new(path).file_name().and_then(std::ffi::OsStr::to_str).unwrap_or_default(); - gridcraft_xlsx::write_xlsx_named(named.as_ref().unwrap_or(wb), gridcraft_xlsx::XlsxKind::from_path(path), book) + gridcraft_xlsx::write_xlsx_named_with(named.as_ref().unwrap_or(wb), gridcraft_xlsx::XlsxKind::from_path(path), book, XLSX_FUNCTIONS) .map_err(|e| EngineError::Other(e.to_string())) } FileKind::Xls => { diff --git a/crates/engine/tests/legacy_array_functions.rs b/crates/engine/tests/legacy_array_functions.rs new file mode 100644 index 000000000..1549b6f70 --- /dev/null +++ b/crates/engine/tests/legacy_array_functions.rs @@ -0,0 +1,335 @@ +//! Legacy formulas (no dynamic-array metadata) keep one value through GridCraft (issue #410), on +//! a calculated workbook: +//! - every function that returns several values: a legacy cell formula calling it opens with +//! Excel's implicit `@` and does not spill, and a typed `=@F(…)` saves and reopens with the same +//! single value. The library's functions are found by calling each with sample arguments, so +//! one added later is covered without editing this test. +//! - a legacy formula the analysis cannot finish does not spill and saves as read, without `cm`. + +use std::io::Write; + +use gridcraft_engine::Session; +use gridcraft_engine::core::{CellRef, DateSystem, Value}; +use gridcraft_functions::{Arg, Ctx, FnSpec}; +use serde_json::json; + +struct Probe; + +impl Ctx for Probe { + fn date_system(&self) -> DateSystem { + DateSystem::D1900 + } + fn now_serial(&self) -> f64 { + 45_000.0 + } + fn random(&mut self) -> f64 { + 0.5 + } +} + +/// The 2×2 block of numbers at `A1:B2` in the probe sheets. +fn block() -> Value { + Value::Array(std::sync::Arc::new( + gridcraft_engine::core::Array::new(2, 2, vec![Value::Number(1.0), Value::Number(2.0), Value::Number(3.0), Value::Number(4.0)]).unwrap(), + )) +} + +/// Sample arguments (formula text, value). Single values only at a position the function lifts +/// over, so an array result is the function's own, not lifting. +fn candidates(scalar: bool) -> Vec<(&'static str, Arg)> { + let mut out = vec![ + ("3", Arg::val(Value::Number(3.0))), + ("1", Arg::val(Value::Number(1.0))), + ("\"a1,b22\"", Arg::val(Value::text("a1,b22"))), + ("\",\"", Arg::val(Value::text(","))), + ("\"[0-9]+\"", Arg::val(Value::text("[0-9]+"))), + ("TRUE", Arg::val(Value::Bool(true))), + ]; + if !scalar { + out.insert(0, ("A1:B2", Arg::reference(block()))); + out.insert(1, ("{1,2;3,4}", Arg::val(block()))); + } + out +} + +/// The first sample call of `spec` that returns more than one value, as formula text. +fn array_call(spec: &FnSpec) -> Option { + let max = spec.max_args.min(spec.min_args.max(1) + 2).min(4).max(spec.min_args); + for n in spec.min_args..=max { + let per: Vec<_> = (0..n).map(|i| candidates(spec.is_scalar_arg(i))).collect(); + let per: Vec<_> = if n > 4 { per.into_iter().map(|c| c.into_iter().take(3).collect::>()).collect() } else { per }; + let mut idx = vec![0usize; n]; + let mut tries = 0; + loop { + let args: Vec = idx.iter().zip(&per).map(|(&i, c)| c[i].1.clone()).collect(); + if let Value::Array(a) = gridcraft_functions::call(spec, &args, &mut Probe) + && a.data.len() > 1 + { + let text: Vec<&str> = idx.iter().zip(&per).map(|(&i, c)| c[i].0).collect(); + return Some(format!("{}({})", spec.name, text.join(","))); + } + tries += 1; + // Next combination, last position fastest. + let mut k = n; + loop { + if k == 0 || tries > 3000 { + break; + } + k -= 1; + idx[k] += 1; + if idx[k] < per[k].len() { + break; + } + idx[k] = 0; + if k == 0 { + k = usize::MAX; + break; + } + } + if n == 0 || k == usize::MAX || tries > 3000 { + break; + } + } + } + None +} + +/// Every library function with an array result, as a sample call, and the functions the +/// evaluator implements itself that return several values or a range. +fn array_calls() -> Vec { + let mut calls: Vec = gridcraft_functions::all().iter().filter_map(array_call).collect(); + for found in ["MUNIT", "MODE.MULT", "TEXTSPLIT", "REGEXEXTRACT", "MMULT", "SEQUENCE"] { + assert!(calls.iter().any(|c| c.starts_with(&format!("{found}("))), "{found} not found: {calls:?}"); + } + calls.extend( + ["TRIMRANGE(A1:B2)", "OFFSET(A1:B2,0,0)", "INDIRECT(\"A1:B2\")", "INDEX(A1:B2,0,1)", "FILTERXML(\"12\",\"//b\")"] + .map(String::from), + ); + calls +} + +/// Where the `i`th sample call goes: column E, five rows apart, away from the data in `A1:B2`. +fn slot(i: usize) -> CellRef { + CellRef::new(9 + 5 * i as u32, 4) +} + +fn value(s: &Session, c: CellRef) -> Value { + s.doc().unwrap().wb.active().unwrap().value(c) +} + +/// The cells right of and below `c` are empty and `c` is not #SPILL!: the formula did not spill. +fn one_value(s: &Session, c: CellRef, what: &str) -> Result { + let v = value(s, c); + let right = value(s, CellRef::new(c.row, c.col + 1)); + let below = value(s, CellRef::new(c.row + 1, c.col)); + if right != Value::Empty || below != Value::Empty || v.display().contains("SPILL") { + return Err(format!("{what}: {} spilled ({right:?} right, {below:?} below)", v.display())); + } + Ok(v) +} + +#[test] +fn legacy_formulas_with_array_functions_show_one_value() { + let calls = array_calls(); + let formulas: Vec<_> = calls.iter().enumerate().map(|(i, f)| (slot(i).a1(), f.clone())).collect(); + let s = open(&legacy_xlsx(&formulas, "")); + let wrong: Vec<_> = calls.iter().enumerate().filter_map(|(i, f)| one_value(&s, slot(i), f).err()).collect(); + assert!(wrong.is_empty(), "{wrong:#?}"); +} + +#[test] +fn typed_at_of_array_functions_saves_one_value() { + let calls = array_calls(); + let mut s = session(); + for (a, v) in [("A1", 1), ("B1", 2), ("A2", 3), ("B2", 4)] { + s.execute("cell.set", json!({"cell": a, "input": v.to_string()})).unwrap(); + } + for (i, f) in calls.iter().enumerate() { + s.execute("cell.set", json!({"cell": slot(i).a1(), "input": format!("=@{f}")})).unwrap(); + } + let before: Vec<_> = (0..calls.len()).map(|i| value(&s, slot(i))).collect(); + let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let mut back = Session::new(); + back.execute("file.open", json!({"name": "typed.xlsx", "base64": saved["base64"]})).unwrap(); + let wrong: Vec<_> = calls + .iter() + .enumerate() + .filter_map(|(i, f)| match one_value(&back, slot(i), f) { + Ok(v) if v == before[i] => None, + // A volatile function recalculates on open: one number again. + Ok(Value::Number(_)) if volatile(f) => None, + Ok(v) => Some(format!("@{f}: {} before saving, {} after", before[i].display(), v.display())), + Err(e) => Some(e), + }) + .collect(); + assert!(wrong.is_empty(), "{wrong:#?}"); +} + +fn volatile(call: &str) -> bool { + call.split('(').next().and_then(gridcraft_functions::lookup).is_some_and(|spec| spec.volatile) +} + +/// Formulas the analysis covers at Excel's nesting limits, and one it cannot finish (names too +/// large to analyse, whose ranges would spill three rows): calculated, none spills, and saving +/// writes each as read, as a plain ``. +#[test] +fn deep_and_unfinished_legacy_formulas_calculate_and_save_as_read() { + let nested = format!("{}A1:A3{}", "ABS(".repeat(200), ")".repeat(200)); + let parens = format!("{}A1:A3{}", "ABS((".repeat(64), "))".repeat(64)); + let parts = (1..=30).map(|i| format!("Part_{i}")).collect::>().join("+"); + let part = format!("Sheet1!$A$1:$A$3+{}", vec!["Sheet1!$B$1"; 599].join("+")); + let names: String = (1..=30).map(|i| format!(r#"{part}"#)).collect(); + let formulas = [("E10".to_string(), nested), ("E15".into(), parens), ("E20".into(), parts)]; + let mut s = open(&legacy_xlsx(&formulas, &names)); + for (a, _) in &formulas { + one_value(&s, CellRef::parse(a).unwrap(), a).unwrap(); + } + // Legacy Excel intersects A1:A3 with row 10 and 15: #VALUE!. The unfinished formula shows the + // top-left value of its sum, 1 + 599 × 2 per name. + assert_eq!(value(&s, CellRef::parse("E10").unwrap()).display(), "#VALUE!"); + assert_eq!(value(&s, CellRef::parse("E15").unwrap()).display(), "#VALUE!"); + assert_eq!(value(&s, CellRef::parse("E20").unwrap()), Value::Number(30.0 * (1.0 + 599.0 * 2.0))); + let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap(); + let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); + let mut xml = String::new(); + std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap(); + for (a, f) in &formulas { + assert!(xml.contains(&format!("{f}")), "{a} not saved as read: {xml}"); + } + assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}"); +} + +/// Legacy unions where one value is expected, a name whose definition is IF of a range, and +/// multi-valued results in the file's own parentheses: calculated, none spills (legacy Excel +/// intersects a union to #VALUE!), and saving writes each as read, as a plain ``. +#[test] +fn legacy_unions_names_and_parentheses_calculate_and_save_as_read() { + let formulas: Vec<(String, String)> = ["SQRT((A1:A2,B1:B2))", "ABS(((A1:A2,B1:B2)))", "(A1:A2,B1:B2)+1", "SQRT(Flag)", "({1,2}+1)", "{1,2}+1"] + .iter() + .enumerate() + .map(|(i, f)| (slot(i).a1(), f.to_string())) + .collect(); + let names = r#"IF(Sheet1!$A$1:$A$3>1,1,0)"#; + let mut s = open(&legacy_xlsx(&formulas, names)); + for (a, _) in &formulas { + one_value(&s, CellRef::parse(a).unwrap(), a).unwrap(); + } + let shown: Vec = formulas.iter().map(|(a, _)| value(&s, CellRef::parse(a).unwrap()).display()).collect(); + assert_eq!(shown, ["#VALUE!", "#VALUE!", "#VALUE!", "0", "2", "2"]); + let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap(); + let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); + let mut xml = String::new(); + std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap(); + for (a, f) in &formulas { + assert!(xml.contains(&format!("{f}")), "{a} not saved as read: {xml}"); + } + assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}"); +} + +/// One-cell ranges written as ranges (`B1:B1`) outside any `@`, and formulas whose reading +/// reaches the parser's nesting limit (254 nested functions, far past Excel's 64): saved as read, +/// without `@`. Only a one-cell range under an `@` counts as one an edit may have shrunk. All but +/// the last are calculated without spilling and saved as a plain ``. +#[test] +fn one_cell_ranges_and_formulas_at_the_nesting_limit_save_as_read() { + let deep = |inner: &str| format!("{}{inner}{}", "ABS(".repeat(254), ")".repeat(254)); + let formulas: Vec<(String, String)> = [ + "SQRT(A1:A3)+B1:B1".to_string(), + "A1:A3*B2:B2".into(), + "SQRT(A1:A3)+$B$2:$B$2".into(), + // Names with a one-cell range inside CHOOSE's index or IF's condition. + "SQRT(NC)".into(), + "SQRT(NI)".into(), + "NC+1".into(), + "NI*2".into(), + deep("A1:A3%"), + deep("-A1:A3"), + ] + .into_iter() + .enumerate() + .map(|(i, f)| (slot(i).a1(), f)) + .collect(); + let names = r#"CHOOSE(Sheet1!$B$1:$B$1,Sheet1!$A$1:$A$3,Sheet1!$A$1:$A$3)IF(Sheet1!$B$1:$B$1>0,Sheet1!$A$1:$A$3)"#; + let (last, rest) = formulas.split_last().unwrap(); + let mut s = open(&legacy_xlsx(&formulas, names)); + for (a, _) in rest { + one_value(&s, CellRef::parse(a).unwrap(), a).unwrap(); + } + let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap(); + let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); + let mut xml = String::new(); + std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap(); + for (a, f) in rest { + assert!(xml.contains(&format!("{f}")), "{a} not saved as read: {xml}"); + } + // A reading the parser cannot nest leaves the formula as read: it calculates as a + // dynamic-array formula would, but the file never gets an `@`. + let (a, f) = last; + assert!(xml.contains(&format!("{f}")), "{a} not saved as read: {xml}"); + assert!(!xml.contains('@') && !xml.contains("SINGLE"), "{xml}"); + assert_eq!(xml.matches("cm=").count(), 1, "{xml}"); +} + +/// Deleting rows shrinks the ranges names return (directly, through another name, CHOOSE's and +/// IF's results, parentheses) to one cell: the `@` the reader put on each name is still the one +/// legacy Excel implies, so the formulas save as legacy ones, without `_xlfn.SINGLE`. +#[test] +fn names_shrunk_by_deleting_rows_save_as_legacy() { + let formulas: Vec<(String, String)> = + ["SQRT(MM)", "SQRT(CHM)", "SQRT(NCH)", "SQRT(NIF)", "SQRT(PM)"].into_iter().enumerate().map(|(i, f)| (slot(i).a1(), f.to_string())).collect(); + let names = r#"Sheet1!$A$1:$A$3MMCHOOSE(1,Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)IF(1,Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)(Sheet1!$A$1:$A$3)"#; + let mut s = open(&legacy_xlsx(&formulas, names)); + s.execute("home.deleteRows", json!({"rows": "2:3"})).unwrap(); + let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap(); + let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); + let mut xml = String::new(); + std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap(); + for (a, f) in &formulas { + assert!(xml.contains(&format!("{f}")), "{a} not saved as legacy {f}: {xml}"); + } + assert!(!xml.contains("SINGLE") && !xml.contains("cm="), "{xml}"); +} + +const CT: &str = r#""#; + +/// A one-sheet package: numbers 1–4 in `A1:B2` and `formulas` (address, file text) saved as +/// legacy formulas, without dynamic-array metadata; `names` is `` elements. +fn legacy_xlsx(formulas: &[(String, String)], names: &str) -> Vec { + let esc = |s: &str| s.replace('&', "&").replace('<', "<").replace('>', ">").replace('"', """); + let mut rows = + String::from(r#"1234"#); + for (a, f) in formulas { + let r: String = a.chars().filter(char::is_ascii_digit).collect(); + rows.push_str(&format!(r#"{}"#, esc(f))); + } + let names = if names.is_empty() { String::new() } else { format!("{names}") }; + let parts: [(&str, String); 5] = [ + ("[Content_Types].xml", CT.into()), + ("_rels/.rels", r#""#.into()), + ("xl/workbook.xml", format!(r#"{names}"#)), + ("xl/_rels/workbook.xml.rels", r#""#.into()), + ("xl/worksheets/sheet1.xml", format!(r#"{rows}"#)), + ]; + let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); + for (name, data) in parts { + z.start_file(name, zip::write::SimpleFileOptions::default()).unwrap(); + z.write_all(data.as_bytes()).unwrap(); + } + z.finish().unwrap().into_inner() +} + +fn session() -> Session { + let mut s = Session::new(); + s.new_workbook(); + s +} + +fn open(bytes: &[u8]) -> Session { + let mut s = Session::new(); + s.execute("file.open", json!({"name": "legacy.xlsx", "base64": gridcraft_engine::io::base64_encode(bytes)})).unwrap(); + s +} diff --git a/crates/formula/src/parser.rs b/crates/formula/src/parser.rs index f906c4782..2b6f6d408 100644 --- a/crates/formula/src/parser.rs +++ b/crates/formula/src/parser.rs @@ -302,11 +302,15 @@ impl Parser { { return Ok(Expr::Unary(UnOp::Spill, Box::new(r))); } - // Files write `@x` as `_xlfn.SINGLE(x)`. + // Files write `@x` as `_xlfn.SINGLE(x)`; `SINGLE(A1:A3*2)` is `@(A1:A3*2)`. if name == "SINGLE" && args.len() == 1 && let Some(x) = args.pop() { + let x = match x { + Expr::Binary(..) | Expr::Unary(UnOp::Neg | UnOp::Plus | UnOp::Percent, _) => Expr::Paren(Box::new(x)), + x => x, + }; return Ok(Expr::Unary(UnOp::At, Box::new(x))); } // Files may spell a trim reference as a function call (`_xlfn._TRO_ALL(A1:E10)`). diff --git a/crates/formula/src/tests.rs b/crates/formula/src/tests.rs index 8b8215455..6e3829925 100644 --- a/crates/formula/src/tests.rs +++ b/crates/formula/src/tests.rs @@ -106,6 +106,15 @@ fn spill_operator() -> Result<(), Box> { Ok(()) } +/// Files spell `@x` as `_xlfn.SINGLE(x)`; an operation inside keeps its parentheses. +#[test] +fn single_is_the_implicit_intersection_operator() { + assert_eq!(rt("_xlfn.SINGLE(A1:A3)*2"), "@A1:A3*2"); + assert_eq!(rt("_xlfn.SINGLE(A1:A3*2)"), "@(A1:A3*2)"); + assert_eq!(rt("_xlfn.SINGLE(-A1:A3)"), "@(-A1:A3)"); + assert_eq!(rt("SQRT(_xlfn.SINGLE(Data[Qty]))"), "SQRT(@Data[Qty])"); +} + #[test] fn delete_sheet_shrinks_3d_span() -> Result<(), Box> { let order: Vec = ["Sheet1", "Sheet2", "Sheet3", "Results"].iter().map(|s| s.to_string()).collect(); diff --git a/crates/xlsx/src/array_formula.rs b/crates/xlsx/src/array_formula.rs index 33d3f304c..8d871dd75 100644 --- a/crates/xlsx/src/array_formula.rs +++ b/crates/xlsx/src/array_formula.rs @@ -1,70 +1,409 @@ -//! Export-time implicit-intersection analysis, without evaluating the workbook. +//! Legacy (pre-dynamic-array) cell formulas and implicit intersection, without evaluating. //! -//! A range is harmless in SUM, but not in SQRT or arithmetic. Keep the result's shape separate -//! from affected operations inside it: SUM(range * range) returns a scalar yet needs array -//! evaluation. Legacy array-aware contexts such as SUMPRODUCT already preserve those operations. +//! Excel evaluates a cell formula saved without dynamic-array metadata (no `cm` attribute) the +//! legacy way: a reference that may cover several cells, used where one value is expected, is +//! intersected with the formula's row or column, and a multi-valued result shows its top-left +//! value. Dynamic-array Excel displays such a formula with the implicit intersection operator `@` +//! inserted where that happens (Microsoft support, "Implicit intersection operator: @"). The +//! reader does the same ([`apply_legacy`]), so the dynamic-array evaluator computes the legacy +//! result; the writer ([`export`]) saves a formula that is the legacy reading of some text as +//! that text, without the implied `@` (an `@` of a single value stays, as `_xlfn.SINGLE`), and +//! gives every other formula dynamic-array metadata. +//! +//! A range is harmless in SUM, but not in SQRT or arithmetic: `SUM(A1:A3*B1:B3)` intersects both +//! operands. Legacy array-aware parameters such as SUMPRODUCT's evaluate what is inside as +//! arrays, so nothing there is intersected, except IF's condition and IFERROR's and IFNA's +//! arguments ([`Param::Single`]). This is file-format policy, not a second evaluator: unknown +//! results are never calculated. + +use std::collections::HashMap; +use std::sync::Arc; use gridcraft_core::CellRef; -use gridcraft_formula::{BinOp, Expr, SheetSel, StructItem, UnOp}; +use gridcraft_formula::{BinOp, Expr, RefKind, SheetSel, StructItem, UnOp}; use gridcraft_model::{Formula, Workbook}; -#[derive(Clone, Copy, Default)] -struct Shape { - array: bool, - affected: bool, +use crate::FunctionInfo; + +/// What an expression produces, as far as implicit intersection is concerned. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Kind { + /// One value, or a reference to one cell. + Scalar, + /// A reference that may cover several cells. + Ref, + /// Several values that are not a reference. + Array, } -impl Shape { - fn array() -> Self { - Self { array: true, affected: false } +impl Kind { + fn join(self, other: Kind) -> Kind { + match (self, other) { + (Kind::Array, _) | (_, Kind::Array) => Kind::Array, + (Kind::Ref, _) | (_, Kind::Ref) => Kind::Ref, + _ => Kind::Scalar, + } } } -pub(crate) fn needs_array(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula) -> bool { - let Some(expr) = formula.expr_arc() else { return false }; - // The budget covers every node of a formula as long as Excel allows (8,192 characters make - // at most about 8,200 nodes), so only names that expand a lot fall back to array evaluation. - let mut scan = Scan { wb, at, remaining: 16_384, locals: Vec::new() }; - let shape = scan.expr(&expr, sheet, false, 0); - // An explicit `@` is written as `_xlfn.SINGLE`, which only dynamic-array formulas carry. - let mut at = false; - expr.walk(&mut |e| at |= matches!(e, Expr::Unary(UnOp::At, _))); - shape.array || shape.affected || at +/// How a position consumes its operand in a legacy formula. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Ctx { + /// One value: a multi-cell reference is intersected. + Value, + /// A reference or array as such (SUM's arguments); nested operations still take values. + Ref, + /// Array evaluation of everything inside (SUMPRODUCT's arguments): nothing is intersected. + Array, +} +/// A function parameter's legacy evaluation. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Param { + Value, + /// One value even inside array evaluation: IF's condition and IFERROR's and IFNA's + /// arguments. Legacy Excel evaluates them as arrays inside SUMPRODUCT only in a + /// Ctrl+Shift+Enter formula, which this analysis never sees. A name's definition is still + /// evaluated as an array throughout ([`Legacy::kind_of`]). + Single, + Ref, + Array, + /// The parameter is evaluated like the call itself (IF's and CHOOSE's results). + Pass, + /// A value read from the top-left cell of a reference, never intersected (T: Excel caches + /// `T(H12:J12)` as T of H12 in a formula outside columns H:J). The function lifts over the + /// cells, so the legacy result is the top-left one. + First, + /// An array parameter of a dynamic-array-era function: whether Excel evaluates a legacy + /// formula there as an array is not established, so such formulas are written as dynamic. + Uncertain, } -struct Scan<'a> { +/// How the writer saves a formula that neither spills nor is a legacy CSE array. +pub(crate) enum Export { + /// As it is, without dynamic-array metadata. + Unchanged, + /// As this expression (the formula without its implicit `@`), without dynamic-array metadata. + Legacy(Expr), + /// With dynamic-array metadata: legacy evaluation would differ. + Dynamic, +} + +pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo) -> Export { + let Some(expr) = formula.expr_arc() else { return Export::Unchanged }; + let mut explicit = false; + expr.walk(&mut |e| explicit |= matches!(e, Expr::Unary(UnOp::At, _))); + let mut legacy = Legacy::new(wb, at, functions); + if !explicit { + legacy.formula(expr.as_ref().clone(), sheet); + return if !legacy.certain { + Export::Dynamic + } else if !legacy.complete || legacy.inserted == 0 { + // A formula that does not spill and needs no `@`. One the analysis cannot finish + // reads back with at most a whole-formula `@`, which keeps its one value. + Export::Unchanged + } else { + Export::Dynamic + }; + } + // The legacy text of the formula: each `@` legacy evaluation implies at its position is + // dropped (the reader puts it back), and so is one of a range an edit made one cell; any other + // `@` of a single value stays, written as `_xlfn.SINGLE` in the legacy formula as Excel does. + legacy.explicit = true; + let out = legacy.formula(expr.as_ref().clone(), sheet); + if !legacy.complete || !legacy.certain { + // A formula this analysis cannot finish or whose semantics it does not know, but that is + // the reader's reading of some text: saved as that text. The reader adds a whole-formula + // `@` over such formulas, or `@` where it intersects. + let mut candidates = vec![remove_at(expr.as_ref().clone())]; + if let Expr::Unary(UnOp::At, x) = expr.as_ref() { + candidates.insert(0, unwrap_at(x.as_ref().clone())); + } + for text in candidates { + let read = self::legacy(wb, sheet, at, &Formula::from_expr(text.clone()), functions); + if read.map_or(text == *expr, |f| f.expr_arc().is_some_and(|e| *e == *expr)) { + return reads_back(text).map_or(Export::Dynamic, Export::Legacy); + } + } + return Export::Dynamic; + } + if !legacy.foreign && legacy.inserted == 0 { reads_back(out).map_or(Export::Dynamic, Export::Legacy) } else { Export::Dynamic } +} + +/// `e` without any `@` and the parentheses [`at`] adds with one. +fn remove_at(e: Expr) -> Expr { + e.map(&mut |x| match x { + Expr::Unary(UnOp::At, x) => unwrap_at(*x), + other => other, + }) +} + +/// Rewrites the legacy cell formulas at `cells` (model sheet index, positions) to their +/// dynamic-array reading, with `@` where Excel intersects. Runs once names and tables are loaded. +pub(crate) fn apply_legacy(wb: &mut Workbook, cells: Vec<(usize, Vec)>, functions: FunctionInfo) { + for (si, positions) in cells { + let Some(sheet) = wb.sheet(si) else { continue }; + let mut changed = Vec::new(); + for at in positions { + let Some(formula) = sheet.cell(at).and_then(|c| c.formula.as_deref()) else { continue }; + if let Some(f) = legacy(wb, si, at, formula, functions) { + changed.push((at, f)); + } + } + if changed.is_empty() { + continue; + } + let Some(sheet) = wb.sheet_mut(si) else { continue }; + for (at, f) in changed { + if let Some(cell) = sheet.cells.get_mut(at) { + cell.formula = Some(Arc::new(f)); + } + } + } +} + +/// The dynamic-array reading of a legacy formula, when it differs. +fn legacy(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo) -> Option { + if formula.array.is_some() { + return None; + } + let expr = formula.expr_arc()?; + let mut legacy = Legacy::new(wb, at, functions); + let out = legacy.formula(expr.as_ref().clone(), sheet); + if legacy.certain && legacy.complete { + if legacy.inserted == 0 { + return None; + } + // Text round trip: the stored expression is exactly what the text parses to. + if let Some(out) = reads_back(out) { + return Some(Formula::new(&gridcraft_formula::print(&out))); + } + // The reading's `@` take it past the parser's nesting limit: handled as unfinished. + } + // An analysis that stopped at its limits, or met semantics it does not know (the array + // arguments of dynamic-array functions, LAMBDA), leaves the formula as it was read, with only + // the whole-formula `@` when the result may be several values: legacy Excel shows one value, + // so the formula must not spill. When that text does not parse either, the formula stays as + // it was read. + let whole = matches!(expr.as_ref(), Expr::Unary(UnOp::At, _)); + let multi = legacy.top != Kind::Scalar || (legacy.certain && legacy.complete); + (multi && !whole).then(|| self::at(expr.as_ref().clone())).and_then(reads_back).map(|e| Formula::new(&gridcraft_formula::print(&e))) +} + +/// `e` when its text (the printer writes only the parentheses the expression has) reads back as +/// the same expression, so a dropped `@` never takes parentheses the text needs with it. An `@` +/// of a bare union has no such text, so its formula is saved as a dynamic one. +fn reads_back(e: Expr) -> Option { + gridcraft_formula::parse(&gridcraft_formula::print(&e)).is_ok_and(|p| p == e).then_some(e) +} + +/// The operand of an `@` without the parentheses [`at`] adds. +fn unwrap_at(e: Expr) -> Expr { + match e { + Expr::Paren(p) if !bare(&p) => *p, + other => other, + } +} + +/// `@e`, parenthesized unless `e` prints as one operand. An operand already in parentheses (the +/// file's own) gets a second pair, so [`unwrap_at`] can tell them from the ones added here. +fn at(e: Expr) -> Expr { + let e = if bare(&e) { e } else { Expr::Paren(Box::new(e)) }; + Expr::Unary(UnOp::At, Box::new(e)) +} + +/// Prints as one operand of `@`: Excel shows a spill reference's `@` without parentheses +/// (`=@F4#`, `=@F5#%`). +fn bare(e: &Expr) -> bool { + atomic(e) || matches!(e, Expr::Unary(UnOp::Spill, r) if matches!(**r, Expr::Ref(_))) +} + +fn atomic(e: &Expr) -> bool { + matches!(e, Expr::Ref(_) | Expr::Name(_) | Expr::Struct(_) | Expr::Call(..) | Expr::Array(_)) +} + +/// Nesting the analysis follows: the formula parser's own limit. Functions, parentheses and +/// prefix operators count one level each here as there, so any formula that parses is analysed +/// in full unless names, or levels the parser does not count (postfix `%` and `#`), take it +/// deeper; one that is not keeps its legacy reading ([`legacy`]). Operator chains do not count +/// (see [`Legacy::binary`]). +const MAX_DEPTH: usize = 256; + +struct Legacy<'a> { wb: &'a Workbook, at: CellRef, + functions: FunctionInfo, remaining: usize, - locals: Vec<(String, Shape)>, + locals: Vec<(String, Kind)>, + /// Kinds of the names analysed so far, by upper-case name, scope and whether one-cell ranges + /// counted as collapsed ones, with whether the definition reached a one-cell range written as + /// a range; `None` while a name's definition is being analysed (a cycle). + names: HashMap<(String, usize, bool), Option<(Kind, bool)>>, + /// How many `@` were inserted. + inserted: usize, + /// The analysis covered only established legacy semantics. + certain: bool, + /// The analysis finished within its nesting and size limits. + complete: bool, + /// Analysing a formula that has its own `@` (see [`Legacy::explicit_at`]). + explicit: bool, + /// A one-cell reference written as a range (`A1:A1`, what an edit leaves of `A1:A3`) counts + /// as one that may cover several cells, so an `@` the range had before the edit is implied. + collapsed: bool, + /// The analysis reached a one-cell reference written as a range, which `collapsed` changes. + one_cell_range: bool, + /// An explicit `@` is neither implied by legacy evaluation nor of a single value. + foreign: bool, + /// Analysing an expression whose rewrite is discarded (see [`Legacy::kind_of`]). + discarded: bool, + /// Analysing a union that is a function's single-value argument (see [`Legacy::expr`]). + union_arg: bool, + /// The kind of the whole formula, before any whole-formula `@`. + top: Kind, } -impl Scan<'_> { - fn expr(&mut self, e: &Expr, sheet: usize, array_context: bool, depth: usize) -> Shape { - // Names can be cyclic, and a short chain of repeated names can expand exponentially. - // Preserve array evaluation when analysis cannot safely finish. - if depth >= 64 || self.remaining == 0 { - return Shape { array: true, affected: true }; +impl<'a> Legacy<'a> { + fn new(wb: &'a Workbook, at: CellRef, functions: FunctionInfo) -> Self { + Legacy { + wb, + at, + functions, + remaining: 16384, + locals: Vec::new(), + names: HashMap::new(), + inserted: 0, + certain: true, + complete: true, + explicit: false, + collapsed: false, + one_cell_range: false, + foreign: false, + discarded: false, + union_arg: false, + top: Kind::Scalar, + } + } + + /// The kind of an expression analysed only for its kind (a name's definition, INDIRECT's + /// text), evaluated as legacy Excel evaluates a name: as an array throughout, IF's condition + /// included. Its rewrite is discarded, so it inserts no `@` into the formula. + fn kind_of(&mut self, e: Expr, sheet: usize, depth: usize) -> Kind { + let outer = (std::mem::replace(&mut self.discarded, true), std::mem::replace(&mut self.explicit, false), self.inserted); + let (_, kind) = self.expr(e, sheet, Ctx::Array, depth); + (self.discarded, self.explicit, self.inserted) = outer; + kind + } + + /// The whole cell formula: a multi-valued result shows its top-left value. + fn formula(&mut self, e: Expr, sheet: usize) -> Expr { + let e = match e { + Expr::Unary(UnOp::At, x) if self.explicit => return self.explicit_at(*x, sheet, Ctx::Value, 0, true).0, + e => e, + }; + let (e, kind) = self.expr(e, sheet, Ctx::Value, 0); + self.top = kind; + if kind == Kind::Array && self.certain { + self.inserted += 1; + at(e) + } else { + e + } + } + + /// An `@` already in the formula, at a position the analysis reads in `ctx`: analysed as its + /// operand alone. The `@` is the one legacy evaluation implies when the analysis puts it back + /// (that insertion is then not counted, and the `@` is dropped from the legacy text). An `@` + /// of a single value stays, as Excel keeps it (`_xlfn.SINGLE` in a legacy formula); any other + /// `@` makes the formula a dynamic-array one (`foreign`). + fn explicit_at(&mut self, x: Expr, sheet: usize, ctx: Ctx, depth: usize, top: bool) -> (Expr, Kind) { + let added = matches!(&x, Expr::Paren(p) if !bare(p)); + let operand = unwrap_at(x); + let nested = matches!(operand, Expr::Unary(UnOp::At, _)); + let before = self.inserted; + // A one-cell range the `@` applies to, directly or through what a name returns, may be + // what an edit left of a larger one: when the `@` is otherwise of a single value, it is + // implied if it is with such ranges counted as several cells. The retry's analysis is + // kept only when it implies the `@`, and runs only when the first pass reached such a + // range (otherwise it cannot differ). + let retry = (!nested && !self.collapsed && matches!(operand, Expr::Ref(_) | Expr::Name(_))).then(|| operand.clone()); + let seen = std::mem::replace(&mut self.one_cell_range, false); + let (mut y, mut kind) = self.expr(operand, sheet, ctx, depth + 1); + let mut implied = !nested && self.inserted > before; + let shrinkable = self.one_cell_range; + self.one_cell_range |= seen; + if let Some(operand) = retry.filter(|_| !implied && kind == Kind::Scalar && shrinkable) { + let state = (self.remaining, self.certain, self.complete, self.foreign); + self.collapsed = true; + let (y2, kind2) = self.expr(operand, sheet, ctx, depth + 1); + self.collapsed = false; + if self.inserted > before { + (y, kind, implied) = (y2, kind2, true); + } else { + (self.remaining, self.certain, self.complete, self.foreign) = state; + } + } + match y { + Expr::Unary(UnOp::At, inner) if implied => { + self.inserted -= 1; + (unwrap_at(*inner), Kind::Scalar) + } + y if top && kind == Kind::Array => { + // The whole-formula `@` the reader puts back over a multi-valued result. + self.top = kind; + (y, Kind::Scalar) + } + y => { + if kind != Kind::Scalar { + self.foreign = true; + } + let y = if added { Expr::Paren(Box::new(y)) } else { y }; + (Expr::Unary(UnOp::At, Box::new(y)), Kind::Scalar) + } + } + } + + fn expr(&mut self, e: Expr, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { + // Names can be cyclic or nest deeply; their kinds are remembered, so repeated names + // cost one analysis each. + if depth >= MAX_DEPTH || self.remaining == 0 { + self.complete = false; + return (e, Kind::Array); } self.remaining -= 1; + // The flag reaches the union through parentheses and `@`, and not its operands. + let union_arg = std::mem::take(&mut self.union_arg); + self.union_arg = union_arg && matches!(e, Expr::Paren(_) | Expr::Unary(UnOp::At, _)); + let (e, kind) = self.node(e, sheet, ctx, depth); + self.union_arg = false; + if ctx == Ctx::Value && kind == Kind::Ref { + if matches!(e, Expr::Binary(BinOp::Union, ..)) && !union_arg { + // A union, which only stands in parentheses, as an operand: Excel has no legacy + // text for its `@` (it shows `(F25,F26)%`), and the operator gives #VALUE! either + // way. A function's argument is intersected, as the function would spill it. + return (e, Kind::Scalar); + } + self.inserted += 1; + (at(e), Kind::Scalar) + } else { + (e, kind) + } + } + + fn node(&mut self, e: Expr, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { + let inner = |wanted: Ctx| if ctx == Ctx::Array { Ctx::Array } else { wanted }; match e { - Expr::Ref(r) => Shape { array: !r.range().is_single() || matches!(r.sheet, SheetSel::Span(_, _)), affected: false }, - Expr::Array(rows) => Shape { array: rows.len() > 1 || rows.first().is_some_and(|r| r.len() > 1), affected: false }, + Expr::Ref(r) => { + let one_cell_range = r.range().is_single() && !matches!(r.sheet, SheetSel::Span(_, _)) && matches!(r.kind, RefKind::Range(..)); + self.one_cell_range |= one_cell_range; + let single = r.range().is_single() && !matches!(r.sheet, SheetSel::Span(_, _)) && !(self.collapsed && one_cell_range); + (Expr::Ref(r), if single { Kind::Scalar } else { Kind::Ref }) + } + Expr::Array(rows) => { + let kind = if rows.len() > 1 || rows.first().is_some_and(|r| r.len() > 1) { Kind::Array } else { Kind::Scalar }; + (Expr::Array(rows), kind) + } Expr::Name(n) => { - if let Some((_, shape)) = self.locals.iter().rev().find(|(key, _)| key.eq_ignore_ascii_case(n)) { - return *shape; - } - let (scope, bare) = match n.rsplit_once('!') { - Some((s, name)) => (self.wb.sheet_index(&gridcraft_core::unquote_sheet(s)).unwrap_or(sheet), name), - None => (sheet, n.as_str()), - }; - if let Some(def) = self.wb.name(bare, scope) { - return match gridcraft_formula::parse_name(def.formula.trim_start_matches('=')) { - Ok(expr) => self.expr(&expr, def.scope.unwrap_or(scope), array_context, depth + 1), - Err(_) => Shape::array(), - }; - } - Shape { array: self.wb.table(bare).is_some(), affected: false } + let kind = self.name(&n, sheet, depth); + (Expr::Name(n), kind) } Expr::Struct(r) => { let table = if r.table.is_empty() { @@ -78,150 +417,378 @@ impl Scan<'_> { (Some(_), None) => true, _ => table.is_some_and(|t| t.columns.len() == 1), }; - Shape { array: !(one_row && one_col), affected: false } + (Expr::Struct(r), if one_row && one_col { Kind::Scalar } else { Kind::Ref }) + } + Expr::Paren(x) => { + let (x, kind) = self.expr(*x, sheet, ctx, depth + 1); + (Expr::Paren(Box::new(x)), kind) + } + Expr::Unary(UnOp::At, x) if self.explicit => self.explicit_at(*x, sheet, ctx, depth, false), + Expr::Unary(UnOp::At, x) => { + let (x, _) = self.expr(*x, sheet, inner(Ctx::Ref), depth + 1); + (Expr::Unary(UnOp::At, Box::new(x)), Kind::Scalar) + } + // A trim reference is a reference operator, like `:`. + Expr::Unary(op @ (UnOp::Spill | UnOp::Trim(_)), x) => { + let (x, _) = self.expr(*x, sheet, inner(Ctx::Ref), depth + 1); + (Expr::Unary(op, Box::new(x)), Kind::Ref) } - Expr::Paren(x) => self.expr(x, sheet, array_context, depth + 1), Expr::Unary(op, x) => { - let child = self.expr(x, sheet, array_context, depth + 1); - match op { - UnOp::At => Shape { array: false, affected: child.affected }, - // A trim reference is a reference operator, like `:`. - UnOp::Spill | UnOp::Trim(_) => Shape { array: true, affected: child.affected }, - _ => Shape { array: child.array, affected: child.affected || (child.array && !array_context) }, - } + let (x, kind) = self.expr(*x, sheet, inner(Ctx::Value), depth + 1); + (Expr::Unary(op, Box::new(x)), if kind == Kind::Scalar { Kind::Scalar } else { Kind::Array }) } - Expr::Binary(op, a, b) => { - // A flat chain such as A1+A2+… nests one level per operator on its left: follow - // it in a loop, so a long chain is neither deep recursion nor past the depth - // limit (which would write a plain sum as an array formula). - let mut spine = vec![(*op, b.as_ref())]; - let mut left = a.as_ref(); - while let Expr::Binary(op, a, b) = left { - if self.remaining == 0 { - return Shape { array: true, affected: true }; - } - self.remaining -= 1; - spine.push((*op, b.as_ref())); - left = a.as_ref(); - } - let mut acc = self.expr(left, sheet, array_context, depth + 1); - while let Some((op, b)) = spine.pop() { - let b = self.expr(b, sheet, array_context, depth + 1); - let reference = matches!(op, BinOp::Range | BinOp::Union | BinOp::Intersect); - let array = reference || acc.array || b.array; - acc = Shape { array, affected: acc.affected || b.affected || (!reference && array && !array_context) }; - } - acc + Expr::Binary(..) => self.binary(e, sheet, ctx, depth), + Expr::Call(name, args) => self.call(name, args, sheet, ctx, depth + 1), + Expr::Invoke(f, args) => { + self.certain = false; + (Expr::Invoke(f, args), Kind::Array) } - Expr::Call(name, args) => self.call(name, args, sheet, array_context, depth + 1), - Expr::Invoke(_, _) => Shape { array: true, affected: false }, - _ => Shape::default(), + other => (other, Kind::Scalar), } } - fn call(&mut self, name: &str, args: &[Expr], sheet: usize, array_context: bool, depth: usize) -> Shape { - if name == "LET" - && args.len() >= 3 - && let Some((last, bindings)) = args.split_last() + fn name(&mut self, n: &str, sheet: usize, depth: usize) -> Kind { + if let Some((_, kind)) = self.locals.iter().rev().find(|(key, _)| key.eq_ignore_ascii_case(n)) { + return *kind; + } + let (scope, bare) = match n.rsplit_once('!') { + Some((s, name)) => (self.wb.sheet_index(&gridcraft_core::unquote_sheet(s)).unwrap_or(sheet), name), + None => (sheet, n), + }; + if let Some(def) = self.wb.name(bare, scope) { + let key = (bare.to_ascii_uppercase(), scope, self.collapsed); + match self.names.get(&key) { + Some(Some((kind, one_cell_range))) => { + self.one_cell_range |= *one_cell_range; + return *kind; + } + Some(None) => { + // A name defined in terms of itself has no legacy value to keep. + self.certain = false; + return Kind::Array; + } + None => {} + } + let Ok(expr) = gridcraft_formula::parse_name(def.formula.trim_start_matches('=')) else { return Kind::Array }; + self.names.insert(key.clone(), None); + // A name's own formula does not see the formula's LET variables. + let locals = std::mem::take(&mut self.locals); + let seen = std::mem::replace(&mut self.one_cell_range, false); + let kind = self.kind_of(expr, def.scope.unwrap_or(scope), depth + 1); + let one_cell_range = std::mem::replace(&mut self.one_cell_range, seen); + self.one_cell_range |= one_cell_range; + self.locals = locals; + self.names.insert(key, Some((kind, one_cell_range))); + return kind; + } + if self.wb.table(bare).is_some() { Kind::Ref } else { Kind::Scalar } + } + + fn call(&mut self, name: String, args: Vec, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { + if name == "LET" && args.len() >= 3 { + return self.let_(args, sheet, ctx, depth); + } + let lambda_name = !(self.functions.is_builtin)(&name) && !host_function(&name) && self.wb.name(&name, sheet).is_some(); + if args.len() > 255 + || lambda_name + || matches!(name.as_str(), "LAMBDA" | "MAP" | "REDUCE" | "SCAN" | "BYROW" | "BYCOL" | "MAKEARRAY" | "ISOMITTED") { - let saved = self.locals.len(); - let mut affected = false; - for pair in bindings.as_chunks::<2>().0 { - if let [Expr::Name(n), value] = pair { - let shape = self.expr(value, sheet, array_context, depth); - affected |= shape.affected; - self.locals.push((n.clone(), shape)); - } - } - let mut result = self.expr(last, sheet, array_context, depth); - result.affected |= affected; - self.locals.truncate(saved); - return result; + self.certain = false; + return (Expr::Call(name, args), Kind::Array); } - let legacy_array = - matches!(name, "SUMPRODUCT" | "MMULT" | "MINVERSE" | "MDETERM" | "TRANSPOSE" | "FREQUENCY" | "LINEST" | "LOGEST" | "TREND" | "GROWTH"); - let mut affected = false; let mut lifted = false; - let mut shapes = Vec::with_capacity(args.len().min(256)); - if args.len() > 255 { - return Shape { array: true, affected: true }; - } - for (i, arg) in args.iter().enumerate() { - let context = array_context || legacy_array || (name == "INDEX" && i == 0); - let shape = self.expr(arg, sheet, context, depth); - affected |= shape.affected; - if shape.array && scalar_arg(name, i) { - lifted = true; - affected |= !context; - } - shapes.push(shape); - } - let first_array = shapes.first().is_some_and(|s| s.array); - let array = match name { - "INDIRECT" => match args.first() { - Some(Expr::Text(text)) if !matches!(args.get(1), Some(Expr::Bool(false))) => { - gridcraft_formula::parse(text).map(|e| self.expr(&e, sheet, array_context, depth).array).unwrap_or(true) + let mut kinds = Vec::with_capacity(args.len()); + let mut out = Vec::with_capacity(args.len()); + for (i, arg) in args.into_iter().enumerate() { + let param = param(&name, i, self.functions); + let arg_ctx = match param { + Param::Value | Param::Single => Ctx::Value, + Param::Ref | Param::First => Ctx::Ref, + Param::Pass => ctx, + Param::Array => Ctx::Array, + Param::Uncertain => { + self.certain = false; + Ctx::Array } - _ => true, - }, - "OFFSET" => { - // Omitted dimensions retain the source reference's shape; unknown dimensions - // might become multiple cells on a later calculation. - if args.len() <= 3 { first_array } else { !args.get(3).is_some_and(one) || args.get(4).map_or(first_array, |e| !one(e)) } - } - "INDEX" => { - let row = args.get(1); - let col = args.get(2); - let row_positive = row.is_some_and(|e| positive_index(e, shapes.get(1).copied())); - let col_positive = col.is_some_and(|e| positive_index(e, shapes.get(2).copied())); - !row_positive || col.is_some_and(|_| !col_positive) || (col.is_none() && !args.first().is_some_and(single_column)) + }; + let arg_ctx = if ctx == Ctx::Array && (param != Param::Single || self.discarded) { Ctx::Array } else { arg_ctx }; + // A union argument, under the file's parentheses and any `@` the reader put on it. + let mut operand = &arg; + while let Expr::Paren(x) | Expr::Unary(UnOp::At, x) = operand { + operand = x; } + self.union_arg = arg_ctx == Ctx::Value && matches!(operand, Expr::Binary(BinOp::Union, ..)); + let (arg, kind) = self.expr(arg, sheet, arg_ctx, depth); + // A value parameter given several values evaluates the function once per value. + lifted |= match param { + Param::Value | Param::Single => kind == Kind::Array || (kind == Kind::Ref && arg_ctx == Ctx::Array), + Param::First => kind != Kind::Scalar, + _ => false, + }; + kinds.push(kind); + out.push(arg); + } + let first = kinds.first().copied().unwrap_or(Kind::Scalar); + let kind = match name.as_str() { + "IF" | "CHOOSE" if !lifted => kinds.iter().skip(1).fold(Kind::Scalar, |k, x| k.join(*x)), "ROW" | "COLUMN" => { - affected |= first_array && !array_context; - first_array + if first == Kind::Scalar { + Kind::Scalar + } else if ctx == Ctx::Array { + Kind::Array + } else { + // Outside array evaluation, ROW and COLUMN of a range give its first row or column. + self.inserted += 1; + return (at(Expr::Call(name, out)), Kind::Scalar); + } } - "IF" | "IFERROR" | "IFNA" | "IFS" | "SWITCH" | "CHOOSE" => shapes.iter().any(|s| s.array), - "XLOOKUP" => { - lifted - || match (args.get(1), args.get(2)) { - (Some(Expr::Ref(lookup)), Some(Expr::Ref(result))) => { - if lookup.range().width() == 1 { - result.range().width() > 1 - } else { - result.range().height() > 1 - } - } - _ => true, + "INDIRECT" => match out.first() { + Some(Expr::Text(text)) if !matches!(out.get(1), Some(Expr::Bool(false))) => match gridcraft_formula::parse(text) { + Ok(e) => { + let kind = self.kind_of(e, sheet, depth); + if kind == Kind::Scalar { Kind::Scalar } else { Kind::Ref } } - } + Err(_) => Kind::Ref, + }, + _ => Kind::Ref, + }, + "OFFSET" => offset_kind(&out, first, lifted), + "INDEX" => index_kind(&out, &kinds, first, lifted), + "XLOOKUP" if !lifted => match (out.get(1), out.get(2)) { + (Some(Expr::Ref(lookup)), Some(Expr::Ref(result))) => { + let multi = if lookup.range().width() == 1 { result.range().width() > 1 } else { result.range().height() > 1 }; + if multi { Kind::Ref } else { Kind::Scalar } + } + _ => Kind::Ref, + }, + "ANCHORARRAY" | "TRIMRANGE" => Kind::Ref, "FILTER" | "SORT" | "SORTBY" | "UNIQUE" | "SEQUENCE" | "RANDARRAY" | "VSTACK" | "HSTACK" | "TOROW" | "TOCOL" | "WRAPROWS" | "WRAPCOLS" | "TAKE" | "DROP" | "CHOOSEROWS" | "CHOOSECOLS" | "EXPAND" | "GROUPBY" | "PIVOTBY" | "TRANSPOSE" | "MMULT" | "MINVERSE" - | "FREQUENCY" | "LINEST" | "LOGEST" | "TREND" | "GROWTH" | "MAP" | "SCAN" | "BYROW" | "BYCOL" | "MAKEARRAY" | "ANCHORARRAY" => true, - _ => lifted || self.wb.name(name, sheet).is_some(), + | "MUNIT" | "FREQUENCY" | "LINEST" | "LOGEST" | "TREND" | "GROWTH" | "MODE.MULT" | "TEXTSPLIT" | "FILTERXML" => Kind::Array, + // Every match (mode 1) or the capture groups (mode 2); the first match by default. + "REGEXEXTRACT" => match out.get(2) { + None | Some(Expr::Missing) => Kind::Scalar, + Some(Expr::Number(n)) if *n == 0.0 => Kind::Scalar, + _ => Kind::Array, + }, + // Google Sheets' export placeholder fails with #NAME? so an outer IFERROR keeps the cached scalar. + "__XLUDF.DUMMYFUNCTION" => Kind::Scalar, + // A user-defined function may return a range or an array. + n if user_function(n, self.functions) => Kind::Array, + _ if lifted => Kind::Array, + _ => Kind::Scalar, }; - Shape { array, affected } + (Expr::Call(name, out), kind) + } +} + +fn offset_kind(out: &[Expr], first: Kind, lifted: bool) -> Kind { + // Omitted dimensions keep the source reference's shape; unknown dimensions might + // become several cells on a later calculation. + let first_multi = first != Kind::Scalar; + let multi = if out.len() <= 3 { first_multi } else { !out.get(3).is_some_and(one) || out.get(4).map_or(first_multi, |e| !one(e)) }; + if lifted { + Kind::Array + } else if multi { + Kind::Ref + } else { + Kind::Scalar + } +} + +fn index_kind(out: &[Expr], kinds: &[Kind], first: Kind, lifted: bool) -> Kind { + let row = out.get(1); + let col = out.get(2); + let row_positive = row.is_some_and(|e| positive_index(e, kinds.get(1).copied())); + let col_positive = col.is_some_and(|e| positive_index(e, kinds.get(2).copied())); + let multi = !row_positive || col.is_some_and(|_| !col_positive) || (col.is_none() && !out.first().is_some_and(one_line)); + if multi { + if first == Kind::Array { Kind::Array } else { Kind::Ref } + } else if lifted { + Kind::Array + } else { + Kind::Scalar + } +} + +impl Legacy<'_> { + /// A binary operation. `A1+A2+…+A70` parses as a left-nested chain as deep as it is long, so + /// the chain's left operands are walked iteratively, at the same depth; each operation still + /// costs one node of the budget. + fn binary(&mut self, e: Expr, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { + let operand = |op: BinOp, node: Ctx| { + if node == Ctx::Array { + Ctx::Array + } else if matches!(op, BinOp::Range | BinOp::Union | BinOp::Intersect) { + Ctx::Ref + } else { + Ctx::Value + } + }; + // (operator, right operand, the operation's context), outermost first. + let mut spine = Vec::new(); + let mut cur = e; + let mut cur_ctx = ctx; + while let Expr::Binary(op, a, b) = cur { + if !spine.is_empty() { + // The caller's `expr` counted the outermost operation. + if self.remaining == 0 { + self.complete = false; + } else { + self.remaining -= 1; + } + } + spine.push((op, *b, cur_ctx)); + cur_ctx = operand(op, cur_ctx); + cur = *a; + } + let (mut acc, mut kind) = self.expr(cur, sheet, cur_ctx, depth); + for (i, (op, b, node_ctx)) in spine.into_iter().enumerate().rev() { + let (b, kb) = self.expr(b, sheet, operand(op, node_ctx), depth); + let node = Expr::Binary(op, Box::new(acc), Box::new(b)); + let k = if matches!(op, BinOp::Range | BinOp::Union | BinOp::Intersect) { + Kind::Ref + } else if kind == Kind::Scalar && kb == Kind::Scalar { + Kind::Scalar + } else { + Kind::Array + }; + // An inner operation is an operand: `expr` would intersect it there. + (acc, kind) = if i > 0 && node_ctx == Ctx::Value && k == Kind::Ref { + self.inserted += 1; + (at(node), Kind::Scalar) + } else { + (node, k) + }; + } + (acc, kind) + } + + fn let_(&mut self, args: Vec, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { + let saved = self.locals.len(); + let count = args.len(); + let mut out = Vec::with_capacity(count); + let mut pending: Option = None; + let mut kind = Kind::Scalar; + for (i, arg) in args.into_iter().enumerate() { + if i + 1 == count { + let (body, k) = self.expr(arg, sheet, ctx, depth); + out.push(body); + kind = k; + } else if i % 2 == 0 { + pending = match &arg { + Expr::Name(n) => Some(n.clone()), + _ => None, + }; + out.push(arg); + } else { + let value_ctx = if ctx == Ctx::Array { Ctx::Array } else { Ctx::Ref }; + let (value, k) = self.expr(arg, sheet, value_ctx, depth); + if let Some(n) = pending.take() { + self.locals.push((n, k)); + } + out.push(value); + } + } + self.locals.truncate(saved); + (Expr::Call("LET".into(), out), kind) } } fn positive(e: &Expr) -> bool { matches!(e, Expr::Number(n) if *n >= 1.0) } - -fn positive_index(e: &Expr, shape: Option) -> bool { - // MATCH returns a one-based position or an error, never INDEX's zero "whole row" selector. - positive(e) || (matches!(e, Expr::Call(name, _) if matches!(name.as_str(), "MATCH" | "XMATCH")) && shape.is_some_and(|s| !s.array)) +fn positive_index(e: &Expr, kind: Option) -> bool { + // MATCH returns a one-based position or an error, never INDEX's zero whole-row selector. + positive(e) || (matches!(e, Expr::Call(name, _) if matches!(name.as_str(), "MATCH" | "XMATCH")) && kind.is_some_and(|k| k != Kind::Ref)) } fn one(e: &Expr) -> bool { matches!(e, Expr::Number(n) if *n == 1.0) } -fn single_column(e: &Expr) -> bool { - matches!(e, Expr::Ref(r) if r.range().width() == 1) +/// A one-column or one-row reference: INDEX with a single index picks one cell of it. +fn one_line(e: &Expr) -> bool { + matches!(e, Expr::Ref(r) if r.range().width() == 1 || r.range().height() == 1) } -/// Legacy reference/array-taking parameters. Other parameters consume a single value. -/// This is file-format policy, not a second evaluator: unknown results are never calculated. +/// Functions the calculation engine evaluates itself, whose parameters are described here +/// rather than by the function library's lifting metadata. +fn host_function(name: &str) -> bool { + matches!( + name, + "IF" | "IFS" + | "IFERROR" + | "IFNA" + | "CHOOSE" + | "SWITCH" + | "ROW" + | "COLUMN" + | "ROWS" + | "COLUMNS" + | "AREAS" + | "ISREF" + | "ISFORMULA" + | "FORMULATEXT" + | "SHEET" + | "SHEETS" + | "OFFSET" + | "INDIRECT" + | "INDEX" + | "CELL" + | "TRIMRANGE" + | "SUBTOTAL" + | "AGGREGATE" + | "TEXT" + | "LET" + | "LAMBDA" + | "ISOMITTED" + | "MAP" + | "REDUCE" + | "SCAN" + | "BYROW" + | "BYCOL" + | "MAKEARRAY" + | "ANCHORARRAY" + | "GETPIVOTDATA" + ) +} + +/// Not a built-in worksheet function: an add-in or VBA function. +fn user_function(name: &str, functions: FunctionInfo) -> bool { + !host_function(name) && !(functions.is_builtin)(name) +} + +fn param(name: &str, i: usize, functions: FunctionInfo) -> Param { + match name { + "IF" if i == 0 => Param::Single, + "IFERROR" | "IFNA" => Param::Single, + "CHOOSE" if i == 0 => Param::Value, + "IF" | "CHOOSE" => Param::Pass, + "SUMPRODUCT" | "MMULT" | "MINVERSE" | "MDETERM" | "TRANSPOSE" | "FREQUENCY" | "LINEST" | "LOGEST" | "TREND" | "GROWTH" => Param::Array, + "INDEX" if i == 0 => Param::Array, + // LOOKUP's vectors and AGGREGATE's array form evaluate their arguments as arrays. + "LOOKUP" if i > 0 => Param::Array, + "AGGREGATE" if i == 2 => Param::Array, + "ROW" | "COLUMN" | "ROWS" | "COLUMNS" | "AREAS" | "ISREF" | "ISFORMULA" | "FORMULATEXT" | "SHEET" | "SHEETS" | "TRIMRANGE" + | "ANCHORARRAY" => Param::Ref, + "CELL" | "GETPIVOTDATA" if i == 1 => Param::Ref, + "T" => Param::First, + "N" if i == 0 => Param::Value, + "FILTER" | "SORT" | "SORTBY" | "UNIQUE" | "VSTACK" | "HSTACK" | "TOROW" | "TOCOL" | "WRAPROWS" | "WRAPCOLS" | "TAKE" | "DROP" + | "CHOOSEROWS" | "CHOOSECOLS" | "EXPAND" | "GROUPBY" | "PIVOTBY" + if !scalar_arg(name, i) => + { + Param::Uncertain + } + n if user_function(n, functions) => Param::Ref, + n if scalar_arg(n, i) && (host_function(n) || (functions.is_scalar_arg)(n, i)) => Param::Value, + _ => Param::Ref, + } +} + +/// Legacy reference/array-taking parameters. Other parameters consume a single value; the +/// function library's lifting metadata refines this for the functions it implements. fn scalar_arg(name: &str, i: usize) -> bool { match name { "SUM" | "AVERAGE" | "AVERAGEA" | "MIN" | "MINA" | "MAX" | "MAXA" | "COUNT" | "COUNTA" | "COUNTBLANK" | "PRODUCT" | "SUMSQ" | "MEDIAN" diff --git a/crates/xlsx/src/fmla.rs b/crates/xlsx/src/fmla.rs index 29dd20870..a03206702 100644 --- a/crates/xlsx/src/fmla.rs +++ b/crates/xlsx/src/fmla.rs @@ -111,8 +111,16 @@ fn expr_to_file(e: Expr) -> String { // reference that became `#REF!` (its cell was deleted) as plain `#REF!`. Expr::Unary(gridcraft_formula::UnOp::Spill, r) if matches!(*r, Expr::Error(_)) => *r, Expr::Unary(gridcraft_formula::UnOp::Spill, r) => Expr::Call("_xlfn.ANCHORARRAY".into(), vec![*r]), - // Nor an `@` operator: Excel writes `@x` as `_xlfn.SINGLE(x)`. - Expr::Unary(gridcraft_formula::UnOp::At, x) => Expr::Call("_xlfn.SINGLE".into(), vec![*x]), + // Nor an `@` operator: Excel writes `@x` as `_xlfn.SINGLE(x)`. The reader puts back the + // parentheses of `@(A1:A3*2)`, `@(-A1:A3)` and the like, so only those are dropped; + // others (a union's, redundant ones) are kept so the text reads back the same. + Expr::Unary(gridcraft_formula::UnOp::At, x) => Expr::Call( + "_xlfn.SINGLE".into(), + vec![match *x { + Expr::Paren(inner) if rewrapped_by_single(&inner) => *inner, + other => other, + }], + ), // Nor the `[@Col]` shorthand: files spell it `[[#This Row],[Col]]`, or Excel repairs // the file and drops the table. Carried as a name, which prints verbatim. Expr::Struct(st) => Expr::Name(gridcraft_formula::printer::struct_ref_file(&st)), @@ -128,6 +136,13 @@ fn expr_to_file(e: Expr) -> String { gridcraft_formula::print(&e) } +/// The operands whose parentheses the reader adds back to `_xlfn.SINGLE(x)` (see the formula +/// parser): a binary operation other than a union, and a prefix or postfix operator. +fn rewrapped_by_single(e: &Expr) -> bool { + use gridcraft_formula::{BinOp, UnOp}; + matches!(e, Expr::Binary(op, ..) if *op != BinOp::Union) || matches!(e, Expr::Unary(UnOp::Neg | UnOp::Plus | UnOp::Percent, _)) +} + /// `_xlpm.x` → `x` (any case, as the parser upper-cases function names). fn strip_param_prefix(n: String) -> String { match (n.get(..PARAM_PREFIX.len()), n.get(PARAM_PREFIX.len()..)) { diff --git a/crates/xlsx/src/lib.rs b/crates/xlsx/src/lib.rs index 4d5038012..259a1a1a6 100644 --- a/crates/xlsx/src/lib.rs +++ b/crates/xlsx/src/lib.rs @@ -83,25 +83,66 @@ impl ReadReport { } } -/// Reads an `.xlsx` / `.xlsm` / `.xltx` / `.xltm` package (the workbook part is found through the -/// package relationships, so every main-part content type is accepted). -pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { - read::read_xlsx(bytes, None) +/// What reading and writing formulas needs to know about worksheet functions. The function +/// library sits beside this crate (format crates depend only on core, formula and model), so the +/// caller supplies its answers; the engine passes the library's. +/// +/// It decides how legacy formulas (saved without dynamic-array metadata) are read with Excel's +/// implicit `@`, and which formulas are saved with dynamic-array metadata. +#[derive(Clone, Copy, Debug)] +pub struct FunctionInfo { + /// Whether `name` (upper case, without `_xlfn.`) is a built-in worksheet function rather than + /// an add-in, VBA or LAMBDA function. + pub is_builtin: fn(&str) -> bool, + /// Whether parameter `i` of the built-in function `name` takes a single value, as opposed to + /// a reference or array taken as a whole. + pub is_scalar_arg: fn(&str, usize) -> bool, } -/// [`read_xlsx`], and opens formulas from Google Sheets' XLSX export as the formulas written in -/// Google Sheets (#2871). -/// -/// Google writes a formula that uses a function it thinks Excel lacks as -/// `IFERROR(__xludf.DUMMYFUNCTION(""),)`; Excel shows the -/// cached value. A cell formula holding that wrapper, whole or inside a larger formula, is read -/// as the formula inside when it parses and `known` says GridCraft calculates every function it -/// calls. Otherwise (a Google-only function such as `GOOGLETRANSLATE`, inner text that doesn't -/// parse, or nesting past the parser's limit) the wrapper stays, so the cell keeps showing the -/// cached value as in Excel. `known` is passed in because this crate doesn't depend on the -/// function library. -pub fn read_xlsx_with(bytes: &[u8], known: &dyn Fn(&str) -> bool) -> Result<(Workbook, ReadReport), IoError> { - read::read_xlsx(bytes, Some(known)) +impl Default for FunctionInfo { + /// Without the function library: every name counts as built in, and this crate's table of + /// reference- and array-taking parameters alone decides parameter types. + fn default() -> Self { + FunctionInfo { is_builtin: |_| true, is_scalar_arg: |_, _| true } + } +} + +/// Reads an `.xlsx` / `.xlsm` / `.xltx` / `.xltm` package (the workbook part is found through the +/// package relationships, so every main-part content type is accepted), with the default +/// [`FunctionInfo`]. +pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { + read::read_xlsx(bytes, ReadOptions::default()) +} + +/// Caller-supplied function facts for [`read_xlsx_with`]: both legacy-formula [`FunctionInfo`] +/// and Google Sheets export unwrap predicate (`#2871`). +#[derive(Clone, Copy, Default)] +pub struct ReadOptions<'a> { + pub functions: FunctionInfo, + pub known_function: Option<&'a dyn Fn(&str) -> bool>, +} + +impl<'a> From for ReadOptions<'a> { + fn from(functions: FunctionInfo) -> Self { + Self { functions, known_function: None } + } +} + +impl<'a> From<(FunctionInfo, &'a dyn Fn(&str) -> bool)> for ReadOptions<'a> { + fn from((functions, known): (FunctionInfo, &'a dyn Fn(&str) -> bool)) -> Self { + Self { functions, known_function: Some(known) } + } +} + +impl<'a, F: Fn(&str) -> bool> From<&'a F> for ReadOptions<'a> { + fn from(known: &'a F) -> Self { + Self { functions: FunctionInfo::default(), known_function: Some(known) } + } +} + +/// [`read_xlsx`] with caller-supplied function facts and/or Google Sheets export unwrap predicate (#2871). +pub fn read_xlsx_with<'a>(bytes: &[u8], opts: impl Into>) -> Result<(Workbook, ReadReport), IoError> { + read::read_xlsx(bytes, opts.into()) } /// The part of a zip package that declares more than [`MAX_PART`] unpacked bytes, with that size: @@ -153,14 +194,21 @@ impl XlsxKind { } } -/// Writes a workbook as an `.xlsx` package, or as an `.xlsm` one when it holds a VBA project. +/// Writes a workbook as an `.xlsx` package, or as an `.xlsm` one when it holds a VBA project, +/// with the default [`FunctionInfo`]. pub fn write_xlsx(wb: &Workbook) -> Result, IoError> { - write::write_xlsx(wb, if wb.vba_project.is_some() { XlsxKind::MacroWorkbook } else { XlsxKind::Workbook }) + let kind = if wb.vba_project.is_some() { XlsxKind::MacroWorkbook } else { XlsxKind::Workbook }; + write::write_xlsx(wb, kind, FunctionInfo::default()) } -/// Writes a workbook as the given member of the `.xlsx` family. +/// Writes a workbook as the given member of the `.xlsx` family, with the default [`FunctionInfo`]. pub fn write_xlsx_as(wb: &Workbook, kind: XlsxKind) -> Result, IoError> { - write::write_xlsx(wb, kind) + write::write_xlsx(wb, kind, FunctionInfo::default()) +} + +/// [`write_xlsx_as`] with the function library's facts. +pub fn write_xlsx_with(wb: &Workbook, kind: XlsxKind, functions: FunctionInfo) -> Result, IoError> { + write::write_xlsx(wb, kind, functions) } /// Writes a workbook as the given member of the `.xlsx` family for the file `book_name` (a bare @@ -169,6 +217,11 @@ pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: &str) -> Resul write::write_xlsx_named(wb, kind, Some(book_name)) } +/// Writes a workbook with caller-supplied function facts and file name. +pub fn write_xlsx_named_with(wb: &Workbook, kind: XlsxKind, book_name: &str, functions: FunctionInfo) -> Result, IoError> { + write::write_xlsx_named_with(wb, kind, Some(book_name), functions) +} + /// CSV import options. #[derive(Clone, Debug, PartialEq)] pub struct CsvOptions { diff --git a/crates/xlsx/src/read.rs b/crates/xlsx/src/read.rs index 5bdc98216..5ae2a782b 100644 --- a/crates/xlsx/src/read.rs +++ b/crates/xlsx/src/read.rs @@ -35,6 +35,9 @@ pub struct Ctx<'a> { pub cell_pictures: Vec>>, /// x14 conditional-format ids of the current sheet's data bars, with the index of the rule in `cond_formats`. pub cf_ext_ids: Vec<(String, usize)>, + /// Cells of the worksheet being read whose formulas lack dynamic-array metadata (legacy + /// formulas, evaluated with implicit intersection), collected per sheet. + pub legacy_formulas: Vec, /// Bytes charged so far for shared formulas expanded into dependent cells (see /// [`crate::sheet_read::MAX_SHARED_FORMULA_BYTES`]). pub shared_formula_bytes: u64, @@ -241,7 +244,9 @@ fn selected_workbook_view(workbook: &El) -> Option<&El> { views.kids("workbookView").find(|view| view.attr("visibility").unwrap_or("visible") == "visible").or_else(|| views.child("workbookView")) } -pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> bool>) -> Result<(Workbook, ReadReport), IoError> { +pub fn read_xlsx<'a>(bytes: &'a [u8], opts: crate::ReadOptions<'a>) -> Result<(Workbook, ReadReport), IoError> { + let functions = opts.functions; + let known_function = opts.known_function; let pkg = Package::open(bytes)?; let mut cx = Ctx { pkg, @@ -258,6 +263,7 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> num_fmts: Default::default(), cell_pictures: vec![], cf_ext_ids: vec![], + legacy_formulas: vec![], shared_formula_bytes: 0, known_function, python: Default::default(), @@ -309,6 +315,8 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> // Sheets. `file_to_model[i]` maps the i-th to a model sheet index. let mut file_to_model: Vec> = vec![]; + // Legacy formula cells per model sheet, rewritten once names and tables are known. + let mut legacy_formulas: Vec<(usize, Vec)> = vec![]; let mut pivot_caches = crate::pivot::CachesIn::new(&wb_xml, &wb_rels); let sheet_els: Vec = wb_xml.child("sheets").map(|s| s.kids("sheet").cloned().collect()).unwrap_or_default(); let mut names = SheetNames::default(); @@ -331,14 +339,17 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> continue; } let name = names.unique(&name, i); + cx.legacy_formulas.clear(); let (mut sheet, read_controls) = match crate::sheet_read::read_sheet(&mut cx, &rel.target, &name) { Ok(s) => s, Err(IoError::TooLarge(m)) => return Err(IoError::TooLarge(m)), Err(e) => { cx.warn(format!("sheet \"{name}\" could not be read: {e}")); + cx.legacy_formulas.clear(); (Sheet::new(name.clone()), Default::default()) } }; + legacy_formulas.push((wb.sheets.len(), std::mem::take(&mut cx.legacy_formulas))); match crate::pivot::read_pivots(&mut cx, &mut pivot_caches, &rel.target, &name) { Ok(p) => sheet.pivots = p, Err(IoError::TooLarge(m)) => return Err(IoError::TooLarge(m)), @@ -458,6 +469,8 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> } // Calls of LAMBDA names are spelled like the names (the parser upper-cases them). wb.apply_name_call_case(); + // Legacy formulas read as dynamic-array formulas with Excel's implicit `@`. + crate::array_formula::apply_legacy(&mut wb, legacy_formulas, functions); wb.styles = cx.styles; wb.phonetics = cx.phonetics; Ok((wb, cx.report)) diff --git a/crates/xlsx/src/sheet_read.rs b/crates/xlsx/src/sheet_read.rs index e68d14389..40d4b1c36 100644 --- a/crates/xlsx/src/sheet_read.rs +++ b/crates/xlsx/src/sheet_read.rs @@ -31,6 +31,7 @@ struct Shared { anchor: CellRef, expr: Option, text: String, + unwrapped: bool, /// Approximate bytes one dependent cell costs (its printed text plus its expression tree); /// what exceeds [`SHARED_FORMULA_FREE_COST`] is charged against [`MAX_SHARED_FORMULA_BYTES`]. cost: u64, @@ -626,6 +627,11 @@ fn read_cell(cx: &mut Ctx<'_>, sheet: &mut Sheet, st: &mut RowState, c: &El, pos if py.is_some() { record_python_spill(st, sheet, pos); } + // A formula without dynamic-array metadata is a legacy formula, unless entered as a + // Ctrl+Shift+Enter array or unwrapped from Google Sheets' export (#2871). + if formula.as_ref().is_some_and(|f| !f.dynamic && f.array.is_none() && f.expr_arc().is_some()) { + cx.legacy_formulas.push(pos); + } let spilled_from = if formula.is_none() && py.is_none() { dynamic_range(st, pos) } else { None }; let python_anchor = spilled_from.is_some_and(|r| sheet.cells.get(r.start).is_some_and(|a| a.py.is_some())); let value = match spilled_from { @@ -649,16 +655,21 @@ fn read_cell(cx: &mut Ctx<'_>, sheet: &mut Sheet, st: &mut RowState, c: &El, pos } fn read_formula(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option { - // Keep the file's dynamic-array mark, so saving never turns the formula into a legacy one. - read_formula_text(cx, st, f, pos, dynamic).map(|fm| Formula { dynamic, ..fm }) + // Keep the file's dynamic-array mark (and treat Google Sheets unwrapped formulas as dynamic), + // so saving never turns the formula into a legacy one. + let mut unwrapped = false; + let fm = read_formula_text(cx, st, f, pos, dynamic, &mut unwrapped)?; + let dynamic = dynamic || unwrapped; + Some(Formula { dynamic, ..fm }) } -fn read_formula_text(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option { +fn read_formula_text(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool, unwrapped: &mut bool) -> Option { let mut model = from_file(f.text.trim()); if let Some(known) = cx.known_function - && let Some(unwrapped) = unwrap_google_sheets(&model, known) + && let Some(u) = unwrap_google_sheets(&model, known) { - model = unwrapped; + model = u; + *unwrapped = true; } let text = model.as_str(); match f.attr("t").unwrap_or("normal") { @@ -667,7 +678,7 @@ fn read_formula_text(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, if !text.is_empty() { let expr = gridcraft_formula::parse(text).ok(); let cost = shared_cost(text, expr.as_ref()); - st.shared.insert(si, Shared { anchor: pos, expr, text: text.to_string(), cost }); + st.shared.insert(si, Shared { anchor: pos, expr, text: text.to_string(), unwrapped: *unwrapped, cost }); Some(Formula::new(text)) } else { let Some(sh) = st.shared.get(&si) else { @@ -690,6 +701,7 @@ fn read_formula_text(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, cx.warn("shared formulas expand to too much data; kept the cached values of the remaining cells that share them"); return None; } + *unwrapped |= sh.unwrapped; Some(match &sh.expr { Some(e) => { let dr = pos.row as i64 - sh.anchor.row as i64; diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index 4a0f4fd03..d3ba1426b 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -5,8 +5,8 @@ use std::fmt::Write as _; use gridcraft_core::{CellError, CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value}; use gridcraft_model::{ - Cell, CfOperator, CfRule, CfValueKind, Color, CondFormat, ErrorStyle, FilterCriterion, LineInfo, NoteBox, NoteCorner, Orientation, PageOrder, - Sheet, SparklineKind, Style, Table, TotalsFn, Validation, ValidationKind, Workbook, default_icon_thresholds, icon_count, + Cell, CfOperator, CfRule, CfValueKind, Color, CondFormat, ErrorStyle, FilterCriterion, Formula, LineInfo, NoteBox, NoteCorner, Orientation, + PageOrder, Sheet, SparklineKind, Style, Table, TotalsFn, Validation, ValidationKind, Workbook, default_icon_thresholds, icon_count, }; use crate::drawing::{Obj, image_content_type, image_ext}; @@ -643,13 +643,28 @@ fn write_sheet_data(s: &mut String, data: SheetCellData<'_>, sid: &dyn Fn(gridcr continue; } s.push('>'); - for &(cell_ref, cell) in cells { + for &(cell_ref, mut cell) in cells { let ov = overrides.get(&cell_ref); let ex = extra.get(&cell_ref); + let mut legacy_cell = None; let array = dynamic.get(&cell_ref).copied().or_else(|| { - let f = cell?.formula.as_ref()?; - (f.array.is_none() && crate::array_formula::needs_array(workbook, index, cell_ref, f)).then_some(RangeRef::cell(cell_ref)) + let f = cell?.formula.as_ref().filter(|f| f.array.is_none())?; + if f.dynamic { + return Some(RangeRef::cell(cell_ref)); + } + match crate::array_formula::export(workbook, index, cell_ref, f, out.functions) { + crate::array_formula::Export::Unchanged => None, + crate::array_formula::Export::Dynamic => Some(RangeRef::cell(cell_ref)), + // Every `@` is one Excel implies in a legacy formula: write it without them. + crate::array_formula::Export::Legacy(e) => { + legacy_cell = cell.map(|x| Cell { formula: Some(std::sync::Arc::new(Formula::from_expr(e))), ..x.clone() }); + None + } + } }); + if legacy_cell.is_some() { + cell = legacy_cell.as_ref(); + } write_cell( s, CellWriteInput { diff --git a/crates/xlsx/src/tests/array_export.rs b/crates/xlsx/src/tests/array_export.rs index 646bd1db3..9c7f0700a 100644 --- a/crates/xlsx/src/tests/array_export.rs +++ b/crates/xlsx/src/tests/array_export.rs @@ -333,11 +333,22 @@ fn implicit_intersection_metadata_respects_function_argument_context() { ("SUM(A1:A3)", false), ("SUMPRODUCT(A1:A3*B1:B3)", false), ("SUMPRODUCT(SQRT(A1:A3))", false), - // An explicit `@` is written as `_xlfn.SINGLE`, a dynamic-array formula. Dropping it - // instead would need the reader to put it back on legacy formulas. - ("SQRT(@A1:A3)", true), - ("@A1:A3*2", true), + // An `@` that legacy Excel implies is dropped (the reader puts it back), and so is one + // of a range an edit made one cell (`SQRT(@A1:A1)` after deleting rows under + // `SQRT(@A1:A3)`). An `@` of one value stays as `_xlfn.SINGLE` in a legacy formula, as + // Excel writes it; any other `@` makes the formula a dynamic-array one. + ("SQRT(@A1:A3)", false), + ("@A1:A3*2", false), ("@(A1:A3*2)", true), + ("SQRT(@A1)", false), + ("SQRT(@A1:A1)", false), + ("@SUM(A1:A3)", false), + ("SUMPRODUCT(@ROW(A1:A3))", true), + ("SUMPRODUCT(@SQRT(A1:A3))", true), + // Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT; dynamic-array + // Excel evaluates them as arrays. + ("SUMPRODUCT(IFERROR(VALUE(A1:A3),0))", true), + ("SUMPRODUCT(IFNA(A1:A3,0))", true), ("SQRT(INDIRECT(\"A1\"))", false), ("SUM(INDIRECT(\"A1:A3\"))", false), ("SQRT(INDIRECT(\"A1:A3\"))", true), diff --git a/crates/xlsx/src/tests/fixtures.rs b/crates/xlsx/src/tests/fixtures.rs index dadfc096a..76f40851b 100644 --- a/crates/xlsx/src/tests/fixtures.rs +++ b/crates/xlsx/src/tests/fixtures.rs @@ -728,6 +728,92 @@ fn user_defined_function_prefix_round_trips() { } } +/// A formula saved without dynamic-array metadata is a legacy formula: Excel intersects a range +/// given where one value is expected with the formula's row or column, and shows the formula with +/// `@` there. Dynamic-array formulas (`cm`) and Ctrl+Shift+Enter arrays are read as written. +#[test] +fn legacy_formulas_read_with_implicit_intersection() { + let body = r#" + 1 + 2TANH(A1:A3)0.96TANH(A1:A3)0.76TANH(A1:A3)0.76 + 3SUM(A1:A3)6SUMPRODUCT(A1:A3*A1:A3)14 + A1:A3+Vals*2INDEX(A1:C3,0,1)IF(A1>0,A1:A3,0)SUM(IF(A1:A3>1,A1:A3)) + 456 + TANH(A5:C5)COUNTIF(A1:A3,">1")+ROWS(A1:A3)ROW(A1:A3)T(A5:C5) + "#; + let names = r#"Data!$A$1:$A$3"#; + let (wb, _) = read_xlsx(&minimal(body, &[], "", names)).unwrap(); + let s = wb.sheet(0).unwrap(); + let f = |a: &str| s.cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| (f.text.clone(), f.array)).unwrap(); + let cases = [ + ("B2", "TANH(@A1:A3)"), + ("C2", "TANH(A1:A3)"), + ("B3", "SUM(A1:A3)"), + ("E3", "SUMPRODUCT(A1:A3*A1:A3)"), + ("B4", "@A1:A3+@Vals*2"), + ("E4", "@INDEX(A1:C3,0,1)"), + ("F4", "IF(A1>0,@A1:A3,0)"), + ("G4", "SUM(IF(@A1:A3>1,A1:A3))"), + ("B7", "TANH(@A5:C5)"), + ("C7", "COUNTIF(A1:A3,\">1\")+ROWS(A1:A3)"), + ("D7", "@ROW(A1:A3)"), + // T reads a reference's top-left cell rather than intersecting it. + ("E7", "@T(A5:C5)"), + ]; + for (a, text) in cases { + assert_eq!(f(a), (text.to_string(), None), "{a}"); + } + assert_eq!(f("D2"), ("TANH(A1:A3)".to_string(), RangeRef::parse("D2:D4"))); +} + +/// The caller's function facts decide what a legacy formula means: an add-in function takes the +/// range whole and its result is intersected; a parameter the library says takes an array is not +/// intersected. Saving the intersected add-in call writes it back as the legacy formula it was. +#[test] +fn legacy_formulas_use_the_callers_function_facts() { + let body = r#" + 1 + 2MYUDF(A1:A3)TANH(A1:A3) + 3 + "#; + let bytes = minimal(body, &[], "", ""); + let library = crate::FunctionInfo { is_builtin: |n| n != "MYUDF", is_scalar_arg: |n, _| n != "TANH" }; + let text = + |wb: &gridcraft_model::Workbook, a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()).unwrap(); + let (plain, _) = read_xlsx(&bytes).unwrap(); + assert_eq!(text(&plain, "B2"), "MYUDF(@A1:A3)", "without facts every function is built in"); + assert_eq!(text(&plain, "C2"), "TANH(@A1:A3)"); + let (wb, _) = crate::read_xlsx_with(&bytes, library).unwrap(); + assert_eq!(text(&wb, "B2"), "@MYUDF(A1:A3)"); + assert_eq!(text(&wb, "C2"), "TANH(A1:A3)"); + let out = crate::write_xlsx_with(&wb, crate::XlsxKind::Workbook, library).unwrap(); + let mut pkg = crate::package::Package::open(&out).unwrap(); + let sheet = pkg.read_xml("xl/worksheets/sheet1.xml").unwrap().unwrap(); + let b2 = sheet.child("sheetData").unwrap().kids("row").flat_map(|r| r.kids("c")).find(|c| c.attr("r") == Some("B2")).unwrap(); + assert_eq!(b2.attr("cm"), None); + assert_eq!(b2.child("f").map(|f| f.text.as_str()), Some("MYUDF(A1:A3)")); + let (back, _) = crate::read_xlsx_with(&out, library).unwrap(); + assert_eq!(text(&back, "B2"), "@MYUDF(A1:A3)"); +} + +/// Legacy IF does not evaluate its condition as an array inside SUMPRODUCT, nor IFERROR and IFNA +/// their arguments (that needs Ctrl+Shift+Enter): they are intersected there, while IF's results +/// stay arrays. +#[test] +fn legacy_if_and_iferror_arguments_are_intersected_inside_array_functions() { + let body = r#" + 1 + 2SUMPRODUCT(IFERROR(A1:A3/A1:A3,0))SUMPRODUCT(IFNA(A1:A3,0),A1:A3) + 3 + SUMPRODUCT(IF(A1:A3>1,1,0))SUMPRODUCT(IF(A1>1,A1:A3,0)) + "#; + let (wb, _) = read_xlsx(&minimal(body, &[], "", "")).unwrap(); + let text = |a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()).unwrap(); + assert_eq!(text("D2"), "SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0))"); + assert_eq!(text("E2"), "SUMPRODUCT(IFNA(@A1:A3,0),A1:A3)"); + assert_eq!(text("D4"), "SUMPRODUCT(IF(@A1:A3>1,1,0))"); + assert_eq!(text("E4"), "SUMPRODUCT(IF(A1>1,A1:A3,0))"); +} /// `customWidth="1"` pins a column even when its width rounds to the default: Standard Width does not /// move it, and a save must keep its `` (openpyxl writes 9.08 for Excel's 8.43, #1607). #[test] diff --git a/crates/xlsx/src/tests/roundtrip.rs b/crates/xlsx/src/tests/roundtrip.rs index eae22d829..ab55c8445 100644 --- a/crates/xlsx/src/tests/roundtrip.rs +++ b/crates/xlsx/src/tests/roundtrip.rs @@ -1012,10 +1012,11 @@ fn icon_sets_always_write_one_cfvo_per_icon() { assert_eq!(rules, [3, 4, 5]); } -/// Files have no `@` operator: Excel writes `@x` as `_xlfn.SINGLE(x)` in a dynamic-array -/// formula, and reads it back as `@x`. +/// Files have no `@` operator. An `@` that legacy Excel implies is written without it in a +/// legacy formula (no dynamic-array metadata); any other `@x` is written as `_xlfn.SINGLE(x)` +/// in a dynamic-array formula. Both read back as written in the model. #[test] -fn implicit_intersection_is_written_as_single() { +fn implicit_intersection_is_written_as_legacy_or_single() { let mut wb = Workbook::new(); let s = wb.sheet_mut(0).unwrap(); for (r, n) in [(0, 1.0), (1, 3.0), (2, 5.0)] { @@ -1023,16 +1024,258 @@ fn implicit_intersection_is_written_as_single() { } s.cells.set(at("D3"), Cell { value: Value::Number(10.0), ..Cell::formula(Formula::new("@A1:A3*2")) }); s.cells.set(at("D4"), Cell { value: Value::Number(2.0), ..Cell::formula(Formula::new("A1*2")) }); + s.cells.set(at("D5"), Cell { value: Value::Number(2.0), ..Cell::formula(Formula::new("@(A1:A3*2)")) }); let bytes = write_xlsx(&wb).unwrap(); let (xml, metadata) = sheet1_xml(&bytes); - assert!(xml.contains("_xlfn.SINGLE(A1:A3)*210"), "{xml}"); + assert!(xml.contains("A1:A3*210"), "{xml}"); assert!(xml.contains("A1*2"), "{xml}"); + assert!(xml.contains("_xlfn.SINGLE(A1:A3*2)2"), "{xml}"); assert!(!xml.contains('@'), "{xml}"); assert!(metadata); let (back, _) = read_xlsx(&bytes).unwrap(); let sh = back.sheet(0).unwrap(); - assert_eq!(sh.cell(at("D3")).and_then(|c| c.formula.as_ref()).map(|f| f.text.as_str()), Some("@A1:A3*2")); - assert_eq!(sh.cell(at("D3")).and_then(|c| c.formula.as_ref()).and_then(|f| f.array), None); + let text = |a: &str| sh.cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()); + assert_eq!(text("D3").as_deref(), Some("@A1:A3*2")); + assert_eq!(text("D4").as_deref(), Some("A1*2")); + assert_eq!(text("D5").as_deref(), Some("@(A1:A3*2)")); + assert_eq!(sh.cell(at("D5")).and_then(|c| c.formula.as_ref()).and_then(|f| f.array), None); +} + +/// A legacy formula read from a file (shown with Excel's implicit `@`) is saved as it was read, +/// without dynamic-array metadata; a dynamic-array formula keeps its metadata. +#[test] +fn legacy_and_dynamic_formulas_keep_their_kind() { + let body = r#" + 1 + 2TANH(A1:A3)+A1:A34TANH(A1:A3)0.76TANH(A1:A3)0.76 + 3SUM(A1:A3)6 + "#; + let (wb, _) = read_xlsx(&super::minimal(body, &[], "", "")).unwrap(); + let text = |wb: &Workbook, a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| (f.text.clone(), f.array)).unwrap(); + assert_eq!(text(&wb, "B2"), ("TANH(@A1:A3)+@A1:A3".to_string(), None)); + let bytes = write_xlsx(&wb).unwrap(); + let (xml, _) = sheet1_xml(&bytes); + assert!(xml.contains("TANH(A1:A3)+A1:A3"), "{xml}"); + assert!(xml.contains("SUM(A1:A3)"), "{xml}"); + // Uncalculated, the spill range is not known yet; the formula stays dynamic. + assert!(xml.contains("TANH(A1:A3)"), "{xml}"); + assert!(xml.contains("TANH(A1:A3)"), "{xml}"); + let (back, _) = read_xlsx(&bytes).unwrap(); + for a in ["B2", "B3", "C2", "D2"] { + assert_eq!(text(&back, a), text(&wb, a), "{a}"); + } +} + +/// The `` text and `cm` attribute of a cell in a written package's first sheet. +fn written_formula(bytes: &[u8], a: &str) -> (Option, Option) { + let mut pkg = Package::open(bytes).unwrap(); + let sheet = pkg.read_xml("xl/worksheets/sheet1.xml").unwrap().unwrap(); + let c = sheet.child("sheetData").unwrap().kids("row").flat_map(|r| r.kids("c")).find(|c| c.attr("r") == Some(a)).unwrap(); + (c.child("f").map(|f| f.text.clone()), c.attr("cm").map(str::to_string)) +} + +/// A long operator chain is analysed in full (its ranges are intersected), and so are 64 nested +/// functions with parenthesised arguments (128 levels), 200 nested functions and a long name used +/// several times. A legacy formula the analysis cannot finish (names too large to analyse) reads +/// with one whole-formula `@`, so it shows one value as in legacy Excel, and saves as read. +/// Saving writes none of them as `_xlfn.SINGLE`, which pre-dynamic-array Excel shows as #NAME?, +/// nor with `cm`. (`legacy_array_functions.rs` in the engine checks the same on a calculated +/// workbook.) +#[test] +fn long_and_deeply_nested_legacy_formulas_never_gain_single() { + let chain = (1..=70).map(|r| format!("B{r}")).collect::>().join("+"); + let nested = |depth: usize, inner: &str| format!("{}{inner}{}", "ABS(".repeat(depth), ")".repeat(depth)); + let parens = |depth: usize, inner: &str| format!("{}{inner}{}", "ABS((".repeat(depth), "))".repeat(depth)); + let big = vec!["Data!$B$1"; 399].join("+"); + let uses = format!("{}A1:A3", "Big+".repeat(6)); + let parts = (1..=30).map(|i| format!("Part_{i}")).collect::>().join("+"); + let body = format!( + r#" + 1 + 2{chain} + 3A1:A3+{chain} + {} + {uses} + {} + {} + {parts} + "#, + nested(64, "A1:A3"), + nested(200, "A1:A3"), + parens(64, "A1:A3"), + ); + let part = format!("Data!$A$1:$A$3+{}", vec!["Data!$B$1"; 599].join("+")); + let defs: String = (1..=30).map(|i| format!(r#"{part}"#)).collect(); + let names = format!(r#"{big}{defs}"#); + let (wb, _) = read_xlsx(&super::minimal(&body, &[], "", &names)).unwrap(); + let text = |wb: &Workbook, a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()).unwrap(); + assert_eq!(text(&wb, "D2"), chain); + assert_eq!(text(&wb, "D3"), format!("@A1:A3+{chain}")); + assert_eq!(text(&wb, "D4"), nested(64, "@A1:A3")); + assert_eq!(text(&wb, "D5"), format!("{}@A1:A3", "Big+".repeat(6))); + assert_eq!(text(&wb, "D6"), nested(200, "@A1:A3")); + assert_eq!(text(&wb, "D7"), parens(64, "@A1:A3")); + assert_eq!(text(&wb, "D8"), format!("@({parts})")); + let bytes = write_xlsx(&wb).unwrap(); + assert_eq!(written_formula(&bytes, "D2"), (Some(chain.clone()), None)); + assert_eq!(written_formula(&bytes, "D3"), (Some(format!("A1:A3+{chain}")), None)); + assert_eq!(written_formula(&bytes, "D4"), (Some(nested(64, "A1:A3")), None)); + assert_eq!(written_formula(&bytes, "D5"), (Some(uses), None)); + assert_eq!(written_formula(&bytes, "D6"), (Some(nested(200, "A1:A3")), None)); + assert_eq!(written_formula(&bytes, "D7"), (Some(parens(64, "A1:A3")), None)); + assert_eq!(written_formula(&bytes, "D8"), (Some(parts), None)); + let (xml, _) = sheet1_xml(&bytes); + assert!(!xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}"); +} + +/// `@` of a union keeps the union's parentheses when saved, with or without `_xlfn.SINGLE`: +/// without them the comma separates function arguments or the text does not parse. Redundant +/// parentheses under `@` survive too, as only those the reader adds back are dropped. +#[test] +fn implicit_intersection_keeps_its_parentheses() { + let mut wb = Workbook::new(); + let s = wb.sheet_mut(0).unwrap(); + let formulas = [ + "@(A1:A2,C1:C2)+1", + "SQRT(@(A1:A2,C1:C2))", + "@(A1:A2,C1:C2)+A1:A3", + "@((A1:A2,C1:C2))", + "SQRT(@((A1:A2,C1:C2)))", + "@(((A1:A2,C1:C2)))", + "@((A1:A3*2))", + "@(A1:A3)", + "@(A1:A3*2)", + "@(-A1:A3)", + ]; + for (i, f) in formulas.iter().enumerate() { + s.cells.set(CellRef::new(4 + i as u32, 4), Cell::formula(Formula::new(f))); + } + let bytes = write_xlsx(&wb).unwrap(); + let (back, _) = read_xlsx(&bytes).unwrap(); + for (i, f) in formulas.iter().enumerate() { + let cell = back.sheet(0).unwrap().cell(CellRef::new(4 + i as u32, 4)).and_then(|c| c.formula.as_ref()); + assert_eq!(cell.map(|c| (c.text.as_str(), c.expr().is_some())), Some((*f, true)), "{f}"); + } +} + +/// A legacy formula read from a file saves back exactly as it was read, without `cm` or +/// `_xlfn.SINGLE`: a union already in parentheses where one value is expected, a name whose +/// definition holds IF of a range (legacy Excel evaluates a name's definition as an array), and a +/// multi-valued result in the file's own parentheses. A union given to a function is intersected +/// (`SQRT((@(A1:A2,C1:C2)))`), as the function would otherwise spill it; one under an operator is +/// shown as Excel shows it. The engine's `legacy_array_functions.rs` checks the same formulas on a +/// calculated workbook, where nothing spills. +#[test] +fn legacy_unions_in_parentheses_and_names_with_if_save_as_read() { + let body = r#" + 14SQRT((A1:A2,C1:C2)) + 29(A1:A2,C1:C2)+1 + 3SUM(Flag)SQRT(Flag) + ({1,2}+1){1,2}+1(((MYUDF(A1)*2))) + "#; + let names = r#"IF(Data!$A$1:$A$3>1,1,0)"#; + let (wb, _) = read_xlsx(&super::minimal(body, &[], "", names)).unwrap(); + let text = |a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()).unwrap(); + let bytes = write_xlsx(&wb).unwrap(); + let cases = [ + ("E1", "SQRT((A1:A2,C1:C2))"), + ("E2", "(A1:A2,C1:C2)+1"), + ("E3", "SUM(Flag)"), + ("F3", "SQRT(Flag)"), + ("E4", "({1,2}+1)"), + ("F4", "{1,2}+1"), + ("G4", "(((MYUDF(A1)*2)))"), + ]; + let wrong: Vec<_> = + cases.iter().map(|(a, f)| (*a, *f, written_formula(&bytes, a))).filter(|(_, f, w)| *w != (Some(f.to_string()), None)).collect(); + assert!(wrong.is_empty(), "{wrong:?}"); + assert_eq!(text("E1"), "SQRT((@(A1:A2,C1:C2)))"); + assert_eq!(text("E2"), "(A1:A2,C1:C2)+1"); + assert_eq!(text("F3"), "@SQRT(Flag)"); + let (xml, _) = sheet1_xml(&bytes); + assert!(!xml.contains("SINGLE") && !xml.contains("cm=") && !xml.contains("t=\"array\""), "{xml}"); +} + +/// An implied `@` whose operand an edit has made a single cell (deleting rows under +/// `SQRT(@A1:A3)` leaves `A1:A1`, and a name's `$A$1:$A$3` becomes `$A$1:$A$1`) is the one legacy +/// Excel implies for that range: the formula still saves as a legacy one, without +/// `_xlfn.SINGLE` or `cm`. +#[test] +fn implied_intersection_made_redundant_by_an_edit_saves_legacy() { + let body = r#" + SQRT(A1:A3)SQRT(Rng) + "#; + let names = r#"Data!$A$1:$A$3"#; + let (mut wb, _) = read_xlsx(&super::minimal(body, &[], "", names)).unwrap(); + let text = |wb: &Workbook, a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()).unwrap(); + assert_eq!(text(&wb, "B5"), "SQRT(@A1:A3)"); + assert_eq!(text(&wb, "C5"), "SQRT(@Rng)"); + // What deleting rows 2:3 leaves. + wb.sheet_mut(0).unwrap().cells.set(at("B5"), Cell::formula(Formula::new("SQRT(@A1:A1)"))); + wb.names.iter_mut().find(|n| n.name == "Rng").unwrap().formula = "Data!$A$1:$A$1".into(); + let bytes = write_xlsx(&wb).unwrap(); + assert_eq!(written_formula(&bytes, "B5"), (Some("SQRT(A1:A1)".into()), None)); + assert_eq!(written_formula(&bytes, "C5"), (Some("SQRT(Rng)".into()), None)); + let (xml, _) = sheet1_xml(&bytes); + assert!(!xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}"); +} + +/// Excel keeps an `@` that legacy evaluation does not imply as `_xlfn.SINGLE` inside a legacy +/// formula (Excel Online's UnionSpillIntersectionOperatorTest.xlsx, EPPlus's +/// DefinedNameRelative.xlsx): such formulas save back with it, without `cm`. The reader shows the +/// implied intersection of a union that is an operator's operand as Excel does, without `@`, and a +/// spill reference's `@` without parentheses. A typed `@` of a single value saves the same way. +#[test] +fn single_in_a_legacy_formula_saves_as_read() { + let body = r#" + 1_xlfn.SINGLE(SUM(_xlfn.ANCHORARRAY(F5)))_xlfn.ANCHORARRAY(F5)%_xlfn.ANCHORARRAY($F$5) + 1(F25,F26)%SUM((F25,F26)%) + _xlfn.SINGLE(5)_xlfn.SINGLE(RelativeRow)_xlfn.SINGLE(H12) + _xlfn.SINGLE(SUM(F25:F26)) + (F25, _xlfn.SINGLE(F26)) + "#; + let names = r#"Data!$H1"#; + let (mut wb, _) = read_xlsx(&super::minimal(body, &[], "", names)).unwrap(); + let text = |wb: &Workbook, a: &str| wb.sheet(0).unwrap().cell(at(a)).and_then(|c| c.formula.as_ref()).map(|f| f.text.clone()).unwrap(); + let shown = [ + ("O6", "@SUM(F5#)"), + ("J6", "@F5#%"), + ("K6", "@$F$5#"), + ("G25", "(F25,F26)%"), + ("H25", "SUM((F25,F26)%)"), + ("F83", "@5"), + ("G83", "@RelativeRow"), + ("H83", "@H12"), + ("F85", "@SUM(F25:F26)"), + ("F102", "(F25,@F26)"), + ]; + let wrong: Vec<_> = shown.iter().map(|(a, f)| (*a, *f, text(&wb, a))).filter(|(_, f, t)| t != f).collect(); + assert!(wrong.is_empty(), "{wrong:?}"); + let s = wb.sheet_mut(0).unwrap(); + s.cells.set(at("B1"), Cell::formula(Formula::new("@SQRT(A1)"))); + s.cells.set(at("B2"), Cell::formula(Formula::new("@MUNIT(3)"))); + let bytes = write_xlsx(&wb).unwrap(); + let saved = [ + ("O6", "_xlfn.SINGLE(SUM(_xlfn.ANCHORARRAY(F5)))"), + ("J6", "_xlfn.ANCHORARRAY(F5)%"), + ("K6", "_xlfn.ANCHORARRAY($F$5)"), + ("G25", "(F25,F26)%"), + ("H25", "SUM((F25,F26)%)"), + ("F83", "_xlfn.SINGLE(5)"), + ("G83", "_xlfn.SINGLE(RelativeRow)"), + ("H83", "_xlfn.SINGLE(H12)"), + ("F85", "_xlfn.SINGLE(SUM(F25:F26))"), + ("F102", "(F25,_xlfn.SINGLE(F26))"), + ("B1", "_xlfn.SINGLE(SQRT(A1))"), + ("B2", "_xlfn.MUNIT(3)"), + ]; + let wrong: Vec<_> = + saved.iter().map(|(a, f)| (*a, *f, written_formula(&bytes, a))).filter(|(_, f, w)| *w != (Some(f.to_string()), None)).collect(); + assert!(wrong.is_empty(), "{wrong:?}"); + let (back, _) = read_xlsx(&bytes).unwrap(); + for (a, _) in saved { + assert_eq!(text(&back, a), text(&wb, a), "{a}"); + } } #[test] diff --git a/crates/xlsx/src/write.rs b/crates/xlsx/src/write.rs index b072043af..5d06d54a0 100644 --- a/crates/xlsx/src/write.rs +++ b/crates/xlsx/src/write.rs @@ -9,7 +9,7 @@ use gridcraft_model::{CalcMode, CalcSettings, PhoneticString, ShowObjects, Style use crate::package::ZipOut; use crate::xml::{esc, esc_attr}; -use crate::{IoError, XlsxKind}; +use crate::{FunctionInfo, IoError, XlsxKind}; pub const NS_MAIN: &str = "http://schemas.openxmlformats.org/spreadsheetml/2006/main"; pub const NS_REL: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"; @@ -132,6 +132,8 @@ pub struct Out { pub cell_pictures: crate::richdata::PicturesOut, /// Kept (pass-through) parts written so far. pub kept: crate::passthrough::KeptOut, + /// Decides which formulas need dynamic-array metadata. + pub functions: FunctionInfo, /// The code of the Python cells written so far, in order: script `i` of the Python part. pub python_scripts: Vec, } @@ -210,8 +212,8 @@ fn check_sheet_names(wb: &Workbook) -> Result<(), IoError> { /// Workbook name written in a PivotChart's link when the file name isn't known. const DEFAULT_BOOK_NAME: &str = "Workbook.xlsx"; -pub fn write_xlsx(wb: &Workbook, kind: XlsxKind) -> Result, IoError> { - write_xlsx_named(wb, kind, None) +pub fn write_xlsx(wb: &Workbook, kind: XlsxKind, functions: FunctionInfo) -> Result, IoError> { + write_xlsx_named_with(wb, kind, None, functions) } /// The `` attributes beyond `activeTab`: the window's display settings @@ -238,7 +240,7 @@ fn workbook_view_attrs(wb: &Workbook) -> String { /// Writes the workbook for the file `book_name` (a bare file name). A PivotChart is bound to its /// PivotTable in Excel only when its link starts with the workbook in brackets. -pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>) -> Result, IoError> { +pub fn write_xlsx_named_with(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>, functions: FunctionInfo) -> Result, IoError> { if wb.sheets.is_empty() { return Err(IoError::Format("a workbook needs at least one sheet".into())); } @@ -268,6 +270,7 @@ pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>) needs_calc: false, cell_pictures: Default::default(), kept: Default::default(), + functions, python_scripts: vec![], }; out.kept.reserve_vml_blocks(wb); @@ -578,3 +581,7 @@ fn write_defined_names(wb: &Workbook, kept: &crate::passthrough::WorkbookKept, c /// Cell metadata declaring dynamic-array formulas (MS-XLSX `XLDAPR`). const METADATA: &str = "\n"; +/// Writes a workbook with a caller-supplied file name and the default function facts. +pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>) -> Result, IoError> { + write_xlsx_named_with(wb, kind, book_name, FunctionInfo::default()) +} diff --git a/docs/decisions.md b/docs/decisions.md index 7ccc445c8..0eea97675 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -26,9 +26,9 @@ GridCraft evaluates ordinary formulas with array semantics even when the final r The exporter inspects the parsed formula without evaluating it. It distinguishes range-taking functions such as `SUM` from scalar arguments and operators: `SUM(A1:A3)` and `A1*2` stay ordinary, while `SUM(A1:A3*B1:B3)` and `SQRT(A1:A3)` retain array evaluation. Legacy array-aware contexts such as `SUMPRODUCT` do not need an extra marker. Explicit `@` scalarizes its result without hiding array-sensitive operations inside it. -Names are resolved using workbook/sheet scope, and structured references distinguish one table-row cell from a column. Literal `INDIRECT` and `INDEX` arguments permit tighter classification; unresolved reference dimensions are conservatively treated as potentially multiple cells. Name/AST traversal is bounded, with array preservation as the fallback. This is an export classifier, not a claim of exact Excel metadata parity for every function or user-defined function. It stays within the format crate's existing dependencies and does not change calculation or import behavior. +Names are resolved using workbook/sheet scope, and structured references distinguish one table-row cell from a column. Literal `INDIRECT` and `INDEX` arguments permit tighter classification; unresolved reference dimensions are conservatively treated as potentially multiple cells. Name/AST traversal is bounded. A formula whose array behaviour the analysis doesn't know (a user-defined or unknown function, an uncertain parameter, a recursive name) is saved with `cm`, unless it has an explicit `@` (below). One without an explicit `@` that the analysis cannot finish, but whose parts are all known, is saved as read, without `cm`. One with an explicit `@` that the analysis cannot finish or doesn't know is saved as the legacy text the reader turns back into it (the formula without its `@`, or without a whole-formula `@`) when there is one, and with `cm` otherwise. This is an export classifier, not a claim of exact Excel metadata parity for every function or user-defined function. It stays within the format crate's existing dependencies. -This is a writer policy; interpreting implicit intersection in imported legacy formulas remains separate work. See Microsoft's [Formula vs. Formula2 documentation](https://learn.microsoft.com/en-us/office/vba/excel/concepts/cells-and-ranges/range-formula-vs-formula2). +The reader applies the same classification in reverse: a formula without dynamic-array metadata (`cm`) and not entered as a CSE array is a legacy formula, and gets the `@` Excel implies where a range meets a single-value parameter, an operator or the cell result. Inside SUMPRODUCT and other array-evaluating parameters, IF's condition and IFERROR's and IFNA's arguments are still intersected, as legacy Excel does without Ctrl+Shift+Enter; IF's results are not. The writer drops an `@` that legacy Excel would imply (also when an edit has shrunk its range to one cell) and saves that formula without `cm`; an `@` of a single value stays `_xlfn.SINGLE` in that legacy formula, as Excel writes it; any other `@` makes the formula a dynamic-array one. A legacy formula too large to analyse, or using functions whose legacy argument semantics are unknown (the array arguments of dynamic-array functions, LAMBDA), is read as it is with only a whole-formula `@` when its result may be several values, so it never spills, and is saved as it was read, without `cm`. Which names are built-in functions and which parameters take a single value comes from the function library, which the format crate may not depend on: the caller passes a `gridcraft_xlsx::FunctionInfo` (`read_xlsx_with`, `write_xlsx_with`; the engine supplies the library's answers), and `read_xlsx`/`write_xlsx`/`write_xlsx_as` use the crate's own table alone. See Microsoft's [Formula vs. Formula2 documentation](https://learn.microsoft.com/en-us/office/vba/excel/concepts/cells-and-ranges/range-formula-vs-formula2). ## Static cell pictures diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index bd3e26c89..12597f313 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -79,6 +79,7 @@ past the parser's limit, and a wrapper inside the text. **Saving writes the real | Format | Excel | GridCraft read | GridCraft write | Fidelity / notes | Tests | |---|---|---|---|---|---| | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), form controls (buttons, scroll bars and spin buttons: control properties parts, `` and VML `x:ClientData`; a VML-only control is read too; a button's command, fixed input range or output cell travels in a hidden defined name and comes back only onto the same button while the file still assigns its macro, or none for a command), protection, calc settings, show objects (`workbookPr showObjects`). **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), other form controls (check boxes, list boxes, drop-downs; written in the same `` element and VML drawing as the modelled ones, also those only the VML drawing lists, with no `` entry), ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | +| `.xlsx` Legacy formula behavior | R/W | yes | yes | Formulas without dynamic-array metadata (`cm`) open with Excel's implicit `@`; implied operators are removed on save without `cm`, while explicit single-value `@` is kept as `_xlfn.SINGLE`; formulas the analyzer cannot finish are saved as read. | `roundtrip.rs`, `fixtures.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, notes (with their shown/hidden state) and threaded comments with replies, resolved state and timestamps (`threadedComments` and `persons` parts, #300), hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings, show objects (`workbookPr showObjects`). **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, Allow Edit Ranges (`protectedRanges` with their password hashes, #307), calc settings, show objects (`workbookPr showObjects`). **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes (the major/minor `latin`, East Asian `ea` and complex-script `cs` typefaces and the per-script `` lists are kept through a save), merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), pictures in headers and footers (`&G`, #1257: their VML drawing and images are written back unchanged; they are not shown, and can't be inserted), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | @@ -209,6 +210,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | 2026-10-14 | trivial | Formula parser: text longer than Excel's 8,192 characters, and postfix chains (`1%%%…`, `f(1)(1)…`) past the 256-level nesting limit, are parse errors (the cell keeps its formula text and cached value) instead of trees deep enough to overflow the stack when a hostile file is opened, saved or dropped | | 2026-10-14 | minor | XLSX formulas read with dynamic-array metadata (`cm`) keep it on save, so Excel does not recalculate them with implicit intersection | | 2026-10-14 | minor | PDF export writes clickable annotations for safe cell URLs and destinations on printed pages; CLI `convert` accepts `.pdf` output (#954) | +| 2026-10-13 | minor | Legacy vs dynamic-array formulas: no `cm` reads as legacy (implicit `@` inserted), implied `@` written back without `cm` | | 2026-10-13 | trivial | XLSX export keeps a picture shared by several sheets as one media part (#985) | | 2026-10-12 | trivial | XLSX chart sheets are kept on save (#727) | | 2026-10-12 | trivial | CSV and TSV export is capped at 512 MiB and refuses a larger used range with an error (#528) | diff --git a/docs/function-parity.md b/docs/function-parity.md index 87081e460..7dec2c03c 100644 --- a/docs/function-parity.md +++ b/docs/function-parity.md @@ -83,6 +83,7 @@ This improves series preparation; it does not change the model-fitting or season | Dynamic arrays, spill, `#SPILL!`, spill operator `A1#` | done; stale spills after a structural edit or a cleared blocker fixed (#133, #139; PRs #148, #157) | | Recalculation after clearing large sparse ranges | every stored cell changed by Clear is reported to recalculation, without a million-coordinate cutoff; removed formula spills are cleared, and removing empty merges still unblocks spills (#1032) | | SUBTOTAL / AGGREGATE input size | references trim to the used range; all areas, arrays and values of one call are charged against the 16-million-cell limit before any cell is read (skipped hidden, nested and error cells count), and `#NUM!` past it (#2175) | +| Implicit intersection `@`, legacy CSE array formulas | done; formulas saved without dynamic-array metadata open with Excel's implicit `@` inserted where a range meets a single-value parameter, operator or cell result, and calculate like Excel's cached values (cells differing from Excel's cached values: POI's FormulaEvalTestData_Copy 197 → 3, StructuredReferences 2 → 0). T reads a reference's top-left cell | | Implicit intersection `@`, legacy CSE array formulas | done; legacy array range fill (PR #156) | | Implicit intersection `@`, legacy CSE array formulas | done, the entered range filled (PR #156) | | Array lifting: an array in a scalar argument evaluates the function per element | done. The Analysis ToolPak functions (EOMONTH, EDATE, WORKDAY(.INTL), NETWORKDAYS(.INTL), YEARFRAC, WEEKNUM, the COUP*, PRICE*, YIELD*, ODD*, TBILL*, ACCRINT*, AMOR*, DISC, DURATION, CUMIPMT/CUMPRINC, EFFECT, NOMINAL, DOLLARDE/FR, XNPV/XIRR rate and guess, FVSCHEDULE principal, ISEVEN/ISODD, QUOTIENT, MROUND, FACTDOUBLE, SQRTPI, RANDBETWEEN, SERIESSUM, the base conversions, CONVERT, DELTA, GESTEP, ERF/ERFC, BESSEL*, COMPLEX and IM* except IMSUM/IMPRODUCT) give `#VALUE!` for a reference to more than one cell in a value argument, as Excel 365 does in both dynamic-array and legacy formulas; array constants and computed arrays (`A1:A3+0`) still lift, and range arguments (holidays, schedule, cash flows) and the .INTL weekend accept ranges. Difference: BYROW/BYCOL pass their row or column to the LAMBDA as values, so `BYROW(A1:B3,LAMBDA(r,EOMONTH(r,0)))` lifts where Excel gives `#VALUE!` | @@ -174,6 +175,8 @@ workbooks; results checked in as data, not Excel files) and fixing what it finds | 2026-10-13 | minor | Clear reports changed stored cells sparsely, including those beyond the first million coordinates; dependent formulas and removed spills update, while clearing empty merges still unblocks spills (#1032) | | 2026-10-14 | minor | Set precision as displayed: results, spills and typed or reformatted numbers are stored as their format shows them when the option is on (#1343) | | 2026-10-14 | minor | Bounded references up to 1,048,576 cells (`D3:AZ400`) keep their full shape instead of stopping at the used range, so `SUMPRODUCT(D3:AZ400*B3:B400,(A3:A400=x)*(D1:AZ1=y))` broadcasts like Excel and `ROWS`, `COUNTBLANK` and `--(r="")` count the blanks; references that run to the edge of the sheet (`A:A`) still stop at the used range (#1010) | +| 2026-10-13 | minor | Legacy formulas from files (no dynamic-array metadata) evaluate with implicit intersection: cells differing from Excel's cached values 68,964 → 49,866 across 4,224 test-corpus workbooks (none gets more) | +| 2026-10-13 | minor | Clear reports changed stored cells sparsely, including those beyond the first million coordinates; dependent formulas and removed spills update, while clearing empty merges still unblocks spills (#1032) | | 2026-10-12 | minor | USDOLLAR added as a Compatibility function (not on Microsoft's list, so the counts don't change); MIRR with no positive flow is -1, COUNTIF's bare `"<="` / `">="` skip empty strings, INDIRECT with `a1` = FALSE gives `#REF!` for A1-style text (#738) | | 2026-10-11 | minor | TEXT / number formats fixed against Excel-cached results: fraction literals and zero fractions, `0` denominators zero padded, `?` pads the thousands separator, booleans use the text section (226 mismatches → 0 in 352 TEXT cells) | | 2026-10-11 | minor | Iterative calculation: circular references iterate up to Maximum Iterations / Maximum Change when enabled | From ec12eba83e75816b9bbb9eac191159572dbc5d27 Mon Sep 17 00:00:00 2001 From: RockerHX Date: Tue, 10 Nov 2026 19:44:19 +0800 Subject: [PATCH 2/6] UI.1265: align CJK fallback text with the Latin baseline (#3081) * UI.1265: align CJK fallback text with the Latin baseline Use epaint's existing baseline alignment for system CJK fallback faces and embedded faces tagged with CJK scripts. Mixed CJK, Latin and digits then share a baseline in cells, editors and window titles. Add regression coverage for zoom, font styles, wrapping, fixed-height clipping, vertical alignment, editor height and title truncation. Refs #1265, #2621. * UI.1265: assert title baselines and document visual validation * UI.1265: add baseline comparison and review validation * UI.1265: adapt tooltip regression after upstream merge The upstream shortcut update uses the runtime OS to choose tooltip text. Use the harness OS for the existing split-button test expectation too. Validation: cargo xtask ci; all 10 steps pass; 3,002 tests passed, 1 ignored, no exclusions. * UI.1265: propagate CJK fallback to installed fonts * UI.1265: align fallback baselines across cells * UI.1265: align wrapped and unwrapped cell text * UI.1265: preserve blank lines in wrapped cell alignment * UI.1265: preserve centered fallback placement * UI.1265: find CJK cell text by its visible bounds Use the overlap of text ink, clip rect and cell rect to locate painted text instead of testing a fixed offset from the galley origin. Baseline-aligned fallbacks can move that origin above the cell on Ubuntu. Keep the existing baseline and wrapping assertions unchanged. Validation: cargo xtask ci on macOS; all 3 cjk_baseline tests on Ubuntu 24.04 x86_64. --------- Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> --- ROADMAP.md | 5 +- contributors/people.toml | 3 + crates/ui-egui/src/grid/cell_paint.rs | 35 ++- crates/ui-egui/src/grid/quadrant.rs | 17 +- crates/ui-egui/src/grid/tests.rs | 67 +++++- crates/ui-egui/src/ribbon/title_bar.rs | 9 +- crates/ui-egui/src/theme/font_fallback.rs | 4 +- crates/ui-egui/src/theme/font_setup.rs | 31 ++- crates/ui-egui/tests/cjk_baseline.rs | 206 ++++++++++++++++++ .../ui-egui/tests/it/text_overflow_scroll.rs | 4 +- docs/architecture.md | 9 +- docs/localization-parity.md | 8 +- docs/ui-parity.md | 27 ++- 13 files changed, 390 insertions(+), 35 deletions(-) create mode 100644 crates/ui-egui/tests/cjk_baseline.rs diff --git a/ROADMAP.md b/ROADMAP.md index 5906fda99..36f8919a3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-05 · **Change:** minor (CJK fallback text shares the Latin baseline; earlier: form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -148,6 +148,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-31 | CSV and text open detects each column's date order: a value that reads only day first (`31/05/2026`) makes the column day first, a value that reads only month first (`12/31/2026`) makes it month first, an all-ambiguous column keeps month first with a warning, and a column with both kinds stays text with a warning (#2765) | | 2026-10-31 | Recently Used lists the functions most recently entered in a formula from any source (ribbon, control channel, MCP, scripts), kept with the preferences and returned by `formulas.recentlyUsed`; the Insert Function dialog takes Excel's title and a working category list (Most Recently Used, All, each category) | | 2026-10-31 | Formulas › Function Library as a big Insert Function button, the AutoSum split button and three columns of labelled small buttons, each category book in its own colour; category menus end with Insert Function… on that category; More Functions cascades into the remaining categories; Home › Editing shares the AutoSum menu; the MR key tip opens Recently Used | +| 2026-10-31 | System and embedded CJK fallback text shares the Latin baseline in cells, editors and window titles (#1265, #2621). Symbol and emoji alignment, manual row heights and clipping are unchanged | | 2026-10-26 | Arabic bidi display/editing preserve logical source text; per-cell reading direction is editable and survives undo, XLSX and conditional-format priority | | 2026-10-26 | Arabic text shows in cells and the interface through a system, fontconfig or craft-fonts Arabic face; workbook fonts keep their script fallbacks | | 2026-10-25 | AutoSum over a selection does what Excel does: a selected row gets one total to its right (in its last cell if that is empty), not a one-cell total under each cell; a block whose last column is empty gets each row's total there, one whose last row is empty each column's, and with both the corner gets the total of the block; otherwise totals go under each column as before | @@ -265,6 +266,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| | 2026-11-06 | trivial | Progress log: scroll positions follow their sheet through sheet inserts, deletes and moves (#3485) | +| 2026-11-05 | minor | Reconcile upstream form-control progress with the CJK baseline update | | 2026-11-02 | minor | Form controls; catalog 277 / 292 → 281 / 296 (Developer › Controls); progress log row | | 2026-11-02 | minor | Progress log: RefEdit reference boxes in dialogs | | 2026-11-02 | minor | Progress log: Python mode in the formula bar | @@ -277,6 +279,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-31 | minor | File-page text fields keep the view open when Esc cancels an edit (#3035) | | 2026-10-31 | minor | Progress log: Recently Used and Insert Function categories | | 2026-10-31 | minor | Progress log: Function Library | +| 2026-10-31 | minor | Progress log: shared CJK/Latin baselines in cells, editors and window titles | | 2026-10-26 | minor | Progress log: Arabic bidi display and editing | | 2026-10-26 | minor | Progress log: Arabic font fallback | | 2026-10-25 | minor | Progress log: AutoSum over a selected row, and a selection's empty last row and column | diff --git a/contributors/people.toml b/contributors/people.toml index 1b8bcf996..395aecf6e 100644 --- a/contributors/people.toml +++ b/contributors/people.toml @@ -110,3 +110,6 @@ sync_github_name = true [people.maxm11] sync_github_name = true + +[people.RockerHX] +sync_github_name = true diff --git a/crates/ui-egui/src/grid/cell_paint.rs b/crates/ui-egui/src/grid/cell_paint.rs index ee03f2726..f16e5e5bc 100644 --- a/crates/ui-egui/src/grid/cell_paint.rs +++ b/crates/ui-egui/src/grid/cell_paint.rs @@ -133,11 +133,10 @@ pub(super) fn display_text( } pub(crate) fn wrapped_cell_text_y(valign: VAlign, rect: Rect, gh: f32, z: f32) -> f32 { - let pad_v = 1.5 * z; - let y_top = rect.top() + pad_v; + let y_top = rect.top() + 1.5 * z; // Not clamped to `y_top`: text taller than the row overflows upward (Bottom) or both ways // (Center), and the cell's clip rect trims it, so Middle and Bottom stay distinct from Top. - let y_bottom = rect.bottom() - gh - pad_v; + let y_bottom = rect.bottom() - gh - 0.5 * z; match valign { VAlign::Top => y_top, VAlign::Center | VAlign::Justify | VAlign::Distributed => (y_top + y_bottom) / 2.0, @@ -145,15 +144,29 @@ pub(crate) fn wrapped_cell_text_y(valign: VAlign, rect: Rect, gh: f32, z: f32) - } } +/// Bounds of the primary font's line boxes, excluding extra row height added only for fallbacks. +/// Centered fallbacks do not identify that baseline; empty or centered-only rows use their layout bounds. +pub(crate) fn galley_baseline_bounds(galley: &egui::Galley) -> (f32, f32) { + let row_bounds = |row: &egui::epaint::text::PlacedRow| { + row.glyphs + .iter() + .find(|glyph| { + glyph.font_face_align == egui::epaint::text::FallbackAlign::Baseline + || (glyph.font_face_ascent == glyph.font_ascent && glyph.font_face_height == glyph.font_height) + }) + .map_or((row.rect().top(), row.rect().bottom()), |glyph| { + let top = row.pos.y + glyph.pos.y - glyph.font_ascent; + (top, top + glyph.font_height) + }) + }; + let first = galley.rows.first().map(|row| row_bounds(row).0); + let last = galley.rows.last().map(|row| row_bounds(row).1); + first.zip(last).unwrap_or((0.0, galley.size().y)) +} + pub(crate) fn cell_text_y(valign: VAlign, rect: Rect, gh: f32, z: f32) -> f32 { - let top_inset = 1.0 * z; - let y_top = rect.top() - top_inset; - let y_bottom = (rect.bottom() - gh - 0.5 * z).max(y_top); - match valign { - VAlign::Top => y_top, - VAlign::Center | VAlign::Justify | VAlign::Distributed => (y_top + y_bottom) / 2.0, - VAlign::Bottom => y_bottom, - } + let y_top = wrapped_cell_text_y(VAlign::Top, rect, gh, z); + wrapped_cell_text_y(valign, rect, gh, z).max(y_top) } /// Where to put a `TextShape` holding a galley of `size` turned by `angle` (egui radians, so diff --git a/crates/ui-egui/src/grid/quadrant.rs b/crates/ui-egui/src/grid/quadrant.rs index c278c4bab..dedead69e 100644 --- a/crates/ui-egui/src/grid/quadrant.rs +++ b/crates/ui-egui/src/grid/quadrant.rs @@ -1,7 +1,7 @@ use super::active_row::paint_active_row; use super::cell_paint::{ - Look, MergedArea, cell_text_y, center_across_end, col32, display_text, look, merged_edges, paint_border, paint_cf_icon, painted_halign, - rotated_text_pos, shrink_font, wrapped_cell_text_y, + Look, MergedArea, cell_text_y, center_across_end, col32, display_text, galley_baseline_bounds, look, merged_edges, paint_border, paint_cf_icon, + painted_halign, rotated_text_pos, shrink_font, wrapped_cell_text_y, }; use super::geo::Geo; use super::tables::{filter_button_rect, paint_filter_button}; @@ -394,7 +394,8 @@ fn paint_cell(q: &QuadrantPaint<'_>, painted: &(CellRef, Rect, Look, Option rect.right() - pad - galley.rect.right(), _ => text_left + pad + indent - galley.rect.left(), }; - let y = wrapped_cell_text_y(st.align.v, *rect, galley.size().y, z); + let (text_top, text_bottom) = galley_baseline_bounds(&galley); + let y = wrapped_cell_text_y(st.align.v, *rect, text_bottom - text_top, z) - text_top; let text_clip = wrapped_text_clip(p, *rect, &galley, y, st.align.v); p.with_clip_rect(text_clip).galley(pos2(x, y), galley, color); return; @@ -498,8 +499,9 @@ fn paint_cell(q: &QuadrantPaint<'_>, painted: &(CellRef, Rect, Look, Option rect.right() - pad - w - indent, _ => text_left + pad + indent, }; - let gh = galley.size().y; - let y = cell_text_y(st.align.v, *rect, gh, z); + let (text_top, text_bottom) = galley_baseline_bounds(&galley); + let text_height = text_bottom - text_top; + let y = cell_text_y(st.align.v, *rect, text_height, z) - text_top; let cp = p.with_clip_rect(clip.intersect(p.clip_rect())); if rotated { let angle = -(st.align.rotation as f32).to_radians(); @@ -518,7 +520,10 @@ fn paint_cell(q: &QuadrantPaint<'_>, painted: &(CellRef, Rect, Look, Option= rect.top()); + assert_eq!(y_top_ovf, y_top); } #[test] @@ -277,9 +330,9 @@ fn wrapped_cell_text_y_keeps_vertical_alignment_when_text_is_taller_than_the_row let y_center = wrapped_cell_text_y(VAlign::Center, rect, gh, z); let y_bottom = wrapped_cell_text_y(VAlign::Bottom, rect, gh, z); // Taller text overflows upward for Bottom and both ways for Center (the cell clips it). - assert!((y_top - 1.5).abs() < 0.01, "Top starts at the top edge, got {y_top}"); - assert!((y_center + gh / 2.0 - rect.center().y).abs() < 0.01, "Center is centred, got {y_center}"); - assert!((y_bottom + gh - (rect.bottom() - 1.5)).abs() < 0.01, "Bottom ends at the bottom edge, got {y_bottom}"); + assert!((y_top - 1.5 * z).abs() < 0.01, "Top leaves space inside the cell, got {y_top}"); + assert!((y_center + gh / 2.0 - (rect.center().y + 0.5 * z)).abs() < 0.01, "Center follows the shared insets, got {y_center}"); + assert!((y_bottom + gh - (rect.bottom() - 0.5 * z)).abs() < 0.01, "Bottom keeps the single-line inset, got {y_bottom}"); } /// The screen box of a galley of `size` drawn at `pos` turned by `angle` about `pos`, as egui diff --git a/crates/ui-egui/src/ribbon/title_bar.rs b/crates/ui-egui/src/ribbon/title_bar.rs index 9f7c2197b..29bd4f270 100644 --- a/crates/ui-egui/src/ribbon/title_bar.rs +++ b/crates/ui-egui/src/ribbon/title_bar.rs @@ -227,13 +227,18 @@ mod title_bar_tests { assert_eq!(title_max_width(500.0, 520.0, 900.0), 0.0); } - /// Lay out a title in a real egui frame, returning (title text, title width, dot width). + /// Lay out a title in a real egui frame, checking baselines and returning (text, width, dot width). fn layout_title(title: &str, dirty: bool, max_w: f32) -> (String, f32, f32) { let ctx = egui::Context::default(); ctx.set_fonts(theme::font_definitions()); let mut out = (String::new(), 0.0, 0.0); let mut frame = ctx.run_ui(egui::RawInput::default(), |ui| { let (galley, dot) = title_galleys(ui.painter(), title, dirty, max_w, Color32::WHITE); + for row in &galley.rows { + if let Some(first) = row.glyphs.first() { + assert!(row.glyphs.iter().all(|glyph| (glyph.pos.y - first.pos.y).abs() < 0.01), "title glyphs share a baseline: {title:?}"); + } + } // The glyphs actually drawn: `galley.text()` is the untruncated source. let shown = galley.rows.iter().flat_map(|r| r.glyphs.iter().map(|g| g.chr)).collect(); out = (shown, galley.size().x, dot.map_or(0.0, |d| d.size().x)); @@ -244,7 +249,7 @@ mod title_bar_tests { #[test] fn long_title_is_truncated_with_an_ellipsis() { - let long = format!("upload-{}-cleaned (1).xlsx", "3FMyNTYg2gDaANiAAACq20AAAAQMBqug".repeat(4)); + let long = format!("资产清单2026第4季度-{}-中文 (1).xlsx", "3FMyNTYg2gDaANiAAACq20AAAAQMBqug".repeat(4)); let (text, w, _) = layout_title(&long, false, 300.0); assert!(text.ends_with('…'), "truncated title ends with an ellipsis: {text:?}"); assert!(text.chars().count() < long.chars().count()); diff --git a/crates/ui-egui/src/theme/font_fallback.rs b/crates/ui-egui/src/theme/font_fallback.rs index 65479d017..7d99a7e88 100644 --- a/crates/ui-egui/src/theme/font_fallback.rs +++ b/crates/ui-egui/src/theme/font_fallback.rs @@ -35,7 +35,9 @@ pub(super) fn load_faces(fonts: &mut FontDefinitions, faces: Vec<(&'static str, if loaded.contains(path) { break; // a face loaded for an earlier script already covers this one } - if let Some(fd) = try_load(&[(path.as_str(), *index)]) { + if let Some(mut fd) = try_load(&[(path.as_str(), *index)]) { + // CJK text shares the Latin baseline; symbols and emoji keep box centering. + fd.tweak.set_baseline_fallback(key.starts_with("sys-cjk-")); fonts.font_data.insert(key.to_string(), Arc::new(fd)); keys.push(key.to_string()); loaded.push(path.clone()); diff --git a/crates/ui-egui/src/theme/font_setup.rs b/crates/ui-egui/src/theme/font_setup.rs index da995c489..31e4c63eb 100644 --- a/crates/ui-egui/src/theme/font_setup.rs +++ b/crates/ui-egui/src/theme/font_setup.rs @@ -15,6 +15,19 @@ pub struct CraftFont { pub bytes: &'static [u8], } +impl CraftFont { + /// CJK fallback faces share the primary baseline; symbols keep their centered metrics. + fn font_data(&self) -> FontData { + let mut data = FontData::from_static(self.bytes); + data.tweak.set_baseline_fallback( + self.scripts + .iter() + .any(|script| matches!(*script, "Hani" | "Hans" | "Hant" | "Jpan" | "Hira" | "Kana" | "Hrkt" | "Hang" | "Kore" | "Bopo")), + ); + data + } +} + mod craft_fonts { use super::CraftFont; include!(concat!(env!("OUT_DIR"), "/craft_fonts.rs")); @@ -199,7 +212,7 @@ pub(super) fn craft_font_faces() -> Vec<(String, &'static CraftFont)> { fn add_craft_fonts(fonts: &mut FontDefinitions) -> Vec { let mut keys = Vec::new(); for (key, font) in craft_font_faces() { - fonts.font_data.insert(key.clone(), Arc::new(FontData::from_static(font.bytes))); + fonts.font_data.insert(key.clone(), Arc::new(font.font_data())); keys.push(key); } keys @@ -621,6 +634,22 @@ fn append_craft_faces(fonts: &mut FontDefinitions) { mod font_binding_tests { use super::*; + /// Synthetic bytes make this metadata check independent of the host's installed fonts. + #[test] + fn craft_cjk_scripts_opt_into_the_shared_baseline() { + use egui::epaint::text::FallbackAlign; + const CJK: &[&[&str]] = &[&["Hani", "Hans"], &["Hant", "Bopo"], &["Jpan", "Latn"], &["Hira", "Kana"], &["Hrkt", "Latn"], &["Hang", "Kore"]]; + for scripts in CJK { + let font = CraftFont { family: "Synthetic", style: "Regular", scripts, bytes: epaint_default_fonts::UBUNTU_LIGHT }; + assert_eq!(font.font_data().tweak.fallback_align, FallbackAlign::Baseline); + } + const OTHER: &[&[&str]] = &[&["Zsym", "Zsye"], &["Latn", "Arab"]]; + for scripts in OTHER { + let font = CraftFont { family: "Synthetic", style: "Regular", scripts, bytes: epaint_default_fonts::UBUNTU_LIGHT }; + assert_eq!(font.font_data().tweak.fallback_align, FallbackAlign::Center, "symbols and non-CJK keep their alignment"); + } + } + /// The families [`font_definitions`] always binds, whatever is installed. fn always_bound() -> Vec { [UI, UI_BOLD, CELL, CELL_BOLD, CELL_ITALIC, CELL_BOLD_ITALIC, SERIF, MONO].iter().map(|s| s.to_string()).collect() diff --git a/crates/ui-egui/tests/cjk_baseline.rs b/crates/ui-egui/tests/cjk_baseline.rs new file mode 100644 index 000000000..21b62e557 --- /dev/null +++ b/crates/ui-egui/tests/cjk_baseline.rs @@ -0,0 +1,206 @@ +//! Mixed-script baselines and clipping, without starting the installed-font scanner. + +use egui::{Color32, Context, FontFamily, FontId}; +use gridcraft_engine::{Session, core::CellRef}; +use gridcraft_ui_egui::{SheetApp, i18n::Language, theme}; +use serde_json::json; + +fn activate(ctx: &Context, definitions: egui::FontDefinitions) { + ctx.set_fonts(definitions); + let mut out = ctx.run_ui(Default::default(), |_| {}); + out.textures_delta.clear(); +} + +/// Allow deferred font binding and editor layout to settle before inspecting painted shapes. +fn frames(ctx: &Context, app: &mut SheetApp) -> egui::FullOutput { + let mut out = egui::FullOutput::default(); + for _ in 0..4 { + out = ctx.run_ui( + egui::RawInput { screen_rect: Some(egui::Rect::from_min_size(egui::Pos2::ZERO, egui::vec2(1800.0, 900.0))), ..Default::default() }, + |ui| { + app.logic(ui.ctx()); + app.ui(ui); + }, + ); + out.textures_delta.clear(); + } + out +} + +/// Check layout baselines rather than bitmap bounds, which differ between scripts. +fn assert_shared_baselines(galley: &egui::Galley) { + for row in &galley.rows { + if let Some(first) = row.glyphs.first() { + for glyph in &row.glyphs { + assert!((glyph.pos.y - first.pos.y).abs() < 0.01, "{:?} is off the shared baseline in {:?}", glyph.chr, galley.text()); + } + } + } +} + +fn primary_text_bounds(galley: &egui::Galley) -> (f32, f32) { + let first = galley.rows.iter().find_map(|row| row.glyphs.first().map(|glyph| row.pos.y + glyph.pos.y - glyph.font_ascent)); + let last = + galley.rows.iter().rev().find_map(|row| row.glyphs.first().map(|glyph| row.pos.y + glyph.pos.y - glyph.font_ascent + glyph.font_height)); + first.zip(last).unwrap_or((0.0, galley.size().y)) +} + +/// Exercise fallback alignment across cell styles, title fonts, zoom, and automatic wrapping. +#[test] +fn cjk_and_latin_share_a_baseline_in_cells_and_window_titles() { + let ctx = Context::default(); + ctx.set_pixels_per_point(2.0); + activate(&ctx, theme::font_definitions_for_language(Language::Zh)); + for role in [theme::CELL, theme::CELL_BOLD, theme::CELL_ITALIC, theme::CELL_BOLD_ITALIC, theme::UI_BOLD] { + for zoom in [0.8, 1.0, 2.8] { + let font = FontId::new(if role == theme::UI_BOLD { 13.0 } else { 16.0 * zoom }, FontFamily::Name(role.into())); + if !ctx.fonts_mut(|f| "中文".chars().all(|c| f.has_glyph(&font, c))) { + eprintln!("skipped {role} baseline: no CJK face"); + break; + } + for width in [f32::INFINITY, 100.0 * zoom] { + let galley = ctx.fonts_mut(|f| f.layout("中文ABC2026中文ABC2026".into(), font.clone(), Color32::BLACK, width)); + assert_shared_baselines(&galley); + } + } + } +} + +/// Baseline alignment must not override workbook row heights, clipping, or editor sizing. +#[test] +fn mixed_script_wrapping_keeps_fixed_row_clips_alignment_and_editor_height() { + const TEXT: &str = "中文ABC2026\n文字xyz123"; + for zoom in [80, 100, 280] { + let ctx = Context::default(); + ctx.set_pixels_per_point(2.0); + let mut session = Session::new(); + session.new_workbook(); + session.execute("range.setValues", json!({"range":"A1:C1", "values":[[TEXT, TEXT, TEXT]]})).unwrap(); + session.execute("home.columnWidth", json!({"cols":"A:C", "width":180})).unwrap(); + session.execute("home.rowHeight", json!({"rows":"1:1", "height":32})).unwrap(); + for (cell, align) in [("A1", "Top"), ("B1", "Center"), ("C1", "Bottom")] { + session.execute("home.formatCells", json!({"range":cell, "style":{"align":{"wrap":true, "v":align}}})).unwrap(); + } + session.execute("view.zoom", json!({"percent":zoom})).unwrap(); + let mut app = SheetApp::new(session, Default::default()); + let out = frames(&ctx, &mut app); + let font = { + let doc = app.session.active().unwrap(); + doc.wb.styles.get(doc.wb.active().unwrap().style_id(CellRef::new(0, 0))).font.clone() + }; + let id = FontId::new(font.size * 96.0 / 72.0 * zoom as f32 / 100.0, theme::cell_family(&font.name, font.bold, font.italic)); + if !ctx.fonts_mut(|f| "中文字".chars().all(|c| f.has_glyph(&id, c))) { + eprintln!("skipped CJK grid/editor layout: no CJK face"); + return; + } + let sheet = app.session.active().unwrap().wb.active().unwrap(); + let geo = gridcraft_ui_egui::grid::Geo::new(sheet, app.grid.rect.unwrap(), app.view().scroll); + assert_eq!(sheet.row_height(0), 32.0, "rendering must preserve the manual row height"); + for col in 0..3 { + let rect = geo.cell_rect(sheet, CellRef::new(0, col)); + let (shape, clip) = out + .shapes + .iter() + .find_map(|s| match &s.shape { + egui::Shape::Text(t) if t.galley.text() == TEXT && (t.pos.x - rect.left()).abs() < 5.0 * geo.z => Some((t, s.clip_rect)), + _ => None, + }) + .expect("cell text is painted"); + assert_shared_baselines(&shape.galley); + assert_eq!(shape.galley.rows.len(), 2); + assert!(clip.top() >= rect.top() && clip.bottom() <= rect.bottom(), "fixed-height cells clip their own text"); + let (text_top, text_bottom) = primary_text_bounds(&shape.galley); + let expected_top = match col { + 0 => rect.top() + 1.5 * geo.z, + 1 => rect.center().y - (text_bottom - text_top) / 2.0 + 0.5 * geo.z, + _ => rect.bottom() - (text_bottom - text_top) - 0.5 * geo.z, + }; + assert!((shape.pos.y + text_top - expected_top).abs() < 1.0, "vertical alignment stays effective at {zoom}%"); + } + app.begin_edit(None, false); + let out = frames(&ctx, &mut app); + let field = ctx.read_response(egui::Id::unique("gridcraft.cell_editor")).expect("editor is visible").rect; + let editor = out + .shapes + .iter() + .find_map(|s| match &s.shape { + egui::Shape::Text(t) if t.galley.text() == TEXT && field.contains(t.pos) => Some(t), + _ => None, + }) + .expect("editor text is painted"); + assert!(field.height() >= editor.galley.size().y, "the editor grows to fit its painted CJK lines at {zoom}%"); + assert_shared_baselines(&editor.galley); + } +} + +#[test] +fn same_row_cells_keep_their_baseline_with_baseline_aligned_fallbacks() { + for wraps in [[false, false], [true, true], [false, true], [true, false]] { + for valign in ["Top", "Bottom", "Center"] { + let mixed = if wraps[1] && valign == "Top" { "Hello\n中文" } else { "Hello 中文" }; + let ctx = Context::default(); + ctx.set_pixels_per_point(2.0); + activate(&ctx, theme::font_definitions_for_language(Language::Zh)); + let mut session = Session::new(); + session.new_workbook(); + session.execute("range.setValues", json!({"range":"A1:B1", "values":[[1, mixed]]})).unwrap(); + session.execute("home.columnWidth", json!({"cols":"A:B", "width":180})).unwrap(); + for (cell, wrap) in ["A1", "B1"].into_iter().zip(wraps) { + session.execute("home.formatCells", json!({"range":cell, "style":{"align":{"wrap":wrap, "v":valign}}})).unwrap(); + } + let font = { + let doc = session.active().unwrap(); + doc.wb.styles.get(doc.wb.active().unwrap().style_id(CellRef::new(0, 0))).font.clone() + }; + let font_id = FontId::new(font.size * 96.0 / 72.0, theme::cell_family(&font.name, font.bold, font.italic)); + if !ctx.fonts_mut(|fonts| fonts.has_glyph(&font_id, '中')) { + eprintln!("skipped same-row fallback baseline: no CJK face"); + return; + } + let mut app = SheetApp::new(session, Default::default()); + let out = frames(&ctx, &mut app); + let sheet = app.session.active().unwrap().wb.active().unwrap(); + let geo = gridcraft_ui_egui::grid::Geo::new(sheet, app.grid.rect.unwrap(), app.view().scroll); + let mut baselines = Vec::new(); + for (col, text) in ["1", mixed].iter().enumerate() { + let rect = geo.cell_rect(sheet, CellRef::new(0, col as u32)); + let shape = out + .shapes + .iter() + .find_map(|s| match &s.shape { + egui::Shape::Text(t) + if t.galley.text() == *text && t.visual_bounding_rect().intersect(s.clip_rect).intersect(rect).is_positive() => + { + Some(t) + } + _ => None, + }) + .expect("cell text is painted"); + assert_shared_baselines(&shape.galley); + if valign == "Top" { + let (text_top, _) = primary_text_bounds(&shape.galley); + assert!(shape.pos.y + text_top > rect.top(), "top-aligned text keeps space inside the cell above its primary line box"); + } + let row = shape.galley.rows.first().expect("cell text has a row"); + let glyph = row.glyphs.first().expect("first-line baseline is present"); + if text.contains('中') { + assert_eq!(shape.galley.rows.len(), if mixed.contains('\n') { 2 } else { 1 }); + assert!( + shape + .galley + .rows + .iter() + .flat_map(|row| &row.glyphs) + .any(|glyph| { glyph.chr == '中' && glyph.font_face_align == egui::epaint::text::FallbackAlign::Baseline }), + "CJK fallback must use baseline alignment" + ); + } + baselines.push(shape.pos.y + row.pos.y + glyph.pos.y); + } + assert!( + (baselines[0] - baselines[1]).abs() < 0.6, + "same-row cells with wraps={wraps:?}, {valign} alignment share a baseline: {baselines:?}" + ); + } + } +} diff --git a/crates/ui-egui/tests/it/text_overflow_scroll.rs b/crates/ui-egui/tests/it/text_overflow_scroll.rs index 371c0aeee..4790674c6 100644 --- a/crates/ui-egui/tests/it/text_overflow_scroll.rs +++ b/crates/ui-egui/tests/it/text_overflow_scroll.rs @@ -211,8 +211,8 @@ fn wrapped_text_in_fixed_height_row_shows_only_complete_lines() { let mut full_rows_visible = 0; for (index, row) in text.galley.rows.iter().enumerate() { - let top = text.pos.y + row.rect().top(); - let bottom = text.pos.y + row.rect().bottom(); + let top = text.pos.y + row.pos.y + row.visuals.mesh_bounds.top(); + let bottom = text.pos.y + row.pos.y + row.visuals.mesh_bounds.bottom(); let full_height = bottom - top; let visible_height = (bottom.min(clip.bottom()) - top.max(clip.top())).max(0.0); diff --git a/docs/architecture.md b/docs/architecture.md index 68639541e..febe96fcd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,6 @@ # Architecture -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-02 · **Change:** minor (Python mode in the formula bar; earlier: Python cells as data; earlier: equations: the `math` crate and the layout cache; earlier: recently used functions kept in the preferences; Function Library module; accent icons; earlier: the hidden names that hold macros) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-03 · **Change:** minor (shared CJK fallback baselines; earlier: Python mode in the formula bar; earlier: Python cells as data; earlier: equations: the `math` crate and the layout cache; earlier: recently used functions kept in the preferences; Function Library module; accent icons; earlier: the hidden names that hold macros) · **Target:** Microsoft Excel (Microsoft 365) How GridCraft is built today. Rules for contributors are in [`AGENTS.md`](../AGENTS.md). @@ -84,6 +84,12 @@ drawing its own: ahead of its input, so the field never gets a Tab or an Enter of a focused editor; and it scrolls past `max_lines` with the caret in view. The tokenizer and the key edits are pure functions with tests, including hostile input. +- **CJK text baselines**: system CJK fallback faces and permitted embedded faces tagged with + CJK scripts use epaint's baseline fallback alignment in cells, editors and window titles. + Other scripts, symbols and emoji keep their existing alignment. Baseline alignment can + increase line height; manual row heights and clipping remain unchanged. Reports: + [#1265](https://github.com/storytold/gridcraft/issues/1265) and + [#2621](https://github.com/storytold/gridcraft/issues/2621). - **Code editor marks**: `code_editor_marked` underlines a byte range in the error colour (the equation editor marks the position of a LaTeX error with it). - **Formula bar**: grows with its content from one line to `formula_bar::MAX_LINES` (five), then @@ -242,6 +248,7 @@ rpm, tarball), FreeBSD, and web (WASM; WebGPU or WebGL2). Release workflows in | 2026-11-01 | minor | Equations: the `math` crate (L1), the layout cache and code editor marks | | 2026-10-31 | minor | Recently used functions (`recent.rs`, `Prefs::recent_functions`) behind `formulas.recentlyUsed` | | 2026-10-31 | minor | Function Library module (`function_library.rs`); accent icons (`icons::paint_accent`, `widgets::small_button_accent`) | +| 2026-10-31 | minor | System and embedded CJK fallback faces share the Latin baseline without changing other script, symbol or emoji alignment | | 2026-10-25 | trivial | The hidden names that hold macros are `_gc_macro__` | | 2026-10-24 | minor | Macro engine on the building blocks (`macros/`, `cmd/macros.rs`) | | 2026-10-24 | minor | Engine building blocks for multi-command actions: undo groups, recording buffer, document trust, chunked hidden-name storage | diff --git a/docs/localization-parity.md b/docs/localization-parity.md index b59b5dcef..9f5188948 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-27 · **Change:** minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-31 · **Change:** minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Per-language status against Excel, which ships its UI in dozens of languages (Windows language packs cover 100+), with localized function names, argument separators, number and date formats, @@ -61,7 +61,10 @@ Cell text now uses shared Unicode bidi layout while retaining logical source ind #142 is still open pending confirmation). Native CJK fallback also probes Flatpak's read-only `/run/host/fonts`, `/run/host/local-fonts` and `/run/host/user-fonts` layouts, so installed Noto CJK and WenQuanYi faces can render Chinese without a broad filesystem grant. This does - not change the installed-font catalogue or add bundled fonts. Lao falls back to Lao Sangam MN / Lao MN (macOS), + not change the installed-font catalogue or add bundled fonts. System CJK fallback faces and + embedded faces tagged with CJK scripts share the Latin baseline in cells, editors and titles + (#1265, #2621); other script, emoji and symbol fallbacks retain their alignment. + Lao falls back to Lao Sangam MN / Lao MN (macOS), Leelawadee UI / DokChampa (Windows) or Noto Sans Lao / Phetsarath OT (Linux). egui uses harfrust shaping and Unicode bidi layout; script-specific coverage beyond Arabic still needs verification. **Sheet Right-to-Left** Leelawadee UI / DokChampa (Windows) or Noto Sans Lao / Phetsarath OT (Linux). Arabic falls back to a system Arabic face, fontconfig's @@ -130,6 +133,7 @@ language (none so far) and of localized function names. | Date | Change | Summary | |---|---|---| +| 2026-10-31 | minor | System and embedded CJK text fallbacks share the Latin baseline in cells, editors and window titles; other scripts, symbols and emoji keep their alignment | | 2026-10-26 | minor | Shared bidi layout supports Arabic clusters, marks, visual cursor navigation and directional accessibility; cell Context/LTR/RTL is stored separately from sheet RTL and round-trips through XLSX | | 2026-10-26 | minor | Arabic text renders in cells and the interface: a system Arabic face (Tahoma, Arial or Segoe UI on Windows, Geeza Pro on macOS, Noto Sans or Naskh Arabic on Linux) joins the script fallbacks, fontconfig's Arabic choice, when it has Arabic letters, leads the Arabic fallbacks and draws Arabic wherever the UI or cell font has none, such as the default cell fonts (part of #229; a UI font with Arabic of its own, such as DejaVu Sans in the Flatpak, still draws interface Arabic, left for a follow-up), and craft-fonts' Noto Sans Arabic is a required face allowed on the web. Workbook fonts without Arabic keep every script fallback. Shaping and bidirectional layout remain separate work | | 2026-10-27 | minor | Number formats honour the `B1`/`B2` calendar prefixes: never shown; `B2` renders year, month, day and Arabic month and weekday names in the tabular Hijri calendar (1 Jan 2001 = 6 Shawwal 1421; leap years 2, 5, 7, 10, 13, 15, 18, 21, 24, 26 and 29 of the 30-year cycle, which is what Excel shows). Checked against Excel 16.0 (build 20527, en-US) on every serial and about 1,700 format/value pairs: a locale or currency tag before `B2` keeps the date Gregorian, a misplaced `B` and digit or a prefix before a number format is rejected, serial 60 has no Hijri date, serial 0 shows as serial 1, two-word month names use a no-break space. Not matched: Excel's weekday/month-name mix-ups in one `B2` section and its weekday-only quirks | diff --git a/docs/ui-parity.md b/docs/ui-parity.md index b77acc0b4..f5996328e 100644 --- a/docs/ui-parity.md +++ b/docs/ui-parity.md @@ -1,6 +1,6 @@ # UI parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-03 · **Change:** minor (form controls: buttons, sliders and spin buttons on the sheet; earlier: Highlight the active cell's row, a global preference in File › Options › General; earlier: a linked cell's menu edits, opens or removes the link; earlier: Display options for this workbook: sheet tabs and scroll bars; earlier: dragging over row or column headers in point mode extends the reference; earlier: a picture on the system clipboard pastes as a floating picture at the active cell with Cmd/Ctrl+V or Paste (#699); earlier: frozen columns keep their text, gridlines and merged borders whole while the sheet scrolls (#2544); earlier: every Paste Special option has its access key; earlier: the Draw tab's live stroke preview uses the pen's colour and thickness; earlier: copy quotes only cells with a tab or line break, as Excel does; earlier: Ctrl+. Next Corner of Selection; earlier: dragging the selection's border onto data asks before replacing it; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis use them, the arrow keys select while a dialog is collapsed and F4 cycles a reference's anchors; earlier: Python cells: Python mode in the formula bar; earlier: Insert › Equation and the equation editor; earlier: every floating object selects, resizes from eight handles and deletes with the keyboard; earlier: Formulas › Error Checking › Trace Error; trace arrows stay through a zoom (#3097); earlier: Page Setup sets the page order, Down, then over or Over, then down, which printing, the print preview, Page Break Preview and Page Layout view follow; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and the Insert Function categories; Esc leaves a File-page text field before closing the view; earlier: checkbox cells print as boxes; Ctrl+arrow and Ctrl+Shift+arrow while pointing a formula reference; editing a hyperlink loads its existing fields; AutoSum over a selected row puts its total to the right; ribbon catalog count 276 / 292) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-05 · **Change:** minor (preserve centered emoji and symbol fallback placement; earlier: preserve leading and trailing blank cell lines; earlier: align wrapped and unwrapped cells; earlier: CJK title baseline regression and mixed-script rendering validation; earlier: form controls: buttons, sliders and spin buttons on the sheet; earlier: Highlight the active cell's row, a global preference in File › Options › General; earlier: a linked cell's menu edits, opens or removes the link; earlier: Display options for this workbook: sheet tabs and scroll bars; earlier: dragging over row or column headers in point mode extends the reference; earlier: a picture on the system clipboard pastes as a floating picture at the active cell with Cmd/Ctrl+V or Paste (#699); earlier: frozen columns keep their text, gridlines and merged borders whole while the sheet scrolls (#2544); earlier: every Paste Special option has its access key; earlier: the Draw tab's live stroke preview uses the pen's colour and thickness; earlier: copy quotes only cells with a tab or line break, as Excel does; earlier: Ctrl+. Next Corner of Selection; earlier: dragging the selection's border onto data asks before replacing it; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis use them, the arrow keys select while a dialog is collapsed and F4 cycles a reference's anchors; earlier: Python cells: Python mode in the formula bar; earlier: Insert › Equation and the equation editor; earlier: every floating object selects, resizes from eight handles and deletes with the keyboard; earlier: Formulas › Error Checking › Trace Error; trace arrows stay through a zoom (#3097); earlier: Page Setup sets the page order, Down, then over or Over, then down, which printing, the print preview, Page Break Preview and Page Layout view follow; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and the Insert Function categories; Esc leaves a File-page text field before closing the view; earlier: checkbox cells print as boxes; Ctrl+arrow and Ctrl+Shift+arrow while pointing a formula reference; editing a hyperlink loads its existing fields; AutoSum over a selected row puts its total to the right; ribbon catalog count 276 / 292) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Interaction, controls and feel, against Excel. Part of [target-app-parity.md](target-app-parity.md). **UI/UX fidelity: ~55% ready, 50–80 h** (estimated). Ribbon command coverage is measured @@ -235,6 +235,24 @@ pairs (text 4.5:1, glyphs 3:1) are checked by `theme::contrast_tests`. ## Feel +Mixed CJK, Latin and digits share a baseline in the grid, editor and window title when CJK +fallback fonts are used. Baseline alignment can increase line height; manual row heights +continue to clip oversized text. + +Wrapped and unwrapped cells use the same vertical margins. With the same font and top +alignment, a number aligns with the first line of neighbouring wrapped text; fallback font +height does not shift the primary font's baseline between cells. Leading and trailing blank +lines retain their space when positioning wrapped text. Emoji and symbol fallbacks keep their +centering when they start a line or occupy it alone. + +Offscreen macOS renders at 80%, 100% and 280% cover mixed text, regular/bold/italic/bold-italic +roles, automatic wrapping, explicit line breaks and a `yyyy年m月d日` date. The before/after +comparison uses the same synthetic cells and the default 10 pt cell role: Arial with +Hiragino Sans GB fallback on the tested Mac. +Automated regressions check baseline positions, including the actual truncated title layout. +Glyph scaling is unchanged; the original Calibri 10 visual-size case still needs confirmation +on a host with that font installed. + Pixel comparison against Excel screenshots hasn't been run since M1; fonts depend on the system (no Calibri; Carlito would match metrics, #29, #4, #168 for weights); icons are GridCraft's own by rule and will never match Office's, by design. @@ -249,6 +267,10 @@ scrollbar bugs 2–4 · pixel/feel pass against observed Excel 7–12. | Date | Change | Summary | |---|---|---| +| 2026-11-05 | minor | Reconcile upstream form-control coverage with the CJK baseline and cell-alignment updates | +| 2026-11-04 | minor | Keep centered emoji and symbol fallback placement when a row starts with a fallback or contains only fallbacks; use primary or baseline-aligned glyphs to anchor cell alignment | +| 2026-11-04 | minor | Preserve leading and trailing blank lines in wrapped cells when calculating vertical alignment; regression covers single, multiple and all-empty lines | +| 2026-11-04 | minor | Align wrapped and unwrapped cells with the same vertical margins; regressions compare numbers with mixed CJK/Latin text, including explicit line breaks | | 2026-11-02 | minor | Form controls: buttons, sliders (scroll bars) and spin buttons placed, worked, selected and edited on the sheet (Insert › Controls, Assign Action, Format Control); catalog 281 / 296 | | 2026-11-03 | minor | Highlight the active cell's row: a global preference that tints the whole row, off by default | | 2026-11-03 | minor | The formula bar's bottom edge is a resize handle: dragging it sets the bar's height in whole lines, up to half the window, and dragging it to one line collapses it; expanding (the chevron, Ctrl+Shift+U) returns to the dragged height, which is kept with the UI preferences. `view.expandFormulaBar {lines}` sets it (#1246) | @@ -267,13 +289,16 @@ scrollbar bugs 2–4 · pixel/feel pass against observed Excel 7–12. | 2026-11-02 | minor | Python mode edits a cell's code in the formula bar's code editor (Ctrl+Alt+Shift+P, `=PY` then Tab, editing a Python cell), with the PY badge beside fx; its keys apply in input order, also several in one frame, and Ctrl+Enter commits through `python.set` (empty code clears the cell), keeping the code and showing the command's message when the code is refused | | 2026-11-02 | minor | Insert › Symbols: Equation split button (access keys N E), a gallery of examples, and an equation editor with quick-insert buttons, the error underlined at its position and a live preview; double-clicking an equation edits it | | 2026-11-02 | minor | Floating objects: one selection for charts, pictures, shapes and slicers, eight resize handles with matching cursors (right-to-left sheets included), Escape drops the selection | +| 2026-11-02 | minor | Record macOS mixed CJK/Latin rendering validation, the actual fallback fonts and the unverified Calibri 10 case | | 2026-11-01 | minor | Chart Design › Add Chart Element offers Data Table with/no legend keys and None; Format Chart Area has a Data Table section with visibility, legend keys, horizontal/vertical borders and outline. Both use engine commands. | | 2026-11-01 | minor | Page Setup has a Page order choice (Down, then over, the default, or Over, then down) with a small picture of the order, also when Print Titles opens it on its Sheet tab; OK sets it with the margins in one undo step. Printing, PDF export, the print preview, Page Break Preview's page numbers and Page Layout view's headers and footers follow the order. The dialog is in German now too | | 2026-11-01 | minor | Formulas › Error Checking › Trace Error (`formulas.traceError`): red arrows from the cell an error starts in to the selected cell, through the first reference showing the same error at each step, and blue arrows from that cell's own references, as measured in Excel; the command returns the source and the arrows | | 2026-11-01 | minor | Trace arrows stay when the sheet is zoomed (View › Zoom, the status bar, Ctrl+wheel), as in Excel; every zoom used to remove them (#3097) | +| 2026-11-01 | minor | Assert shared baselines in the actual title layout and record manual visual-size validation of a compiled macOS build | | 2026-10-31 | minor | Esc in a File-page text field leaves the field and cancels editing; the next Esc returns to the sheet (#3035) | | 2026-10-31 | minor | Recently Used lists the functions most recently entered in a formula from any source, kept with the preferences; the Insert Function dialog is titled as in Excel, its category list (Most Recently Used, All, each category) works, and its category column follows the widest name shown | | 2026-10-31 | minor | Formulas › Function Library: Insert Function, the AutoSum split button and three columns of labelled small buttons with category colours; category menus end with Insert Function…; More Functions cascades; Home › Editing shares the AutoSum menu; MR key tip for Recently Used | +| 2026-10-31 | minor | CJK fallback text shares the Latin baseline in cells, editors and window titles; fixed row heights and clipping remain workbook-controlled | | 2026-10-30 | minor | Checkbox cells print and export to PDF as boxes, as on screen: unchecked white with a grey outline, checked green with a white tick (#2972). They printed their value as TRUE or FALSE; other values in a checkbox cell still print as text | | 2026-10-29 | minor | Formula pointing from the keyboard: Ctrl+arrow (⌘arrow) moves the pointed cell to the edge of its data and Ctrl+Shift+arrow (⌘⇧arrow) extends the pointed range there, as the same keys move and extend the selection, so typing `=COUNTIF(`, clicking the top of a column and pressing ⌘⇧↓ points at the whole column of data; the keys used to move or extend by one cell. A range reaching the sheet edge is written as whole columns or rows (`A:B`). The editor reads the modifiers of the arrow's own key event, so a keystroke pressed and released within one frame (a control client's `ui.key`, an automation tool) keeps its Shift and Ctrl | | 2026-10-29 | minor | Insert Hyperlink opens with an existing link's address, display text and ScreenTip (#2876). An address-only edit preserves the cell's content and formula; Cancel leaves the link unchanged, and Undo restores the edit | From 74ab5cc96930cad7d05e4553859b96c746dc5919 Mon Sep 17 00:00:00 2001 From: Diogo Soares Rodrigues Date: Tue, 10 Nov 2026 09:54:53 -0300 Subject: [PATCH 3/6] M12: regional format data (Region, 20 regions) (#3048) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * M12: regional format data (Region, 15 regions, Workbook::region) Adds the data model for Excel-style regional formats, with no behaviour change: nothing reads it yet. - core: region.rs with `Region` (separators, grouping, date order, short and long date and time codes, month and day names, AM/PM words, currency) and the 15 regions en-US, en-GB, en-IN, es-ES, de-DE, fr-FR, it-IT, pt-BR, pt-PT, ru-RU, cs-CZ, ja-JP, ko-KR, zh-CN, zh-TW. Values follow Unicode CLDR; the deliberate departures (four-digit short-date year, Windows grouping spaces) are listed in the module doc. `all`, `find` (strict tag), `lookup` (system or browser tag, best effort) and `default_region` (en-US) select one; `EN_US` reproduces today's behaviour exactly. Unit tests check the tables and the lookups. - model: `Workbook::region`, a `#[serde(skip)]` runtime field that is never saved and defaults to en-US. It is here so that the following changes can be reviewed independently of each other. The region is independent of the interface language and of the formula language. Builds on this, in order: typed input (core `parse_input_in`), display (numfmt and engine display), the engine session and `app.region` command, then the Options UI and the app defaults. * M12: region data: review fixes (tag extensions, zh territory, month periods) - lookup stops at an extension or private-use singleton: en-GB-u-ca-gregory is en-GB (ca was read as Canada) and zh-Hant-u-nu-latn is zh-TW. - For Chinese an explicit territory wins over the script, so zh-Hant-CN is China's conventions, consistent with zh-Hans-TW. - In regions whose month abbreviations end in a period (ru-RU, fr-FR), month_from_name accepts any abbreviation typed with one (мар., июн.); elsewhere a period stays outside the name, so en-US is unchanged. * Region: document where dates and month names follow Windows, not CLDR * M12: Swiss regions (en-CH, de-CH, fr-CH, it-CH) and nl-NL The Swiss regions group digits with an apostrophe (CLDR 48) and accept the typographic one glibc uses; lookup maps CH and LI tags to them. Dutch (Netherlands) follows CLDR 48. * Region: table-driven lookup test, under main's cognitive-complexity limit * Region: French and Italian Liechtenstein tags resolve to the Swiss regions * Region: credit the Unicode CLDR data under the Unicode License v3 (NOTICE, ATTRIBUTION.md) * Region: month_from_name takes one word; the currency format has the currency's own decimals --------- Co-authored-by: quantmind-br Co-authored-by: quantforge-br <218010578+quantforge-br@users.noreply.github.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> --- ATTRIBUTION.md | 1 + NOTICE | 49 ++ crates/core/src/lib.rs | 4 +- crates/core/src/region.rs | 883 +++++++++++++++++++++++++++++++++++ crates/model/src/workbook.rs | 6 + 5 files changed, 942 insertions(+), 1 deletion(-) create mode 100644 crates/core/src/region.rs diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index 2a4b5c96e..7dd388ccd 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -12,6 +12,7 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra | `epaint_default_fonts::HACK_REGULAR` (dependency font input) | Source Foundry Authors (Hack; DejaVu and Bitstream Vera foundations) | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Hack](https://github.com/source-foundry/Hack) | MIT + Bitstream Vera License (full upstream texts in `NOTICE`) | Hack-Regular.ttf is egui's bundled monospace face and GridCraft's last-resort symbol fallback. Dependency asset unmodified here, no font file copied into this repository. | | `epaint_default_fonts::NOTO_EMOJI_REGULAR` (dependency font input) | The Noto Project Authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Noto Emoji](https://fonts.google.com/noto/specimen/Noto+Emoji) | SIL Open Font License 1.1 (full upstream text in `NOTICE`) | NotoEmoji-Regular.ttf is enabled by egui's `monochrome_emoji_fonts` feature and used as the monochrome emoji fallback. Dependency asset unmodified here, no font file copied into this repository. | | `epaint_default_fonts::UBUNTU_LIGHT` (dependency font input) | Canonical Ltd. and the Ubuntu Font Family authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Ubuntu font](https://design.ubuntu.com/font) | Ubuntu Font Licence 1.0 (full upstream text in `NOTICE`) | Ubuntu-Light.ttf is egui's default proportional face. Dependency asset unmodified here, no font file copied into this repository. | +| Regional format data in `crates/core/src/region.rs` (data, no file of its own) | Unicode, Inc. (Unicode CLDR) | [cldr-json](https://github.com/unicode-org/cldr-json) (`cldr-numbers-full`, `cldr-dates-full`) and ICU's CLDR 48 data | Unicode License v3 (`Unicode-3.0`; full text in `NOTICE`) | Separators, month and weekday names, AM/PM words, currency symbols and date order of the built-in regions. No CLDR file is copied into this repository. | | `docs/brand/artcraft-logo.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms (`docs/brand/LICENSE-brand.txt`) | Trademark, not open source | | `docs/brand/artcraft-logo.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | | `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | diff --git a/NOTICE b/NOTICE index d859d99d3..64753f94a 100644 --- a/NOTICE +++ b/NOTICE @@ -275,6 +275,55 @@ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE. + +The regional format data in crates/core/src/region.rs (separators, month and weekday names, AM/PM +words, currency symbols and date order) follows the Unicode Common Locale Data Repository (CLDR), +read from unicode-org/cldr-json and from ICU's CLDR 48 data, under the Unicode License v3. +Source: https://github.com/unicode-org/cldr-json +The following text is reproduced verbatim from cldr-json's LICENSE: + +UNICODE LICENSE V3 + +COPYRIGHT AND PERMISSION NOTICE + +Copyright © 2015-2024 Unicode, Inc. + +NOTICE TO USER: Carefully read the following legal agreement. BY +DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR +SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE +TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT +DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE. + +Permission is hereby granted, free of charge, to any person obtaining a +copy of data files and any associated documentation (the "Data Files") or +software and any associated documentation (the "Software") to deal in the +Data Files or Software without restriction, including without limitation +the rights to use, copy, modify, merge, publish, distribute, and/or sell +copies of the Data Files or Software, and to permit persons to whom the +Data Files or Software are furnished to do so, provided that either (a) +this copyright and permission notice appear with all copies of the Data +Files or Software, or (b) this copyright and permission notice appear in +associated Documentation. + +THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF +THIRD PARTY RIGHTS. + +IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE +BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, +OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, +WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, +ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA +FILES OR SOFTWARE. + +Except as contained in this notice, the name of a copyright holder shall +not be used in advertising or otherwise to promote the sale, use or other +dealings in these Data Files or Software without prior written +authorization of the copyright holder. + +SPDX-License-Identifier: Unicode-3.0 + Linux Wayland window decorations ------------------------------- sctk-adwaita 0.10.1 (https://github.com/PolyMeilex/sctk-adwaita, diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index b3f5dbd67..52b313b25 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -1,5 +1,5 @@ //! GridCraft core types: cell addresses, values and Excel's coercion rules, error values, -//! date serials and typed-input parsing. No dependencies on the rest of the workspace. +//! date serials, regional formats and typed-input parsing. No dependencies on the rest of the workspace. #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] #![forbid(unsafe_code)] @@ -7,6 +7,7 @@ pub mod addr; pub mod color; pub mod date; pub mod parse; +pub mod region; pub mod value; pub use addr::{ @@ -14,4 +15,5 @@ pub use addr::{ split_areas, split_sheet, unquote_sheet, }; pub use date::DateSystem; +pub use region::Region; pub use value::{Array, CellError, Value, compare, compare_numbers, compare_text, number_to_text, sort_compare}; diff --git a/crates/core/src/region.rs b/crates/core/src/region.rs new file mode 100644 index 000000000..30632603c --- /dev/null +++ b/crates/core/src/region.rs @@ -0,0 +1,883 @@ +//! Regional formats: how a region writes numbers, dates, times and money. +//! +//! Excel takes these from the operating system's regional settings; they are independent of the +//! interface language and of the formula language. A [`Region`] changes what people *type* into +//! a cell (`1.234,56`, `25/09/2026`) and what cells *show* (separators, month and day names, +//! AM/PM, the system short and long dates). It never changes what is stored: values are numbers, +//! and the number format codes saved in a file stay in their canonical en-US spelling. +//! +//! [`EN_US`] is the default and reproduces GridCraft's original behaviour exactly. +//! +//! # Where the values come from +//! The separators, month and weekday names, AM/PM words, currency symbols and the order of day, +//! month and year are the Unicode CLDR conventions (`unicode-org/cldr-json`: +//! `cldr-numbers-full/main//numbers.json` and `cldr-dates-full/main//ca-gregorian.json`), +//! which is also what macOS, Android and ICU ship. They were read from those files for en-GB, +//! en-IN, cs and ru, read from ICU's CLDR 48 data for the Swiss regions and nl-NL, and written +//! from the same conventions for the other regions. CLDR data is under the Unicode License v3, +//! reproduced in `NOTICE` (row in `ATTRIBUTION.md`). +//! Deliberate choices where operating systems differ from CLDR: +//! - Short dates use a four-digit year (`dd/mm/yyyy`, where CLDR has `dd/MM/y` or `dd/MM/yy`): +//! Excel's short date is the system's, Windows writes four digits, and a two-digit year would +//! not read back to the same date. +//! - The long date is CLDR's *full* date (with the weekday), like Windows en-US, except where +//! Windows' long date differs: Russian, Japanese and Chinese have no weekday (`23 августа +//! 2005 г.`, `2005年8月23日`), as in Microsoft's National Language Support tables. +//! - Korean writes the short date `yyyy-mm-dd` (CLDR: `yy. M. d.`), also as in Microsoft's +//! National Language Support tables. +//! - Month names for `mmm` are CLDR's abbreviated names (so en-GB has `Sept`), except +//! Portuguese, which drops CLDR's period (`jan`, not `jan.`) as Excel in Brazil shows it. +//! - French and Portuguese (Portugal) group digits with U+00A0 (Windows), not CLDR's U+202F. +//! Parsing accepts every space-like grouping character in those regions. +//! - The Swiss regions (en-CH, de-CH, fr-CH, it-CH; Liechtenstein uses de-CH) group digits with +//! CLDR's ASCII apostrophe (`1'234.50`); parsing also accepts the typographic `’` that glibc +//! uses. French (Switzerland) writes a decimal point like the other Swiss regions and glibc's +//! `fr_CH`, where CLDR has a comma. +//! +//! Regions this module doesn't list (`de-AT`, `es-MX`, `en-CA`…) resolve to the main region of +//! their language through [`lookup`]; their own conventions (a Mexican decimal point) are not +//! modelled. + +/// How digits are grouped. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Grouping { + /// Groups of three (`1,234,567`). + Thousands, + /// The Indian system: three digits, then groups of two (`12,34,567`). + Indian, +} + +/// The order of the parts of a short date. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum DateOrder { + Mdy, + Dmy, + Ymd, +} + +/// One region's conventions. See the module documentation. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Region { + /// BCP 47 tag, `pt-BR`. + pub tag: &'static str, + /// The region's name in its own language. + pub name: &'static str, + /// Windows locale id, used by `[$-409]`-style format tags. + pub lcid: u16, + pub decimal: char, + pub group: char, + pub grouping: Grouping, + pub date_order: DateOrder, + pub date_sep: char, + pub time_sep: char, + /// The system short date as a canonical format code (`dd/mm/yyyy`): built-in format 14. + pub short_date: &'static str, + /// The system long date (`[$-F800]`). + pub long_date: &'static str, + /// The system short time. It contains `AM/PM` in regions that use a 12-hour clock. + pub short_time: &'static str, + /// The system long time (`[$-F400]`). + pub long_time: &'static str, + /// Built-in format 22: the short date and a 24-hour time. + pub datetime: &'static str, + pub months: [&'static str; 12], + pub months_abbr: [&'static str; 12], + /// The month names a date with a day uses (Russian `1 января`, Czech `1. ledna`), where they + /// differ from `months`. + pub months_genitive: Option<[&'static str; 12]>, + /// Sunday first. + pub weekdays: [&'static str; 7], + pub weekdays_abbr: [&'static str; 7], + pub am_pm: [&'static str; 2], + pub currency: &'static str, + /// The currency format with the currency's usual decimals (two, or none for the yen and the + /// won), as a canonical code. Where it starts with a quote the symbol comes before the number, + /// otherwise after. + pub currency_format: &'static str, + /// The same with no decimals (for a typed whole amount). + pub currency_format_int: &'static str, +} + +const EN_MONTHS: [&str; 12] = + ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"]; +const EN_MONTHS_ABBR: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; +const EN_DAYS: [&str; 7] = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; +const EN_DAYS_ABBR: [&str; 7] = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]; +const AM_PM: [&str; 2] = ["AM", "PM"]; + +/// English (United States): the default. Identical to the formats before regions existed. +pub static EN_US: Region = Region { + tag: "en-US", + name: "English (United States)", + lcid: 0x0409, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Mdy, + date_sep: '/', + time_sep: ':', + short_date: "m/d/yyyy", + long_date: "dddd, mmmm dd, yyyy", + short_time: "h:mm AM/PM", + long_time: "h:mm:ss AM/PM", + datetime: "m/d/yyyy h:mm", + months: EN_MONTHS, + months_abbr: EN_MONTHS_ABBR, + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "$", + currency_format: "\"$\"#,##0.00", + currency_format_int: "\"$\"#,##0", +}; + +static EN_GB: Region = Region { + tag: "en-GB", + name: "English (United Kingdom)", + lcid: 0x0809, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: EN_MONTHS, + months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"], + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "£", + currency_format: "\"£\"#,##0.00", + currency_format_int: "\"£\"#,##0", +}; + +static EN_IN: Region = Region { + tag: "en-IN", + name: "English (India)", + lcid: 0x4009, + decimal: '.', + group: ',', + grouping: Grouping::Indian, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "h:mm AM/PM", + long_time: "h:mm:ss AM/PM", + datetime: "dd/mm/yyyy h:mm", + months: EN_MONTHS, + months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"], + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "₹", + currency_format: "\"₹\"#,##0.00", + currency_format_int: "\"₹\"#,##0", +}; + +static EN_CH: Region = Region { + tag: "en-CH", + name: "English (Switzerland)", + // English (Switzerland) has no LCID of its own: Windows reports LOCALE_CUSTOM_UNSPECIFIED. + lcid: 0x1000, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: EN_MONTHS, + months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"], + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "CHF", + currency_format: "\"CHF\" #,##0.00", + currency_format_int: "\"CHF\" #,##0", +}; + +static ES_ES: Region = Region { + tag: "es-ES", + name: "Español (España)", + lcid: 0x0C0A, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d \"de\" mmmm \"de\" yyyy", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "dd/mm/yyyy h:mm", + months: ["enero", "febrero", "marzo", "abril", "mayo", "junio", "julio", "agosto", "septiembre", "octubre", "noviembre", "diciembre"], + months_abbr: ["ene", "feb", "mar", "abr", "may", "jun", "jul", "ago", "sep", "oct", "nov", "dic"], + months_genitive: None, + weekdays: ["domingo", "lunes", "martes", "miércoles", "jueves", "viernes", "sábado"], + weekdays_abbr: ["dom", "lun", "mar", "mié", "jue", "vie", "sáb"], + am_pm: ["a. m.", "p. m."], + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static DE_DE: Region = Region { + tag: "de-DE", + name: "Deutsch (Deutschland)", + lcid: 0x0407, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d. mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"], + months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"], + months_genitive: None, + weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"], + weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static DE_CH: Region = Region { + tag: "de-CH", + name: "Deutsch (Schweiz)", + lcid: 0x0807, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d. mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"], + months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"], + months_genitive: None, + weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"], + weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"], + am_pm: AM_PM, + currency: "CHF", + currency_format: "\"CHF\" #,##0.00", + currency_format_int: "\"CHF\" #,##0", +}; + +static FR_FR: Region = Region { + tag: "fr-FR", + name: "Français (France)", + lcid: 0x040C, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"], + months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."], + months_genitive: None, + weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"], + weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static FR_CH: Region = Region { + tag: "fr-CH", + name: "Français (Suisse)", + lcid: 0x100C, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"], + months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."], + months_genitive: None, + weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"], + weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."], + am_pm: AM_PM, + currency: "CHF", + currency_format: "#,##0.00 \"CHF\"", + currency_format_int: "#,##0 \"CHF\"", +}; + +static IT_IT: Region = Region { + tag: "it-IT", + name: "Italiano (Italia)", + lcid: 0x0410, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"], + months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"], + months_genitive: None, + weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"], + weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static IT_CH: Region = Region { + tag: "it-CH", + name: "Italiano (Svizzera)", + lcid: 0x0810, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"], + months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"], + months_genitive: None, + weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"], + weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"], + am_pm: AM_PM, + currency: "CHF", + currency_format: "\"CHF\" #,##0.00", + currency_format_int: "\"CHF\" #,##0", +}; + +static NL_NL: Region = Region { + tag: "nl-NL", + name: "Nederlands (Nederland)", + lcid: 0x0413, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '-', + time_sep: ':', + short_date: "dd-mm-yyyy", + long_date: "dddd d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd-mm-yyyy hh:mm", + months: ["januari", "februari", "maart", "april", "mei", "juni", "juli", "augustus", "september", "oktober", "november", "december"], + months_abbr: ["jan", "feb", "mrt", "apr", "mei", "jun", "jul", "aug", "sep", "okt", "nov", "dec"], + months_genitive: None, + weekdays: ["zondag", "maandag", "dinsdag", "woensdag", "donderdag", "vrijdag", "zaterdag"], + weekdays_abbr: ["zo", "ma", "di", "wo", "do", "vr", "za"], + am_pm: AM_PM, + currency: "€", + currency_format: "\"€\" #,##0.00", + currency_format_int: "\"€\" #,##0", +}; + +static PT_BR: Region = Region { + tag: "pt-BR", + name: "Português (Brasil)", + lcid: 0x0416, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d \"de\" mmmm \"de\" yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"], + months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"], + months_genitive: None, + weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"], + weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"], + am_pm: AM_PM, + currency: "R$", + currency_format: "\"R$\" #,##0.00", + currency_format_int: "\"R$\" #,##0", +}; + +static PT_PT: Region = Region { + tag: "pt-PT", + name: "Português (Portugal)", + lcid: 0x0816, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d \"de\" mmmm \"de\" yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"], + months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"], + months_genitive: None, + weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"], + weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static RU_RU: Region = Region { + tag: "ru-RU", + name: "Русский (Россия)", + lcid: 0x0419, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "d mmmm yyyy\" г.\"", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["январь", "февраль", "март", "апрель", "май", "июнь", "июль", "август", "сентябрь", "октябрь", "ноябрь", "декабрь"], + months_abbr: ["янв.", "февр.", "март", "апр.", "май", "июнь", "июль", "авг.", "сент.", "окт.", "нояб.", "дек."], + months_genitive: Some(["января", "февраля", "марта", "апреля", "мая", "июня", "июля", "августа", "сентября", "октября", "ноября", "декабря"]), + weekdays: ["воскресенье", "понедельник", "вторник", "среда", "четверг", "пятница", "суббота"], + weekdays_abbr: ["вс", "пн", "вт", "ср", "чт", "пт", "сб"], + am_pm: AM_PM, + currency: "₽", + currency_format: "#,##0.00\u{a0}\"₽\"", + currency_format_int: "#,##0\u{a0}\"₽\"", +}; + +static CS_CZ: Region = Region { + tag: "cs-CZ", + name: "Čeština (Česko)", + lcid: 0x0405, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd d. mmmm yyyy", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "dd.mm.yyyy h:mm", + months: ["leden", "únor", "březen", "duben", "květen", "červen", "červenec", "srpen", "září", "říjen", "listopad", "prosinec"], + months_abbr: ["led", "úno", "bře", "dub", "kvě", "čvn", "čvc", "srp", "zář", "říj", "lis", "pro"], + months_genitive: Some(["ledna", "února", "března", "dubna", "května", "června", "července", "srpna", "září", "října", "listopadu", "prosince"]), + weekdays: ["neděle", "pondělí", "úterý", "středa", "čtvrtek", "pátek", "sobota"], + weekdays_abbr: ["ne", "po", "út", "st", "čt", "pá", "so"], + am_pm: ["dop.", "odp."], + currency: "Kč", + currency_format: "#,##0.00 \"Kč\"", + currency_format_int: "#,##0 \"Kč\"", +}; + +static JA_JP: Region = Region { + tag: "ja-JP", + name: "日本語 (日本)", + lcid: 0x0411, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '/', + time_sep: ':', + short_date: "yyyy/m/d", + long_date: "yyyy\"年\"m\"月\"d\"日\"", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "yyyy/m/d h:mm", + months: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_genitive: None, + weekdays: ["日曜日", "月曜日", "火曜日", "水曜日", "木曜日", "金曜日", "土曜日"], + weekdays_abbr: ["日", "月", "火", "水", "木", "金", "土"], + am_pm: ["午前", "午後"], + currency: "¥", + currency_format: "\"¥\"#,##0", + currency_format_int: "\"¥\"#,##0", +}; + +static KO_KR: Region = Region { + tag: "ko-KR", + name: "한국어 (대한민국)", + lcid: 0x0412, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '-', + time_sep: ':', + short_date: "yyyy-mm-dd", + long_date: "yyyy\"년\" m\"월\" d\"일\" dddd", + short_time: "AM/PM h:mm", + long_time: "AM/PM h:mm:ss", + datetime: "yyyy-mm-dd hh:mm", + months: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"], + months_abbr: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"], + months_genitive: None, + weekdays: ["일요일", "월요일", "화요일", "수요일", "목요일", "금요일", "토요일"], + weekdays_abbr: ["일", "월", "화", "수", "목", "금", "토"], + am_pm: ["오전", "오후"], + currency: "₩", + currency_format: "\"₩\"#,##0", + currency_format_int: "\"₩\"#,##0", +}; + +static ZH_CN: Region = Region { + tag: "zh-CN", + name: "中文 (中国)", + lcid: 0x0804, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '/', + time_sep: ':', + short_date: "yyyy/m/d", + long_date: "yyyy\"年\"m\"月\"d\"日\"", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "yyyy/m/d h:mm", + months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"], + months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_genitive: None, + weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"], + weekdays_abbr: ["周日", "周一", "周二", "周三", "周四", "周五", "周六"], + am_pm: ["上午", "下午"], + currency: "¥", + currency_format: "\"¥\"#,##0.00", + currency_format_int: "\"¥\"#,##0", +}; + +static ZH_TW: Region = Region { + tag: "zh-TW", + name: "中文 (台灣)", + lcid: 0x0404, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '/', + time_sep: ':', + short_date: "yyyy/m/d", + long_date: "yyyy\"年\"m\"月\"d\"日\"", + short_time: "AM/PM hh:mm", + long_time: "AM/PM hh:mm:ss", + datetime: "yyyy/m/d hh:mm", + months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"], + months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_genitive: None, + weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"], + weekdays_abbr: ["週日", "週一", "週二", "週三", "週四", "週五", "週六"], + am_pm: ["上午", "下午"], + currency: "NT$", + currency_format: "\"NT$\"#,##0.00", + currency_format_int: "\"NT$\"#,##0", +}; + +static ALL: [&Region; 20] = [ + &EN_US, &EN_GB, &EN_IN, &EN_CH, &ES_ES, &DE_DE, &DE_CH, &FR_FR, &FR_CH, &IT_IT, &IT_CH, &NL_NL, &PT_BR, &PT_PT, &RU_RU, &CS_CZ, &JA_JP, &KO_KR, + &ZH_CN, &ZH_TW, +]; + +/// Every region, English (United States) first. +pub fn all() -> &'static [&'static Region] { + &ALL +} + +/// The default region, English (United States). +pub fn default_region() -> &'static Region { + &EN_US +} + +/// The region with exactly this tag (`pt-BR`; case and `_` don't matter). +pub fn find(tag: &str) -> Option<&'static Region> { + let t = tag.trim().replace('_', "-"); + ALL.iter().copied().find(|r| r.tag.eq_ignore_ascii_case(&t)) +} + +/// The region for a tag as operating systems and browsers write it: `pt-BR`, `pt_BR.UTF-8`, +/// `de`, `en-AU`, `zh-Hant`, `sr_RS@latin`. An exact match wins; otherwise the language's main +/// region (with the usual variants: `en-AU`→`en-GB`, `pt-AO`→`pt-PT`, `zh-Hant`→`zh-TW`). +/// An unknown language is English (United States). +pub fn lookup(tag: &str) -> &'static Region { + if let Some(r) = find(tag) { + return r; + } + let base = tag.split(['.', '@']).next().unwrap_or(""); + let norm = base.trim().replace('_', "-"); + if let Some(r) = find(&norm) { + return r; + } + let mut parts = norm.split('-').filter(|p| !p.is_empty()); + let lang = parts.next().unwrap_or("").to_ascii_lowercase(); + let mut script = String::new(); + let mut country = String::new(); + for p in parts { + // An extension or private-use singleton (`-u-ca-gregory`, `-x-…`) ends the region part. + if p.len() == 1 { + break; + } + if p.len() == 4 && p.chars().all(|c| c.is_ascii_alphabetic()) { + script = p.to_ascii_lowercase(); + } else if (p.len() == 2 && p.chars().all(|c| c.is_ascii_alphabetic())) || (p.len() == 3 && p.chars().all(|c| c.is_ascii_digit())) { + country = p.to_ascii_uppercase(); + } + } + let tag = match lang.as_str() { + "en" => match country.as_str() { + "GB" | "AU" | "NZ" | "IE" => "en-GB", + "IN" => "en-IN", + "CH" | "LI" => "en-CH", + _ => "en-US", + }, + "pt" => match country.as_str() { + "AO" | "CH" | "CV" | "FR" | "GQ" | "GW" | "LU" | "MO" | "MZ" | "ST" | "TL" | "PT" => "pt-PT", + _ => "pt-BR", + }, + "zh" => match country.as_str() { + // An explicit territory wins over the script (`zh-Hant-CN` is China's conventions). + "CN" | "SG" => "zh-CN", + "TW" | "HK" | "MO" => "zh-TW", + _ if script == "hant" => "zh-TW", + _ => "zh-CN", + }, + "es" => "es-ES", + "de" => match country.as_str() { + "CH" | "LI" => "de-CH", + _ => "de-DE", + }, + "fr" => match country.as_str() { + "CH" | "LI" => "fr-CH", + _ => "fr-FR", + }, + "it" => match country.as_str() { + "CH" | "LI" => "it-CH", + _ => "it-IT", + }, + "nl" => "nl-NL", + "ru" => "ru-RU", + "cs" => "cs-CZ", + "ja" => "ja-JP", + "ko" => "ko-KR", + _ => "en-US", + }; + find(tag).unwrap_or(&EN_US) +} + +impl Region { + /// True for the characters that separate groups of digits in this region. A region that + /// groups with a space also accepts the other space-like characters (NBSP, narrow NBSP, thin + /// space), as people type and paste any of them; one that groups with an apostrophe accepts + /// both `'` and `’`. + pub fn is_group(&self, c: char) -> bool { + c == self.group || (is_space_like(self.group) && is_space_like(c)) || (is_apostrophe_like(self.group) && is_apostrophe_like(c)) + } + + /// True when the currency symbol comes before the number. + pub fn currency_first(&self) -> bool { + self.currency_format.starts_with('"') + } + + /// Month number (1–12) for a month name as typed: the full name, the abbreviation (with or + /// without its period) or any prefix of the full name of three or more letters (`Sept`), + /// ignoring case. `word` is one word: the genitive form used after a day (`января` in + /// `1 января`) matches too, and the caller reads the day itself. + pub fn month_from_name(&self, word: &str) -> Option { + let w = word.trim().to_lowercase(); + // Where the abbreviations end in a period (`сент.`, `janv.`), any abbreviation may be typed + // with one (`мар.`, `juin.`). Elsewhere a period is not part of a month name, so en-US + // `Mar. 4, 2026` stays text as before. + let dotted = self.months_abbr.iter().any(|a| a.ends_with('.')); + let stem = if dotted { w.trim_end_matches('.') } else { w.as_str() }; + if stem.is_empty() { + return None; + } + let long_enough = stem.chars().count() >= 3; + for (i, full) in self.months.iter().enumerate() { + let full_l = full.to_lowercase(); + let abbr_l = self.months_abbr.get(i).map(|a| a.to_lowercase()).unwrap_or_default(); + let gen_l = self.months_genitive.as_ref().and_then(|g| g.get(i)).map(|g| g.to_lowercase()); + let hit = w == full_l + || w == abbr_l + || stem == abbr_l.trim_end_matches('.') + || gen_l.as_deref() == Some(w.as_str()) + || (long_enough && full_l.starts_with(stem)) + || (long_enough && gen_l.as_deref().is_some_and(|g| g.starts_with(stem))); + if hit { + return Some(i as u32 + 1); + } + } + None + } + + /// The name of month `month` (1–12) for `mmmm`; `with_day` selects the genitive form where + /// the region has one. + pub fn month_name(&self, month: u32, with_day: bool) -> &'static str { + let i = (month as usize).wrapping_sub(1); + if with_day && let Some(g) = &self.months_genitive { + return g.get(i).copied().unwrap_or(""); + } + self.months.get(i).copied().unwrap_or("") + } + + /// True when the short time uses a 12-hour clock. + pub fn is_12_hour(&self) -> bool { + self.short_time.contains("AM/PM") + } +} + +fn is_space_like(c: char) -> bool { + matches!(c, ' ' | '\u{a0}' | '\u{202f}' | '\u{2009}') +} + +fn is_apostrophe_like(c: char) -> bool { + matches!(c, '\'' | '\u{2019}') +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn lookup_accepts_system_and_browser_tags() { + let cases = [ + ("pt-BR", "pt-BR"), + ("pt_BR.UTF-8", "pt-BR"), + ("pt", "pt-BR"), + ("pt-PT", "pt-PT"), + ("pt-AO", "pt-PT"), + ("de", "de-DE"), + ("de-AT", "de-DE"), + ("de_CH.UTF-8", "de-CH"), + ("de-LI", "de-CH"), + ("fr-CH", "fr-CH"), + ("fr-LI", "fr-CH"), + ("fr-BE", "fr-FR"), + ("it_CH.UTF-8", "it-CH"), + ("it-LI", "it-CH"), + ("en-CH", "en-CH"), + ("nl_NL.UTF-8", "nl-NL"), + ("nl", "nl-NL"), + ("en-AU", "en-GB"), + ("en_GB.UTF-8", "en-GB"), + ("en-IN", "en-IN"), + ("en-CA", "en-US"), + ("zh-Hant", "zh-TW"), + ("zh-HK", "zh-TW"), + ("zh", "zh-CN"), + // Extensions don't replace the region or script; an explicit territory beats the script. + ("en-GB-u-ca-gregory", "en-GB"), + ("zh-Hant-u-nu-latn", "zh-TW"), + ("zh-Hant-CN", "zh-CN"), + ("zh-Hans-TW", "zh-TW"), + ("cs_CZ.UTF-8", "cs-CZ"), + ("C", "en-US"), + ("", "en-US"), + ("xx-YY", "en-US"), + ]; + for (tag, expected) in cases { + assert_eq!(lookup(tag).tag, expected, "{tag}"); + } + assert_eq!(find("DE-de").map(|r| r.tag), Some("de-DE")); + assert!(find("xx").is_none()); + } + + #[test] + fn every_region_is_consistent() { + let mut tags = std::collections::HashSet::new(); + for r in all() { + assert!(tags.insert(r.tag), "duplicate {}", r.tag); + assert_ne!(r.decimal, r.group, "{}", r.tag); + assert!(r.currency_format.contains(r.currency), "{}", r.tag); + assert!(r.months.iter().all(|m| !m.is_empty()) && r.months_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag); + assert!(r.weekdays.iter().all(|m| !m.is_empty()) && r.weekdays_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag); + assert_eq!(find(r.tag).map(|x| x.tag), Some(r.tag)); + } + } + + #[test] + fn month_names_match_loosely() { + let de = find("de-DE").unwrap(); + assert_eq!(de.month_from_name("März"), Some(3)); + assert_eq!(de.month_from_name("mär"), Some(3)); + assert_eq!(de.month_from_name("DEZ"), Some(12)); + assert_eq!(de.month_from_name("de"), None); + let gb = find("en-GB").unwrap(); + assert_eq!(gb.month_from_name("Sept"), Some(9)); + assert_eq!(gb.month_from_name("sep"), Some(9)); + let ru = find("ru-RU").unwrap(); + assert_eq!(ru.month_from_name("сентября"), Some(9)); + assert_eq!(ru.month_from_name("сент."), Some(9)); + // Abbreviations typed with a period the table doesn't carry. + assert_eq!(ru.month_from_name("мар."), Some(3)); + assert_eq!(ru.month_from_name("июн."), Some(6)); + assert_eq!(find("fr-FR").unwrap().month_from_name("juin."), Some(6)); + assert_eq!(EN_US.month_from_name("Mar."), None); + assert_eq!(de.month_from_name("."), None); + assert_eq!(EN_US.month_from_name("Ma"), None); + assert_eq!(EN_US.month_from_name("May"), Some(5)); + } + + #[test] + fn space_like_and_apostrophe_groups_are_interchangeable() { + let fr = find("fr-FR").unwrap(); + assert!(fr.is_group('\u{a0}') && fr.is_group(' ') && fr.is_group('\u{202f}')); + assert!(!find("de-DE").unwrap().is_group(' ')); + let ch = find("de-CH").unwrap(); + assert!(ch.is_group('\'') && ch.is_group('\u{2019}')); + assert!(!ch.is_group(' ') && !find("de-DE").unwrap().is_group('\'')); + } +} diff --git a/crates/model/src/workbook.rs b/crates/model/src/workbook.rs index 665ec323f..ab401828d 100644 --- a/crates/model/src/workbook.rs +++ b/crates/model/src/workbook.rs @@ -268,6 +268,11 @@ pub struct Workbook { /// `preserved` are relative to. Saving into another folder rewrites them from there. Not in JSON. #[serde(skip)] pub source_dir: Option, + /// The region whose formats this workbook is typed and shown in (Options › Region). A runtime + /// setting of the application, not of the file: it is not saved, and the engine session sets + /// it on every document it holds. en-US until then. + #[serde(skip, default = "gridcraft_core::region::default_region")] + pub region: &'static gridcraft_core::Region, /// The file this workbook reports as its own, which is what `CELL("filename")` returns. /// `None` = no file yet, which Excel reports as an empty string until the workbook has been /// saved once. Deliberately separate from `source_dir`: that one names the folder relative @@ -365,6 +370,7 @@ impl Workbook { macros_unread: Arc::default(), preserved: None, source_dir: None, + region: gridcraft_core::region::default_region(), source_path: None, } } From 50a9e9ede44a0c9bd1840fa78e12fcdc204f0c73 Mon Sep 17 00:00:00 2001 From: Harrison Booth <73434116+Harrison-Booth@users.noreply.github.com> Date: Tue, 10 Nov 2026 08:05:02 -0600 Subject: [PATCH 4/6] XLS: formulas open as legacy formulas with Excel's implicit @ (#410) (#1367) .xls formulas are read as legacy formulas with Excel's implicit @ (apply_legacy via read_xls_with), one-cell Ctrl+Shift+Enter arrays keep their range, @ of a 2-D range meets the formula's row and column, and defined-name analyses are shared across the formulas of a pass and replayed exactly. Closes #410 --- crates/calc/src/eval.rs | 5 + crates/calc/src/tests.rs | 19 ++- crates/engine/src/io.rs | 5 +- crates/engine/src/tests.rs | 73 ++++++++++ crates/engine/tests/legacy_array_functions.rs | 33 ++++- crates/xlsx/src/array_formula.rs | 119 +++++++++++++++- crates/xlsx/src/lib.rs | 2 +- crates/xlsx/src/sheet_write.rs | 2 +- crates/xlsx/src/tests/array_export.rs | 132 ++++++++++++++++++ crates/xlsx/src/tests/xls_tests.rs | 113 ++++++++++++++- crates/xlsx/src/write.rs | 3 + crates/xlsx/src/xls.rs | 33 ++++- docs/file-format-parity.md | 3 +- 13 files changed, 518 insertions(+), 24 deletions(-) diff --git a/crates/calc/src/eval.rs b/crates/calc/src/eval.rs index 41ffca9c2..2fda4442b 100644 --- a/crates/calc/src/eval.rs +++ b/crates/calc/src/eval.rs @@ -543,6 +543,11 @@ impl<'h> Evaluator<'h> { if r.height() == 1 && r.start.col <= self.at.col && self.at.col <= r.end.col { return Some(CellRef::new(r.start.row, self.at.col)); } + // A 2-D range intersects on both the formula's row and column (observed in Excel, on + // another sheet too); one that includes the formula's own cell is a circular reference. + if r.start.row <= self.at.row && self.at.row <= r.end.row && r.start.col <= self.at.col && self.at.col <= r.end.col { + return Some(self.at); + } None } diff --git a/crates/calc/src/tests.rs b/crates/calc/src/tests.rs index b4f820f0c..8784a8d69 100644 --- a/crates/calc/src/tests.rs +++ b/crates/calc/src/tests.rs @@ -1923,8 +1923,8 @@ fn offset_extreme_arguments_return_ref() { } /// `@` intersects a reference with the formula's row (a column range) or column (a row range); -/// a 2D range or one the formula does not line up with gives #VALUE!. An array gives its -/// top-left value. Legacy formulas read from files use it where Excel intersects implicitly. +/// a 2D range both (circular when it includes the formula's cell); a range the formula does not +/// line up with gives #VALUE!. An array gives its top-left value. Legacy formulas read from files use it where Excel intersects implicitly. #[test] fn implicit_intersection_operator() { let mut t = T::new(); @@ -1941,9 +1941,22 @@ fn implicit_intersection_operator() { t.set("D3", "=SUM(@A1:A3)"); assert_eq!(t.num("B2"), 2f64.tanh()); assert_eq!(t.num("B7"), 5f64.tanh()); - for at in ["B3", "B9", "E9"] { + for at in ["B9", "E9"] { assert_eq!(t.get(at), Value::Error(CellError::Value), "{at}"); } + // A 2D range that includes the formula's own cell is a circular reference (Excel shows 0). + assert_eq!(t.get("B3"), Value::Error(CellError::Circ)); + // From another sheet a 2D range intersects on both row and column; outside it is #VALUE!. + t.wb.sheets.push(std::sync::Arc::new(gridcraft_model::Sheet::new("Other"))); + for (at, input) in [("B2", "=@Sheet1!A1:C3"), ("C1", "=@Sheet1!A1:C3"), ("B9", "=@Sheet1!A1:C3"), ("E2", "=@Sheet1!A1:C3")] { + t.wb.sheet_mut(1).unwrap().cells.set(c(at), Cell::formula(Formula::new(input))); + } + t.calc.recalc_all(&mut t.wb); + let other = |at: &str| t.wb.sheet(1).unwrap().value(c(at)); + assert_eq!(other("B2"), Value::Number(2f64.tanh())); // Sheet1!B2 + assert_eq!(other("C1"), Value::Number(0.0)); + assert_eq!(other("B9"), Value::Error(CellError::Value)); + assert_eq!(other("E2"), Value::Error(CellError::Value)); assert_eq!(t.num("D1"), 7.0); assert_eq!(t.num("D2"), 2.0); assert_eq!(t.num("D3"), 3.0); diff --git a/crates/engine/src/io.rs b/crates/engine/src/io.rs index 22c236e61..690efec1d 100644 --- a/crates/engine/src/io.rs +++ b/crates/engine/src/io.rs @@ -9,7 +9,7 @@ use crate::{EngineError, Result}; mod sylk; -/// The function library's answers to what the XLSX reader and writer need to know about functions +/// The function library's answers to what the XLSX and XLS readers and the XLSX writer need to know about functions /// (which names are built in, which parameters take a single value). pub const XLSX_FUNCTIONS: gridcraft_xlsx::FunctionInfo = gridcraft_xlsx::FunctionInfo { is_builtin: |name| gridcraft_functions::lookup(name).is_some(), @@ -271,7 +271,8 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, let kind = FileKind::from_path(name); let sniffed = gridcraft_xlsx::sniff(bytes); if sniffed == gridcraft_xlsx::Format::Xls { - let (wb, report) = gridcraft_xlsx::read_xls(bytes).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?; + let (wb, report) = + gridcraft_xlsx::read_xls_with(bytes, XLSX_FUNCTIONS).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?; return Ok((wb, report.warnings)); } if sniffed == gridcraft_xlsx::Format::LegacyBinary { diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index fcaf19c36..08fc9119f 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -3664,6 +3664,79 @@ fn xls_import_keeps_formulas_and_never_saves_back_to_xls() { assert!(s.execute("file.saveBytes", json!({"format": "xlsx"})).is_ok()); } +#[test] +fn xls_formulas_calculate_with_legacy_implicit_intersection() { + // Data!A1 = 11, B1:G1 = 1, A2 = 5, A3 = 7. Report!B1:G1 are filled, so a formula in Report!A1 + // that spilled would show #SPILL!. Every cached result is 0, so the values below are GridCraft's. + fn rec(id: u16, payload: &[u8]) -> Vec { + [&id.to_le_bytes()[..], &(payload.len() as u16).to_le_bytes(), payload].concat() + } + let u16s = |v: &[u16]| v.iter().flat_map(|x| x.to_le_bytes()).collect::>(); + let bof = |dt: u16| rec(0x0809, &[&0x0600u16.to_le_bytes()[..], &dt.to_le_bytes(), &[0; 12]].concat()); + let number = |r: u16, c: u16, n: f64| rec(0x0203, &[&u16s(&[r, c, 0])[..], &n.to_le_bytes()].concat()); + let formula = |r: u16, c: u16, rgce: &[u8]| { + rec(0x0006, &[&u16s(&[r, c, 0])[..], &0f64.to_le_bytes(), &[0; 6], &(rgce.len() as u16).to_le_bytes(), rgce].concat()) + }; + let area3d = |r0: u16, r1: u16, c0: u16, c1: u16| [&[0x3Bu8, 0, 0][..], &u16s(&[r0, r1, c0, c1])].concat(); + let a2a3 = area3d(1, 2, 0, 0); + let mut data = bof(0x0010); + data.extend([number(0, 0, 11.0), number(1, 0, 5.0), number(2, 0, 7.0)].concat()); + (1..7).for_each(|c| data.extend(number(0, c, 1.0))); + data.extend(rec(0x000A, &[])); + let mut report = bof(0x0010); + (1..7).for_each(|c| report.extend(number(0, c, 0.0))); + // A1 = Data!A1:G1; B2 = Data!$A$2:$A$3*10; C2 = SQRT(Data!$A$2:$A$3) (row 2: SQRT(5)); + // H1 = {SUM(Data!$A$2:$A$3*2)}, a one-cell Ctrl+Shift+Enter array. + let cse = [&a2a3[..], &[0x1E, 2, 0, 0x05, 0x22, 1, 4, 0]].concat(); + report.extend(formula(0, 0, &area3d(0, 0, 0xC000, 6 | 0xC000))); + report.extend(formula(1, 1, &[&a2a3[..], &[0x1E, 10, 0, 0x05]].concat())); + report.extend(formula(1, 2, &[&a2a3[..], &[0x21, 20, 0]].concat())); + report.extend(formula(0, 7, &[1, 0, 0, 7, 0])); + report.extend(rec(0x0221, &[&u16s(&[0, 0])[..], &[7, 7], &[0; 6], &(cse.len() as u16).to_le_bytes(), &cse].concat())); + report.extend(rec(0x000A, &[])); + let boundsheet = |pos: u32, name: &str| rec(0x0085, &[&pos.to_le_bytes()[..], &[0, 0, name.len() as u8, 0], name.as_bytes()].concat()); + let globals = |p1: u32, p2: u32| { + [ + bof(0x0005), + boundsheet(p1, "Data"), + boundsheet(p2, "Report"), + rec(0x01AE, &u16s(&[2, 0x0401])), + rec(0x0017, &u16s(&[1, 0, 0, 0])), + rec(0x000A, &[]), + ] + .concat() + }; + let p1 = globals(0, 0).len() as u32; + let stream = [globals(p1, p1 + data.len() as u32), data, report].concat(); + let bytes = gridcraft_xlsx::testing::compound_file(&[("Workbook", &stream)]); + + let mut s = s(); + s.execute("file.open", json!({"name": "legacy.xls", "base64": crate::io::base64_encode(&bytes)})).unwrap(); + s.execute("formulas.calculateNow", json!({})).unwrap(); + let report = |s: &Session, a1: &str| s.doc().unwrap().wb.sheets[1].value(CellRef::parse(a1).unwrap()); + assert_eq!(report(&s, "A1"), Value::Number(11.0)); + assert_eq!(report(&s, "B2"), Value::Number(50.0)); + assert_eq!(report(&s, "C2"), Value::Number(5f64.sqrt())); + assert_eq!(report(&s, "H1"), Value::Number(24.0)); + // Nothing spilled out of A1 or B2. + assert_eq!(report(&s, "B1"), Value::Number(0.0)); + assert_eq!(report(&s, "B3"), Value::Empty); + // Saved as .xlsx, the formulas carry no dynamic-array metadata. + let r = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let saved = crate::io::base64_decode(r["base64"].as_str().unwrap()).unwrap(); + let (wb, _) = gridcraft_xlsx::read_xlsx(&saved).unwrap(); + let f = |a1: &str| wb.sheets[1].cell(CellRef::parse(a1).unwrap()).and_then(|c| c.formula.clone()).unwrap(); + assert_eq!(f("A1").text, "@Data!A1:G1"); + assert_eq!((f("H1").text.as_str(), f("H1").array), ("SUM(Data!$A$2:$A$3*2)", gridcraft_core::RangeRef::parse("H1"))); + let mut sheet = String::new(); + std::io::Read::read_to_string( + &mut zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap().by_name("xl/worksheets/sheet2.xml").unwrap(), + &mut sheet, + ) + .unwrap(); + assert!(!sheet.contains("cm=") && !sheet.contains("SINGLE"), "{sheet}"); +} + #[test] fn malformed_xlsb_keeps_the_current_workbook() { let mut s = s(); diff --git a/crates/engine/tests/legacy_array_functions.rs b/crates/engine/tests/legacy_array_functions.rs index 1549b6f70..6e86f2e13 100644 --- a/crates/engine/tests/legacy_array_functions.rs +++ b/crates/engine/tests/legacy_array_functions.rs @@ -228,6 +228,28 @@ fn legacy_unions_names_and_parentheses_calculate_and_save_as_read() { assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}"); } +/// A whole column in a scalar argument of a legacy formula meets the formula's row (issue #1636, +/// from PR #1660): `ISBLANK(B:B)` in row 4 tests `B4`, so the formula shows `C4` instead of +/// spilling into `G5`, and in row 6, where `B6` is blank, shows 0. +#[test] +fn whole_column_in_a_legacy_scalar_argument_meets_the_formulas_row() { + let formulas: Vec<(String, String)> = [ + ("B4", r#""Thierry""#), + ("C4", "7"), + ("G4", r#"IF(ISBLANK(B:B),0,IF(B4="Thierry",C4,0))"#), + ("G5", "1"), + ("G6", r#"IF(ISBLANK(B:B),0,IF(B6="Thierry",C6,0))"#), + ] + .iter() + .map(|(a, f)| (a.to_string(), f.to_string())) + .collect(); + let s = open(&legacy_xlsx(&formulas, "")); + let at = |a: &str| CellRef::parse(a).unwrap(); + assert_eq!(value(&s, at("G4")), Value::Number(7.0), "B:B meets row 4 instead of spilling"); + assert_eq!(value(&s, at("G5")), Value::Number(1.0), "nothing spills below G4"); + assert_eq!(one_value(&s, at("G6"), "G6").unwrap(), Value::Number(0.0), "B6 is blank"); +} + /// One-cell ranges written as ranges (`B1:B1`) outside any `@`, and formulas whose reading /// reaches the parser's nesting limit (254 nested functions, far past Excel's 64): saved as read, /// without `@`. Only a one-cell range under an `@` counts as one an edit may have shrunk. All but @@ -302,9 +324,18 @@ fn legacy_xlsx(formulas: &[(String, String)], names: &str) -> Vec { let esc = |s: &str| s.replace('&', "&").replace('<', "<").replace('>', ">").replace('"', """); let mut rows = String::from(r#"1234"#); + // Cells in one row share its ``, in the order given. + let mut by_row: Vec<(String, String)> = Vec::new(); for (a, f) in formulas { let r: String = a.chars().filter(char::is_ascii_digit).collect(); - rows.push_str(&format!(r#"{}"#, esc(f))); + let c = format!(r#"{}"#, esc(f)); + match by_row.iter_mut().find(|(row, _)| *row == r) { + Some((_, cells)) => cells.push_str(&c), + None => by_row.push((r, c)), + } + } + for (r, cells) in by_row { + rows.push_str(&format!(r#"{cells}"#)); } let names = if names.is_empty() { String::new() } else { format!("{names}") }; let parts: [(&str, String); 5] = [ diff --git a/crates/xlsx/src/array_formula.rs b/crates/xlsx/src/array_formula.rs index 8d871dd75..804192969 100644 --- a/crates/xlsx/src/array_formula.rs +++ b/crates/xlsx/src/array_formula.rs @@ -88,11 +88,11 @@ pub(crate) enum Export { Dynamic, } -pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo) -> Export { +pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo, names: &mut NameCache) -> Export { let Some(expr) = formula.expr_arc() else { return Export::Unchanged }; let mut explicit = false; expr.walk(&mut |e| explicit |= matches!(e, Expr::Unary(UnOp::At, _))); - let mut legacy = Legacy::new(wb, at, functions); + let mut legacy = Legacy::new(wb, at, functions, names); if !explicit { legacy.formula(expr.as_ref().clone(), sheet); return if !legacy.certain { @@ -119,7 +119,7 @@ pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula candidates.insert(0, unwrap_at(x.as_ref().clone())); } for text in candidates { - let read = self::legacy(wb, sheet, at, &Formula::from_expr(text.clone()), functions); + let read = self::legacy(wb, sheet, at, &Formula::from_expr(text.clone()), functions, names); if read.map_or(text == *expr, |f| f.expr_arc().is_some_and(|e| *e == *expr)) { return reads_back(text).map_or(Export::Dynamic, Export::Legacy); } @@ -140,12 +140,14 @@ fn remove_at(e: Expr) -> Expr { /// Rewrites the legacy cell formulas at `cells` (model sheet index, positions) to their /// dynamic-array reading, with `@` where Excel intersects. Runs once names and tables are loaded. pub(crate) fn apply_legacy(wb: &mut Workbook, cells: Vec<(usize, Vec)>, functions: FunctionInfo) { + // Defined names do not change during the pass: each is analysed once for all the cells. + let mut names = NameCache::default(); for (si, positions) in cells { let Some(sheet) = wb.sheet(si) else { continue }; let mut changed = Vec::new(); for at in positions { let Some(formula) = sheet.cell(at).and_then(|c| c.formula.as_deref()) else { continue }; - if let Some(f) = legacy(wb, si, at, formula, functions) { + if let Some(f) = legacy(wb, si, at, formula, functions, &mut names) { changed.push((at, f)); } } @@ -162,12 +164,12 @@ pub(crate) fn apply_legacy(wb: &mut Workbook, cells: Vec<(usize, Vec)>, } /// The dynamic-array reading of a legacy formula, when it differs. -fn legacy(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo) -> Option { +fn legacy(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo, names: &mut NameCache) -> Option { if formula.array.is_some() { return None; } let expr = formula.expr_arc()?; - let mut legacy = Legacy::new(wb, at, functions); + let mut legacy = Legacy::new(wb, at, functions, names); let out = legacy.formula(expr.as_ref().clone(), sheet); if legacy.certain && legacy.complete { if legacy.inserted == 0 { @@ -228,8 +230,56 @@ fn atomic(e: &Expr) -> bool { /// (see [`Legacy::binary`]). const MAX_DEPTH: usize = 256; +/// What the analysis of one defined name leaves for a later formula that refers to it. +#[derive(Clone)] +struct NameInfo { + kind: Kind, + /// The definition reached a one-cell reference written as a range. + one_cell_range: bool, + /// The analysis covered only established legacy semantics. + certain: bool, + /// Nodes of the per-formula budget the analysis used. + cost: usize, + /// Nesting below the reference that the analysis reached. + depth: usize, + /// Every name the analysis read, with its result (itself last): a hit records them all, as a + /// fresh analysis does, so a later reference to one of them in the formula costs nothing. + closure: Vec<(NameKey, Kind, bool)>, +} + +/// A defined name as [`Legacy::names`] keys it: upper-case name, scope, collapsed. +type NameKey = (String, usize, bool); + +/// Defined names analysed so far, shared by the formulas of one pass (reading a workbook, or +/// writing one) so that a name is parsed and analysed once, not once per formula that refers +/// to it. Keyed like [`Legacy::names`]. Only analyses that finished within their limits, did not +/// depend on the formula's own position and did not reuse a name the formula had already read +/// (or meet a cycle) are kept. A hit charges the budget and nesting the analysis used and records +/// every name it read, and is taken only when the formula can afford it, so a formula reads the +/// same with the cache as without it. +#[derive(Default)] +pub(crate) struct NameCache { + map: HashMap, + /// How many names were analysed (not found in the cache). + pub(crate) analyses: usize, +} + +/// Names kept at most: the cache is bounded even when a workbook defines a great many. +const MAX_CACHED_NAMES: usize = 65_536; + struct Legacy<'a> { wb: &'a Workbook, + /// Name analyses shared with the other formulas of this pass. + shared: &'a mut NameCache, + /// The analysis read a structured reference without a table name, which depends on `at`. + position_dependent: bool, + /// The analysis reused a name this formula had already read, or met a cycle: its result + /// leaves out what that name cost. + reused: bool, + /// The names read so far in this formula, in order, with their results. + read: Vec<(NameKey, Kind, bool)>, + /// The deepest nesting reached. + deepest: usize, at: CellRef, functions: FunctionInfo, remaining: usize, @@ -262,9 +312,14 @@ struct Legacy<'a> { } impl<'a> Legacy<'a> { - fn new(wb: &'a Workbook, at: CellRef, functions: FunctionInfo) -> Self { + fn new(wb: &'a Workbook, at: CellRef, functions: FunctionInfo, shared: &'a mut NameCache) -> Self { Legacy { wb, + shared, + position_dependent: false, + reused: false, + read: Vec::new(), + deepest: 0, at, functions, remaining: 16384, @@ -364,6 +419,7 @@ impl<'a> Legacy<'a> { fn expr(&mut self, e: Expr, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { // Names can be cyclic or nest deeply; their kinds are remembered, so repeated names // cost one analysis each. + self.deepest = self.deepest.max(depth); if depth >= MAX_DEPTH || self.remaining == 0 { self.complete = false; return (e, Kind::Array); @@ -407,6 +463,7 @@ impl<'a> Legacy<'a> { } Expr::Struct(r) => { let table = if r.table.is_empty() { + self.position_dependent = true; self.wb.sheet(sheet).and_then(|s| s.table_at(self.at)) } else { self.wb.table(&r.table).and_then(|(si, ti)| self.wb.sheet(si).and_then(|s| s.tables.get(ti))) @@ -459,25 +516,73 @@ impl<'a> Legacy<'a> { let key = (bare.to_ascii_uppercase(), scope, self.collapsed); match self.names.get(&key) { Some(Some((kind, one_cell_range))) => { + self.reused = true; self.one_cell_range |= *one_cell_range; return *kind; } Some(None) => { // A name defined in terms of itself has no legacy value to keep. self.certain = false; + self.reused = true; return Kind::Array; } None => {} } + let reached = depth.saturating_add(1); + // Taken only when the formula can afford it and has read none of the names it read: + // otherwise the fresh analysis below differs (cut short, or reusing those names). + let hit = self.shared.map.get(&key).filter(|i| { + reached + i.depth < MAX_DEPTH && i.cost <= self.remaining && i.closure.iter().all(|(k, _, _)| !self.names.contains_key(k)) + }); + if let Some(info) = hit { + // Charged and recorded as the analysis was: the formula reads as a fresh one does. + self.deepest = self.deepest.max(reached + info.depth); + self.certain &= info.certain; + self.one_cell_range |= info.one_cell_range; + self.remaining -= info.cost; + for (k, kind, one_cell_range) in &info.closure { + self.names.insert(k.clone(), Some((*kind, *one_cell_range))); + } + self.read.extend(info.closure.iter().cloned()); + return info.kind; + } let Ok(expr) = gridcraft_formula::parse_name(def.formula.trim_start_matches('=')) else { return Kind::Array }; + self.shared.analyses += 1; self.names.insert(key.clone(), None); // A name's own formula does not see the formula's LET variables. let locals = std::mem::take(&mut self.locals); let seen = std::mem::replace(&mut self.one_cell_range, false); + let before = self.remaining; + let first_read = self.read.len(); + let outer = ( + std::mem::replace(&mut self.certain, true), + std::mem::replace(&mut self.complete, true), + std::mem::replace(&mut self.position_dependent, false), + std::mem::replace(&mut self.reused, false), + std::mem::replace(&mut self.deepest, 0), + ); let kind = self.kind_of(expr, def.scope.unwrap_or(scope), depth + 1); let one_cell_range = std::mem::replace(&mut self.one_cell_range, seen); + self.read.push((key.clone(), kind, one_cell_range)); + let cacheable = self.complete && !self.position_dependent && !self.reused; + let info = cacheable.then(|| NameInfo { + kind, + one_cell_range, + certain: self.certain, + cost: before.saturating_sub(self.remaining), + depth: self.deepest.saturating_sub(depth + 1), + closure: self.read[first_read..].to_vec(), + }); + self.certain &= outer.0; + self.complete &= outer.1; + self.position_dependent |= outer.2; + self.reused |= outer.3; + self.deepest = self.deepest.max(outer.4); self.one_cell_range |= one_cell_range; self.locals = locals; + if let Some(info) = info.filter(|_| self.shared.map.len() < MAX_CACHED_NAMES) { + self.shared.map.insert(key.clone(), info); + } self.names.insert(key, Some((kind, one_cell_range))); return kind; } diff --git a/crates/xlsx/src/lib.rs b/crates/xlsx/src/lib.rs index 259a1a1a6..773e2895f 100644 --- a/crates/xlsx/src/lib.rs +++ b/crates/xlsx/src/lib.rs @@ -52,7 +52,7 @@ pub use csv::{MAX_CSV_BYTES, decode_text, read_csv, read_csv_with_report, write_ pub use ods::read_ods; pub use package::MAX_PART; pub use passthrough::rebase_links; -pub use xls::read_xls; +pub use xls::{read_xls, read_xls_with}; pub use xlsb::read_xlsb; /// Errors from reading or writing files. diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index d3ba1426b..c405667b1 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -652,7 +652,7 @@ fn write_sheet_data(s: &mut String, data: SheetCellData<'_>, sid: &dyn Fn(gridcr if f.dynamic { return Some(RangeRef::cell(cell_ref)); } - match crate::array_formula::export(workbook, index, cell_ref, f, out.functions) { + match crate::array_formula::export(workbook, index, cell_ref, f, out.functions, &mut out.names) { crate::array_formula::Export::Unchanged => None, crate::array_formula::Export::Dynamic => Some(RangeRef::cell(cell_ref)), // Every `@` is one Excel implies in a legacy formula: write it without them. diff --git a/crates/xlsx/src/tests/array_export.rs b/crates/xlsx/src/tests/array_export.rs index 9c7f0700a..9e60f0373 100644 --- a/crates/xlsx/src/tests/array_export.rs +++ b/crates/xlsx/src/tests/array_export.rs @@ -467,6 +467,138 @@ fn implicit_intersection_metadata_distinguishes_table_rows_and_columns() { ); } +/// How `export` saved a formula, as comparable text. +fn export_text(wb: &Workbook, at: CellRef, text: &str, names: &mut crate::array_formula::NameCache) -> String { + use crate::array_formula::{Export, export}; + match export(wb, 0, at, &Formula::new(text), crate::FunctionInfo::default(), names) { + Export::Unchanged => "unchanged".into(), + Export::Dynamic => "dynamic".into(), + Export::Legacy(e) => format!("legacy {}", gridcraft_formula::print(&e)), + } +} + +#[test] +fn defined_names_are_analysed_once_for_many_formulas() { + use crate::array_formula::NameCache; + let mut wb = Workbook::new(); + // A large definition: many operations, ending in a range. + let big = (1..=500).map(|i| format!("Sheet1!$A${i}")).collect::>().join("+"); + for (name, text) in [("Big", format!("({big})*Sheet1!$B$1:$B$3")), ("Rng", "Sheet1!$A$1:$A$3".to_string()), ("Alias", "Rng".to_string())] { + wb.names.push(DefinedName { name: name.into(), scope: None, formula: text, comment: String::new(), hidden: false }); + } + let texts = ["SUM(Big)", "SQRT(Big)", "Big*2", "SQRT(Alias)", "SUM(Alias)", "Alias", "SUM(Rng*2)", "Missing*2"]; + let mut shared = NameCache::default(); + for i in 0..400u32 { + for text in texts { + let at = CellRef::new(i + 10, 5); + let with_shared = export_text(&wb, at, text, &mut shared); + let fresh = export_text(&wb, at, text, &mut NameCache::default()); + assert_eq!(with_shared, fresh, "{text}"); + } + } + assert!(gridcraft_formula::parse(&wb.names[0].formula).is_ok(), "the large name must parse"); + // 3 names, each analysed once for all 3200 formulas (a fresh cache analyses a name per formula). + assert_eq!(shared.analyses, 3); +} + +#[test] +fn shared_name_analysis_keeps_each_formulas_limits_and_position() { + use crate::array_formula::NameCache; + let mut wb = Workbook::new(); + let terms = |n: usize| (1..=n).map(|i| format!("Sheet1!$A${i}")).collect::>().join("+"); + // Twenty names of about a thousand nodes each: a formula using all of them needs more than + // its 16,384-node budget, one using a few does not. + for i in 0..20 { + let text = format!("({})*Sheet1!$B$1:$B$3", terms(500)); + wb.names.push(DefinedName { name: format!("Big_{i}"), scope: None, formula: text, comment: String::new(), hidden: false }); + } + for (name, text) in [ + ("Cyc1", "Cyc2".to_string()), + ("Cyc2", "Cyc1".to_string()), + ("Deep", format!("{}1{}", "(".repeat(100), ")".repeat(100))), + ("Tbl", "Table1[Col]".to_string()), + ] { + wb.names.push(DefinedName { name: name.into(), scope: None, formula: text, comment: String::new(), hidden: false }); + } + assert!(wb.names.iter().all(|n| gridcraft_formula::parse(&n.formula).is_ok()), "every name must parse"); + let uses = |n: usize| (0..n).map(|i| format!("SQRT(Big_{i})")).collect::>().join("+"); + let mut texts: Vec = [1, 5, 10, 15, 17, 20].iter().map(|n| uses(*n)).collect(); + texts.extend(["SQRT(Big_3)", "Cyc1", "SUM(Cyc1)", "SUM(Deep)", "SQRT(Tbl)", "((((((((SQRT(Deep))))))))"].map(String::from)); + // The limit is really reached: a few uses analyse fully, all twenty stop at the budget. + let probe = |n: usize| export_text(&wb, CellRef::new(5, 5), &uses(n), &mut NameCache::default()); + assert_eq!(probe(1), "dynamic"); + assert_eq!(probe(20), "unchanged"); + let mut shared = NameCache::default(); + for round in 0..3 { + for text in &texts { + for at in [CellRef::new(5, 5), CellRef::new(7, 9)] { + let with_shared = export_text(&wb, at, text, &mut shared); + let fresh = export_text(&wb, at, text, &mut NameCache::default()); + assert_eq!(with_shared, fresh, "round {round}: {}", &text[..text.len().min(60)]); + } + } + } +} + +/// A name defined as another name reads the same from the shared cache in either order: after +/// the other name (whose cost and uncertainty the formula already paid) and before it (a hit +/// records the other name too, so the formula does not pay for it twice). +#[test] +fn shared_name_analysis_of_names_defined_by_names_matches_fresh_in_both_orders() { + use crate::array_formula::NameCache; + let mut wb = Workbook::new(); + let terms = (1..=500).map(|i| format!("Sheet1!$A${i}")).collect::>().join("+"); + // `Unsure` is outside the established legacy semantics (ISOMITTED), and `Alias` is it by name. + let mut defs = vec![ + ("Unsure".to_string(), "IF(ISOMITTED(Sheet1!$C$1),Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)".to_string()), + ("Alias".to_string(), "Unsure".to_string()), + ]; + for i in 0..10 { + defs.push((format!("Big_{i}"), format!("({terms})*Sheet1!$B$1:$B$3"))); + defs.push((format!("Via_{i}"), format!("Big_{i}"))); + } + for (name, formula) in defs { + wb.names.push(DefinedName { name, scope: None, formula, comment: String::new(), hidden: false }); + } + // Ten names of about a thousand nodes, each reached twice: within the budget only if the + // second reference is free. + let both = |first: &str, second: &str| (0..10).map(|i| format!("SQRT({first}_{i})+SQRT({second}_{i})")).collect::>().join("+"); + let orders = [ + ["Unsure+SQRT(Alias)", "SQRT(Alias)+Unsure", "SQRT(Alias)+SQRT(Sheet1!$A$1:$A$3)", "@SQRT(Alias)"].map(String::from).to_vec(), + ["SQRT(Alias)", "SQRT(Alias)+Unsure", "Unsure+SQRT(Alias)", "SQRT(Alias)+SQRT(Sheet1!$A$1:$A$3)", "@SQRT(Alias)"].map(String::from).to_vec(), + vec![both("Big", "Via"), both("Via", "Big"), "SQRT(Via_0)".to_string(), both("Via", "Big")], + vec!["SQRT(Via_0)".to_string(), both("Via", "Big"), both("Big", "Via"), both("Via", "Big")], + ]; + let at = CellRef::new(5, 5); + assert_eq!(export_text(&wb, at, &both("Via", "Big"), &mut NameCache::default()), "dynamic", "fresh fits the budget"); + for texts in orders { + let mut shared = NameCache::default(); + for text in &texts { + let with_shared = export_text(&wb, at, text, &mut shared); + let fresh = export_text(&wb, at, text, &mut NameCache::default()); + assert_eq!(with_shared, fresh, "{}", &text[..text.len().min(60)]); + } + // Reading: one pass over the formulas, one row each, against each formula read alone. + let read = |cells: &[(CellRef, &String)], one_pass: bool| -> Vec { + let mut wb = wb.clone(); + for (c, text) in cells { + Arc::make_mut(&mut wb.sheets[0]).cells.set(*c, Cell::formula(Formula::new(text))); + } + let all: Vec = cells.iter().map(|(c, _)| *c).collect(); + if one_pass { + crate::array_formula::apply_legacy(&mut wb, vec![(0, all.clone())], crate::FunctionInfo::default()); + } else { + for c in &all { + crate::array_formula::apply_legacy(&mut wb, vec![(0, vec![*c])], crate::FunctionInfo::default()); + } + } + all.iter().map(|c| gridcraft_formula::print(&wb.sheets[0].cells.get(*c).unwrap().formula.as_ref().unwrap().expr_arc().unwrap())).collect() + }; + let cells: Vec<(CellRef, &String)> = texts.iter().enumerate().map(|(i, t)| (CellRef::new(i as u32 + 5, 5), t)).collect(); + assert_eq!(read(&cells, true), read(&cells, false)); + } +} + #[test] fn long_flat_chains_are_scalar_formulas() { // `=A1+A2+…` nests one level per operator. Past the scan's depth limit of 64, a plain sum diff --git a/crates/xlsx/src/tests/xls_tests.rs b/crates/xlsx/src/tests/xls_tests.rs index bd21b4939..762e512e0 100644 --- a/crates/xlsx/src/tests/xls_tests.rs +++ b/crates/xlsx/src/tests/xls_tests.rs @@ -508,8 +508,11 @@ fn sheets_share_one_record_budget() { stream.extend(eof()); let file = compound("Workbook", &stream); // About n × 2,000 records in all: over a 100,000 shared allowance, though each sheet alone fits. - assert!(matches!(crate::xls::read_xls_with(&file, crate::xls::Budget::with_max_records(100_000)), Err(IoError::TooLarge(_)))); - assert!(crate::xls::read_xls_with(&file, crate::xls::Budget::with_max_records(10_000_000)).is_ok()); + assert!(matches!( + crate::xls::read_xls_budgeted(&file, crate::xls::Budget::with_max_records(100_000), crate::FunctionInfo::default()), + Err(IoError::TooLarge(_)) + )); + assert!(crate::xls::read_xls_budgeted(&file, crate::xls::Budget::with_max_records(10_000_000), crate::FunctionInfo::default()).is_ok()); // Two sheets at the same offset are refused outright. let mut dup = bof(0x0005); @@ -625,6 +628,112 @@ fn a_format_shorter_than_its_count_keeps_its_characters() { assert!(report.warnings.iter().any(|w| w.contains("declared length")), "{:?}", report.warnings); } +/// A workbook whose "Report" sheet holds legacy formulas over ranges of "Data": Data!A1 = 11, +/// B1:G1 = 1, A2 = 5, A3 = 7, and Report!B1:G1 filled, so a formula in Report!A1 that spilled +/// would be blocked. +fn legacy_formulas_book() -> Vec { + let area3d = |r0: u16, r1: u16, c0: u16, c1: u16| cat(&[&[0x3B], &0u16.to_le_bytes(), &u16s(&[r0, r1, c0, c1])]); + let a2a3 = area3d(1, 2, 0, 0); + let sum = |inner: &[u8]| cat(&[inner, &[0x22, 1], &4u16.to_le_bytes()]); + let boundsheet = |pos: u32, name: &str| rec(0x0085, &cat(&[&pos.to_le_bytes(), &[0, 0, name.len() as u8, 0], name.as_bytes()])); + let globals = |p1: u32, p2: u32| { + cat(&[ + &bof(0x0005), + &boundsheet(p1, "Data"), + &boundsheet(p2, "Report"), + &rec(0x01AE, &u16s(&[2, 0x0401])), + &rec(0x0017, &cat(&[&u16s(&[1]), &u16s(&[0, 0, 0])])), + &eof(), + ]) + }; + let number = |r: u16, c: u16, n: f64| rec(0x0203, &cat(&[&cell(r, c, 0), &n.to_le_bytes()])); + let mut data = bof(0x0010); + data.extend(number(0, 0, 11.0)); + for c in 1..7 { + data.extend(number(0, c, 1.0)); + } + data.extend(number(1, 0, 5.0)); + data.extend(number(2, 0, 7.0)); + data.extend(eof()); + let shared = cat(&[&a2a3, &ptg_int(10), &[0x05]]); + let array = cat(&[&a2a3, &ptg_int(2), &[0x05]]); + let single_array = sum(&cat(&[&a2a3, &ptg_int(2), &[0x05]])); + let mut report = bof(0x0010); + for c in 1..7 { + report.extend(number(0, c, 0.0)); + } + report.extend(cat(&[ + // A1 = Data!A1:G1, A2 = +Data!A1:G1 (a title row repeated from another sheet). + &formula(0, 0, num(11.0), &area3d(0, 0, 0xC000, 6 | 0xC000)), + &formula(1, 0, num(11.0), &cat(&[&area3d(0, 0, 0xC000, 6 | 0xC000), &[0x12]])), + // C2:C3 share Data!$A$2:$A$3*10. + &formula(1, 2, num(50.0), &exp(1, 2)), + &rec(0x04BC, &cat(&[&u16s(&[1, 2]), &[2, 2, 0, 2], &(shared.len() as u16).to_le_bytes(), &shared])), + &formula(2, 2, num(70.0), &exp(1, 2)), + // D2:D3 = {Data!$A$2:$A$3*2}, entered with Ctrl+Shift+Enter. + &formula(1, 3, num(10.0), &exp(1, 3)), + &rec(0x0221, &cat(&[&u16s(&[1, 2]), &[3, 3], &[0; 6], &(array.len() as u16).to_le_bytes(), &array])), + &formula(2, 3, num(14.0), &exp(1, 3)), + // H1 = {SUM(Data!$A$2:$A$3*2)}, a one-cell Ctrl+Shift+Enter array. + &formula(0, 7, num(24.0), &exp(0, 7)), + &rec(0x0221, &cat(&[&u16s(&[0, 0]), &[7, 7], &[0; 6], &(single_array.len() as u16).to_le_bytes(), &single_array])), + // H2 = MAX(SUM(Data!$A$2:$A$3/12),0), H3 = SUM(Data!$A$2:$A$3), H4 = SQRT(Data!$A$2:$A$3). + &formula(1, 7, num(5.0 / 12.0), &cat(&[&sum(&cat(&[&a2a3, &ptg_int(12), &[0x06]])), &ptg_int(0), &[0x22, 2], &7u16.to_le_bytes()])), + &formula(2, 7, num(12.0), &sum(&a2a3)), + &formula(3, 7, special(2, 0x0F), &cat(&[&a2a3, &[0x21], &20u16.to_le_bytes()])), + &eof(), + ])); + let p1 = globals(0, 0).len() as u32; + let p2 = p1 + data.len() as u32; + cat(&[&globals(p1, p2), &data, &report]) +} + +#[test] +fn legacy_formulas_read_with_implicit_intersection() { + let bytes = compound("Workbook", &legacy_formulas_book()); + let formula = |wb: &gridcraft_model::Workbook, a1: &str| { + let f = wb.sheets[1].cell(CellRef::parse(a1).unwrap()).and_then(|c| c.formula.clone()).unwrap(); + (f.text.clone(), f.array) + }; + let range = |a1: &str| RangeRef::parse(a1); + let (wb, _) = read_xls(&bytes).unwrap(); + // Every .xls formula is legacy: a range where one value is expected is intersected. + assert_eq!(formula(&wb, "A1"), ("@Data!A1:G1".into(), None)); + assert_eq!(formula(&wb, "A2"), ("+@Data!A1:G1".into(), None)); + // Shared formulas too, each cell on its own. + assert_eq!(formula(&wb, "C2"), ("@Data!$A$2:$A$3*10".into(), None)); + assert_eq!(formula(&wb, "C3"), ("@Data!$A$2:$A$3*10".into(), None)); + // Ctrl+Shift+Enter arrays, one cell or several, stay arrays and are not intersected. + assert_eq!(formula(&wb, "D2"), ("Data!$A$2:$A$3*2".into(), range("D2:D3"))); + assert_eq!(formula(&wb, "H1"), ("SUM(Data!$A$2:$A$3*2)".into(), range("H1"))); + // An operation inside SUM takes values; SUM of the range itself is left alone. + assert_eq!(formula(&wb, "H2"), ("MAX(SUM(@Data!$A$2:$A$3/12),0)".into(), None)); + assert_eq!(formula(&wb, "H3"), ("SUM(Data!$A$2:$A$3)".into(), None)); + assert_eq!(formula(&wb, "H4"), ("SQRT(@Data!$A$2:$A$3)".into(), None)); + + // The caller's function facts decide: here SQRT takes its argument as a whole. + let library = crate::FunctionInfo { is_builtin: |_| true, is_scalar_arg: |n, _| n != "SQRT" }; + let (custom, _) = crate::read_xls_with(&bytes, library).unwrap(); + assert_eq!(formula(&custom, "H4"), ("SQRT(Data!$A$2:$A$3)".into(), None)); + assert_eq!(formula(&custom, "A1"), ("@Data!A1:G1".into(), None)); + + // Saved as .xlsx, they stay legacy: no dynamic-array metadata, no _xlfn.SINGLE, and they read + // back as they were opened. + let saved = write_xlsx(&wb).unwrap(); + let mut sheet = String::new(); + std::io::Read::read_to_string( + &mut zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap().by_name("xl/worksheets/sheet2.xml").unwrap(), + &mut sheet, + ) + .unwrap(); + assert!(!sheet.contains("cm=") && !sheet.contains("SINGLE"), "{sheet}"); + assert!(sheet.contains("Data!A1:G1") && sheet.contains("MAX(SUM(Data!$A$2:$A$3/12),0)"), "{sheet}"); + let (back, _) = read_xlsx(&saved).unwrap(); + for a1 in ["A1", "A2", "C2", "C3", "D2", "H1", "H2", "H3", "H4"] { + assert_eq!(formula(&back, a1), formula(&wb, a1), "{a1}"); + } +} + fn one_sheet_with(sheet_records: &[u8]) -> Vec { let globals = |pos: u32| cat(&[&bof(0x0005), &rec(0x0085, &cat(&[&pos.to_le_bytes(), &[0, 0, 6, 0], b"Sheet1"])), &eof()]); let sheet = cat(&[&bof(0x0010), sheet_records, &rec(0x0203, &cat(&[&cell(0, 0, 0), &7.0f64.to_le_bytes()])), &eof()]); diff --git a/crates/xlsx/src/write.rs b/crates/xlsx/src/write.rs index 5d06d54a0..512144980 100644 --- a/crates/xlsx/src/write.rs +++ b/crates/xlsx/src/write.rs @@ -134,6 +134,8 @@ pub struct Out { pub kept: crate::passthrough::KeptOut, /// Decides which formulas need dynamic-array metadata. pub functions: FunctionInfo, + /// Defined names analysed for implicit intersection, shared by every formula written. + pub(crate) names: crate::array_formula::NameCache, /// The code of the Python cells written so far, in order: script `i` of the Python part. pub python_scripts: Vec, } @@ -271,6 +273,7 @@ pub fn write_xlsx_named_with(wb: &Workbook, kind: XlsxKind, book_name: Option<&s cell_pictures: Default::default(), kept: Default::default(), functions, + names: Default::default(), python_scripts: vec![], }; out.kept.reserve_vml_blocks(wb); diff --git a/crates/xlsx/src/xls.rs b/crates/xlsx/src/xls.rs index 89f4a31f7..1c9285bae 100644 --- a/crates/xlsx/src/xls.rs +++ b/crates/xlsx/src/xls.rs @@ -10,6 +10,9 @@ //! (charts, pictures, comments, conditional formats, validation, macros) is skipped with a //! warning. //! Encrypted workbooks and pre-BIFF8 versions are refused. +//! +//! Every formula in the file is a legacy (pre-dynamic-array) one, so it is read with Excel's +//! implicit `@` (see `array_formula.rs`), like an `.xlsx` formula saved without `cm`. mod formula; mod ftab; @@ -24,7 +27,7 @@ use gridcraft_model::{CalcMode, CalcSettings, Cell, DefinedName, Formula, LineIn use crate::cfb::Compound; use crate::tables::{col_width_to_px, pt_to_px}; -use crate::{IoError, ReadReport}; +use crate::{FunctionInfo, IoError, ReadReport}; use formula::{Links, Mode, SupBook, Unsupported, Xti, cell_error, decode}; use records::{Cursor, Record, Records}; @@ -141,13 +144,20 @@ struct Globals { precision_as_displayed: bool, } -/// Reads an Excel 97-2003 workbook. Inspect the report's warnings for what was skipped. +/// Reads an Excel 97-2003 workbook, with the default [`FunctionInfo`]. Inspect the report's +/// warnings for what was skipped. pub fn read_xls(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { - read_xls_with(bytes, Budget::default()) + read_xls_budgeted(bytes, Budget::default(), FunctionInfo::default()) } -/// [`read_xls`] with an explicit resource budget. -pub(crate) fn read_xls_with(bytes: &[u8], mut budget: Budget) -> Result<(Workbook, ReadReport), IoError> { +/// [`read_xls`] with the function library's facts, which decide where the legacy formulas get +/// Excel's implicit `@`. +pub fn read_xls_with(bytes: &[u8], functions: FunctionInfo) -> Result<(Workbook, ReadReport), IoError> { + read_xls_budgeted(bytes, Budget::default(), functions) +} + +/// [`read_xls_with`] with an explicit resource budget. +pub(crate) fn read_xls_budgeted(bytes: &[u8], mut budget: Budget, functions: FunctionInfo) -> Result<(Workbook, ReadReport), IoError> { let cf = Compound::open(bytes).map_err(invalid)?; let stream = match cf.root_stream("Workbook").map_err(invalid)? { Some(s) => s, @@ -201,6 +211,10 @@ pub(crate) fn read_xls_with(bytes: &[u8], mut budget: Budget) -> Result<(Workboo } wb.calc.precision_as_displayed = g.precision_as_displayed; defined_names(&g, &model_index, &mut wb, &mut report); + // Every .xls formula is a legacy (pre-dynamic-array) one: read with Excel's implicit `@`, like + // an .xlsx formula without `cm`. Ctrl+Shift+Enter arrays are read as they are. + let legacy = wb.sheets.iter().enumerate().map(|(si, s)| (si, legacy_cells(s))).collect(); + crate::array_formula::apply_legacy(&mut wb, legacy, functions); wb.set_active_sheet(wb.sheets.iter().position(|s| s.visibility == Visibility::Visible).unwrap_or(0)); // The overview comes first. report.warnings.insert( @@ -210,6 +224,11 @@ pub(crate) fn read_xls_with(bytes: &[u8], mut budget: Budget) -> Result<(Workboo Ok((wb, report)) } +/// The cells holding a formula that is not a Ctrl+Shift+Enter array. +fn legacy_cells(sheet: &Sheet) -> Vec { + sheet.cells.iter().filter(|(_, c)| c.formula.as_ref().is_some_and(|f| f.array.is_none())).map(|(at, _)| at).collect() +} + fn bof(rec: &Record<'_>) -> Result<(u16, u16), IoError> { if rec.id != BOF { return Err(invalid("a substream does not start with a BOF record")); @@ -463,7 +482,9 @@ impl SheetState<'_> { self.unsupported("syntax GridCraft can't parse yet"); return; } - f.array = array.filter(|r| !r.is_single()); + // A one-cell Ctrl+Shift+Enter array keeps its range, as when an .xlsx is read: + // it evaluates as an array, not with implicit intersection. + f.array = array; if let Some(cell) = self.sheet.cells.get_mut(pos) { cell.formula = Some(Arc::new(f)); } else { diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 12597f313..40942a539 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -92,7 +92,7 @@ past the parser's limit, and a wrapper inside the text. **Saving writes the real | `.xlsm` Macro-Enabled Workbook | R/W | yes | yes, macros kept, not run | The VBA project is kept byte-for-byte and written back only into `.xlsm`/`.xltm` (#6); macros never run. Saved with the macro-enabled workbook content type | `roundtrip.rs` (`vba_project_roundtrip`, `package_kind_sets_workbook_content_type`), engine `io.rs` (`macros_are_saved_only_into_xlsm`) | | `.xltx` / `.xltm` templates | R/W | yes | partial | Saved with the template (and macro-enabled template) content type, chosen by extension; `.xltm` keeps a VBA project, `.xltx` drops it; "new from template" behaviour missing | `roundtrip.rs` (`package_kind_sets_workbook_content_type`), engine `io.rs` | | `.xlsb` Binary Workbook | R/W | **values only** (#101) | no | Names, visibility, numbers, text, booleans, errors, merges, cached formula results; no formulas, formatting, charts, names | `xlsb_tests.rs` | -| `.xls` Excel 97-2003 (BIFF8) | R/W | **yes, partial** (#121) | no (Save As `.xlsx`) | Clean-room from [MS-CFB] and [MS-XLS]. Read: worksheets and visibility, shared strings, numbers, text, booleans, errors, formulas (text for the common tokens, shared and array formulas, 3-D refs, names, newer functions called by name) with cached results, number formats (dates display as dates), cell formatting (fonts, solid and pattern fills, borders with diagonals, alignment with rotation, indent, wrap and shrink, locked and hidden, the custom colour palette, the Normal font as the workbook default, user-defined cell styles), 1904 date system, merges, column widths, row heights, defined names, print area and titles, calculation settings (mode, iteration with its maximum iterations and change, precision as displayed), sheet display options (right-to-left, gridlines, headings, zero values, formulas; a truncated WINDOW2 is ignored). Not read: exact colours beyond the 56-colour palette (XFExt: theme colours and tints show as the palette colour Excel stored next to them), rich-text runs inside a cell, charts, pictures, comments, CF, validation, hyperlinks, freeze panes, external links (formula kept as its value). Chart and macro sheets skipped with a warning; encrypted (FILEPASS) and BIFF5 (Excel 5.0/95) files refused with a message. Opens as a new unsaved workbook. Unverified on a real-world corpus | `xls_tests.rs`, `xls_style_tests.rs` | +| `.xls` Excel 97-2003 (BIFF8) | R/W | **yes, partial** (#121) | no (Save As `.xlsx`) | Clean-room from [MS-CFB] and [MS-XLS]. Read: worksheets and visibility, shared strings, numbers, text, booleans, errors, formulas (text for the common tokens, shared and array formulas, 3-D refs, names, newer functions called by name) with cached results, number formats (dates display as dates), cell formatting (fonts, solid and pattern fills, borders with diagonals, alignment with rotation, indent, wrap and shrink, locked and hidden, the custom colour palette, the Normal font as the workbook default, user-defined cell styles), 1904 date system, merges, column widths, row heights, defined names, print area and titles, calculation settings (mode, iteration with its maximum iterations and change, precision as displayed), sheet display options (right-to-left, gridlines, headings, zero values, formulas; a truncated WINDOW2 is ignored). Every formula except a Ctrl+Shift+Enter array is a legacy one and opens with Excel's implicit `@`, as an `.xlsx` formula without `cm` does (#410); Ctrl+Shift+Enter arrays, one cell or several, stay arrays, and saving as `.xlsx` writes the formulas without `cm`. Not read: exact colours beyond the 56-colour palette (XFExt: theme colours and tints show as the palette colour Excel stored next to them), rich-text runs inside a cell, charts, pictures, comments, CF, validation, hyperlinks, freeze panes, external links (formula kept as its value). Chart and macro sheets skipped with a warning; encrypted (FILEPASS) and BIFF5 (Excel 5.0/95) files refused with a message. Opens as a new unsaved workbook. Unverified on a real-world corpus | `xls_tests.rs`, `xls_style_tests.rs` | | `.xlt` Excel 97-2003 template | R/W | yes (read as `.xls`) | no | | — | | `.xlam` / `.xla` add-ins | R/W | no | no | VBA; out of scope | — | | `.xml` XML Spreadsheet 2003 | R/W | no | no | Simple to support | — | @@ -161,6 +161,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | Date | Change | Summary | |---|---|---| +| 2026-11-10 | minor | `.xls` import: every formula except a Ctrl+Shift+Enter array is read as a legacy one with Excel's implicit `@` (a title row `=Other!A1:G1` shows its first cell instead of `#SPILL!`), one-cell Ctrl+Shift+Enter arrays stay arrays, and `@` of a 2-D range meets the formula's row and column (#410, PR #1367) | | 2026-11-02 | minor | XLSX: buttons, scroll bars and spin buttons read and written (control properties parts, ``, VML `x:ClientData`); other form controls stay with pass-through; a button's command, fixed input or output cell travels in a hidden defined name and comes back only while the file still assigns the button's macro (or none, for a command) | | 2026-11-03 | minor | SYLK (`.slk`) import: values, R1C1 formulas as A1 (shared formulas too) with cached results, number formats and column widths; import only (#1481) | | 2026-11-03 | minor | Opening a web page (HTML) reads its first table as a sheet, whatever the file is called: a web app's "Export to Excel" saved as `.xls` no longer opens as lines of markup. Values read as in a CSV (or as text with `csvAsText`), `` cells are bold and centred, `` names the sheet, and a page not named `.htm`/`.html` opens with a warning (#3166) | From 457bc7e374afbcd9a99c4f4c52c0017149ec8bb9 Mon Sep 17 00:00:00 2001 From: FredApps <153173854+FredApps@users.noreply.github.com> Date: Tue, 10 Nov 2026 16:13:13 +0100 Subject: [PATCH 5/6] Naming: use Diagram and Decorative Text in the feature catalog (#3622) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Naming: use Diagram and Decorative Text in the feature catalog * Naming: align the Chinese decorative text term * test(naming): guard neutral display labels * Naming: Diagram no longer repeats the chart word in de, uk and ro German Schema and Ukrainian Схема match craft-i18n#211; Romanian Schemă avoids GridCraft's own chart word Diagramă. A catalog test checks every shipped language translates both labels and that Diagram differs from Chart/Charts. The display-name guard now handles escaped char literals and any #[cfg(test)] item, asserts it scanned catalog.rs and a minimum of files and literals, and tests word boundaries. The catalog test asserts each ID appears once. Docs carry timestamps and revision history. --------- Co-authored-by: FredApps --- crates/engine/src/catalog.rs | 16 +- crates/ui-egui/src/i18n.rs | 27 +++- crates/ui-egui/src/i18n/cs.rs | 2 + crates/ui-egui/src/i18n/de.rs | 2 + crates/ui-egui/src/i18n/el.rs | 2 + crates/ui-egui/src/i18n/es.rs | 2 + crates/ui-egui/src/i18n/fr.rs | 2 + crates/ui-egui/src/i18n/it.rs | 2 + crates/ui-egui/src/i18n/ja.rs | 2 + crates/ui-egui/src/i18n/ko.rs | 2 + crates/ui-egui/src/i18n/nl.rs | 2 + crates/ui-egui/src/i18n/pl.rs | 2 + crates/ui-egui/src/i18n/pt.rs | 2 + crates/ui-egui/src/i18n/ro.rs | 2 + crates/ui-egui/src/i18n/ru.rs | 2 + crates/ui-egui/src/i18n/sr.rs | 2 + crates/ui-egui/src/i18n/tr.rs | 2 + crates/ui-egui/src/i18n/uk.rs | 2 + crates/ui-egui/src/i18n/vi.rs | 2 + crates/ui-egui/src/i18n/zh.rs | 2 + crates/ui-egui/tests/display_names.rs | 215 ++++++++++++++++++++++++++ docs/feature-names.md | 28 ++++ docs/localization-parity.md | 3 +- docs/parity-checklist.md | 4 +- 24 files changed, 321 insertions(+), 8 deletions(-) create mode 100644 crates/ui-egui/tests/display_names.rs create mode 100644 docs/feature-names.md diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index 234e92e58..41d1157f7 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -118,7 +118,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("insert.picture", "Insert", "Illustrations", "Pictures"), ("insert.shape", "Insert", "Illustrations", "Shapes"), ("insert.icons", "Insert", "Illustrations", "Icons"), - ("insert.smartArt", "Insert", "Illustrations", "SmartArt"), + ("insert.smartArt", "Insert", "Illustrations", "Diagram"), ("insert.screenshot", "Insert", "Illustrations", "Screenshot"), ("insert.recommendedCharts", "Insert", "Charts", "Recommended Charts"), ("insert.chart", "Insert", "Charts", "Charts"), @@ -131,7 +131,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("review.newComment", "Insert", "Comments", "Comment"), ("insert.textBox", "Insert", "Text", "Text Box"), ("pageLayout.headerFooter", "Insert", "Text", "Header & Footer"), - ("insert.wordArt", "Insert", "Text", "WordArt"), + ("insert.wordArt", "Insert", "Text", "Decorative Text"), ("insert.signatureLine", "Insert", "Text", "Signature Line"), ("insert.object", "Insert", "Text", "Object"), ("insert.equation", "Insert", "Symbols", "Equation"), @@ -340,3 +340,15 @@ pub fn parity() -> (usize, usize) { let done = ids.iter().filter(|id| crate::find_command(id).is_some()).count(); (done, ids.len()) } + +#[cfg(test)] +mod feature_name_tests { + #[test] + fn own_feature_labels_keep_command_ids() { + for (id, expected) in [("insert.smartArt", "Diagram"), ("insert.wordArt", "Decorative Text")] { + let entries: Vec<_> = super::CATALOG.iter().filter(|entry| entry.0 == id).collect(); + assert_eq!(entries.len(), 1, "{id} appears once"); + assert_eq!(entries[0].3, expected); + } + } +} diff --git a/crates/ui-egui/src/i18n.rs b/crates/ui-egui/src/i18n.rs index 45c336f87..3aac97d6e 100644 --- a/crates/ui-egui/src/i18n.rs +++ b/crates/ui-egui/src/i18n.rs @@ -194,7 +194,12 @@ impl Language { /// A label that needs two translations (Home's text "Orientation" vs Page Layout's print /// "Orientation") is keyed `context|label`: English shows only the part after the `|`. pub fn tr(self, text: &str) -> &str { - let translated = match self { + self.translated(text).unwrap_or_else(|| text.rsplit_once('|').map_or(text, |(_, label)| label)) + } + + /// The catalog's own translation of `text`, or `None` when it falls back to English. + fn translated(self, text: &str) -> Option<&'static str> { + match self { Self::En => None, Self::Es => es::translate(text), Self::El => el::translate(text), @@ -214,8 +219,7 @@ impl Language { Self::Vi => vi::translate(text), Self::Nl => nl::translate(text), Self::Ro => ro::translate(text), - }; - translated.unwrap_or_else(|| text.rsplit_once('|').map_or(text, |(_, label)| label)) + } } pub(crate) fn tr_error(self, message: &str) -> String { @@ -321,6 +325,23 @@ fn name_sort_key(name: &str) -> String { mod tests { use super::*; + /// Every shipped catalog names GridCraft's own Diagram and Decorative Text features, and + /// Diagram never repeats that catalog's chart word (both sit on the Insert tab). + #[test] + fn every_catalog_has_distinct_diagram_and_decorative_text() { + for language in Language::ALL.into_iter().filter(|l| *l != Language::En) { + let code = language.code(); + let diagram = language.translated("Diagram"); + assert!(diagram.is_some(), "{code}: Diagram"); + assert!(language.translated("Decorative Text").is_some(), "{code}: Decorative Text"); + for chart in ["Chart", "Charts"] { + if let Some(chart_text) = language.translated(chart) { + assert_ne!(diagram, Some(chart_text), "{code}: Diagram equals {chart}"); + } + } + } + } + /// Help tables are sorted and unique: lookups binary-search them. #[test] fn function_help_tables_are_sorted_and_unique() { diff --git a/crates/ui-egui/src/i18n/cs.rs b/crates/ui-egui/src/i18n/cs.rs index 39e6337dd..eaee085cf 100644 --- a/crates/ui-egui/src/i18n/cs.rs +++ b/crates/ui-egui/src/i18n/cs.rs @@ -3,6 +3,8 @@ /// Returns the Czech translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagram", + "Decorative Text" => "Ozdobný text", "Paste" => "Vložit", "Paste Values" => "Vložit hodnoty", "Paste Formulas" => "Vložit vzorce", diff --git a/crates/ui-egui/src/i18n/de.rs b/crates/ui-egui/src/i18n/de.rs index f136e483b..07fdb6710 100644 --- a/crates/ui-egui/src/i18n/de.rs +++ b/crates/ui-egui/src/i18n/de.rs @@ -4,6 +4,8 @@ /// Returns the German translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Schema", + "Decorative Text" => "Dekorativer Text", "Paste" => "Einfügen", "Paste Values" => "Werte einfügen", "Paste Formulas" => "Formeln einfügen", diff --git a/crates/ui-egui/src/i18n/el.rs b/crates/ui-egui/src/i18n/el.rs index dc57982c7..ec42b3473 100644 --- a/crates/ui-egui/src/i18n/el.rs +++ b/crates/ui-egui/src/i18n/el.rs @@ -5,6 +5,8 @@ /// Returns the Greek translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Διάγραμμα", + "Decorative Text" => "Διακοσμητικό κείμενο", "Paste" => "Επικόλληση", "Paste Values" => "Επικόλληση τιμών", "Paste Formulas" => "Επικόλληση τύπων", diff --git a/crates/ui-egui/src/i18n/es.rs b/crates/ui-egui/src/i18n/es.rs index 3c15a16b1..db78d8105 100644 --- a/crates/ui-egui/src/i18n/es.rs +++ b/crates/ui-egui/src/i18n/es.rs @@ -5,6 +5,8 @@ /// Returns the Spanish translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagrama", + "Decorative Text" => "Texto decorativo", "Paste" => "Pegar", "Paste Values" => "Pegar valores", "Paste Formulas" => "Pegar fórmulas", diff --git a/crates/ui-egui/src/i18n/fr.rs b/crates/ui-egui/src/i18n/fr.rs index 3e95d4b72..fd5e23b5b 100644 --- a/crates/ui-egui/src/i18n/fr.rs +++ b/crates/ui-egui/src/i18n/fr.rs @@ -3,6 +3,8 @@ /// Returns the French translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagramme", + "Decorative Text" => "Texte décoratif", "Different First Page" => "Première page différente", "Different Odd & Even Pages" => "Pages paires et impaires différentes", "Scale with Document" => "Mettre à l'échelle avec le document", diff --git a/crates/ui-egui/src/i18n/it.rs b/crates/ui-egui/src/i18n/it.rs index 3ad0ff328..dcede52cc 100644 --- a/crates/ui-egui/src/i18n/it.rs +++ b/crates/ui-egui/src/i18n/it.rs @@ -5,6 +5,8 @@ /// Returns the Italian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagramma", + "Decorative Text" => "Testo decorativo", "Paste" => "Incolla", "Paste Values" => "Incolla valori", "Paste Formulas" => "Incolla formule", diff --git a/crates/ui-egui/src/i18n/ja.rs b/crates/ui-egui/src/i18n/ja.rs index 19fc85f02..d558be6ec 100644 --- a/crates/ui-egui/src/i18n/ja.rs +++ b/crates/ui-egui/src/i18n/ja.rs @@ -8,6 +8,8 @@ /// English label → Japanese, the shared reference set. pub(super) const CORE: &[(&str, &str)] = &[ + ("Diagram", "図解"), + ("Decorative Text", "装飾文字"), ("Paste", "貼り付け"), ("Paste Values", "値の貼り付け"), ("Paste Formulas", "数式の貼り付け"), diff --git a/crates/ui-egui/src/i18n/ko.rs b/crates/ui-egui/src/i18n/ko.rs index 8c28be26e..ca2710272 100644 --- a/crates/ui-egui/src/i18n/ko.rs +++ b/crates/ui-egui/src/i18n/ko.rs @@ -3,6 +3,8 @@ /// Returns the Korean translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "다이어그램", + "Decorative Text" => "장식 텍스트", "Paste" => "붙여넣기", "Paste Values" => "값 붙여넣기", "Paste Formulas" => "수식 붙여넣기", diff --git a/crates/ui-egui/src/i18n/nl.rs b/crates/ui-egui/src/i18n/nl.rs index b5cddb88d..64c24b1f2 100644 --- a/crates/ui-egui/src/i18n/nl.rs +++ b/crates/ui-egui/src/i18n/nl.rs @@ -3,6 +3,8 @@ /// Returns the Dutch translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagram", + "Decorative Text" => "Decoratieve tekst", "Paste" => "Plakken", "Paste Values" => "Waarden plakken", "Paste Formulas" => "Formules plakken", diff --git a/crates/ui-egui/src/i18n/pl.rs b/crates/ui-egui/src/i18n/pl.rs index f2bfd72c7..29264ce91 100644 --- a/crates/ui-egui/src/i18n/pl.rs +++ b/crates/ui-egui/src/i18n/pl.rs @@ -1,6 +1,8 @@ /// Returns the Polish translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagram", + "Decorative Text" => "Tekst ozdobny", "Paste" => "Wklej", "Paste Values" => "Wklej wartości", "Paste Formulas" => "Wklej formuły", diff --git a/crates/ui-egui/src/i18n/pt.rs b/crates/ui-egui/src/i18n/pt.rs index d525f027e..3cc4faf76 100644 --- a/crates/ui-egui/src/i18n/pt.rs +++ b/crates/ui-egui/src/i18n/pt.rs @@ -4,6 +4,8 @@ /// Returns the Portuguese translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagrama", + "Decorative Text" => "Texto decorativo", "Paste" => "Colar", "Paste Values" => "Colar Valores", "Paste Formulas" => "Colar Fórmulas", diff --git a/crates/ui-egui/src/i18n/ro.rs b/crates/ui-egui/src/i18n/ro.rs index 66eb854b7..3d5a193d1 100644 --- a/crates/ui-egui/src/i18n/ro.rs +++ b/crates/ui-egui/src/i18n/ro.rs @@ -8,6 +8,8 @@ /// Returns the Romanian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Schemă", + "Decorative Text" => "Text decorativ", "Paste" => "Lipire", "Paste Values" => "Lipire valori", "Paste Formulas" => "Lipire formule", diff --git a/crates/ui-egui/src/i18n/ru.rs b/crates/ui-egui/src/i18n/ru.rs index 00a3062bd..2d4d3fe1f 100644 --- a/crates/ui-egui/src/i18n/ru.rs +++ b/crates/ui-egui/src/i18n/ru.rs @@ -3,6 +3,8 @@ /// Returns the Russian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Схема", + "Decorative Text" => "Декоративный текст", "Paste" => "Вставить", "Paste Values" => "Вставить значения", "Paste Formulas" => "Вставить формулы", diff --git a/crates/ui-egui/src/i18n/sr.rs b/crates/ui-egui/src/i18n/sr.rs index 9254dc938..a0eac0a4f 100644 --- a/crates/ui-egui/src/i18n/sr.rs +++ b/crates/ui-egui/src/i18n/sr.rs @@ -3,6 +3,8 @@ /// Returns the Serbian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Dijagram", + "Decorative Text" => "Ukrasni tekst", "Paste" => "Nalepi", "Paste Values" => "Nalepi vrednosti", "Paste Formulas" => "Nalepi formule", diff --git a/crates/ui-egui/src/i18n/tr.rs b/crates/ui-egui/src/i18n/tr.rs index c8da06cb1..ed3d0bf01 100644 --- a/crates/ui-egui/src/i18n/tr.rs +++ b/crates/ui-egui/src/i18n/tr.rs @@ -4,6 +4,8 @@ /// Returns the Turkish translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diyagram", + "Decorative Text" => "Dekoratif Metin", "Paste" => "Yapıştır", "Paste Values" => "Değerleri Yapıştır", "Paste Formulas" => "Formülleri Yapıştır", diff --git a/crates/ui-egui/src/i18n/uk.rs b/crates/ui-egui/src/i18n/uk.rs index 35fc9f0f7..ddb9368a0 100644 --- a/crates/ui-egui/src/i18n/uk.rs +++ b/crates/ui-egui/src/i18n/uk.rs @@ -3,6 +3,8 @@ /// Returns the Ukrainian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Схема", + "Decorative Text" => "Декоративний текст", "Paste" => "Вставити", "Paste Values" => "Вставити значення", "Paste Formulas" => "Вставити формули", diff --git a/crates/ui-egui/src/i18n/vi.rs b/crates/ui-egui/src/i18n/vi.rs index 8cb5a32ac..260f9572d 100644 --- a/crates/ui-egui/src/i18n/vi.rs +++ b/crates/ui-egui/src/i18n/vi.rs @@ -4,6 +4,8 @@ /// Returns the Vietnamese translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Sơ đồ", + "Decorative Text" => "Chữ trang trí", "Paste" => "Dán", "Paste Values" => "Dán giá trị", "Paste Formulas" => "Dán công thức", diff --git a/crates/ui-egui/src/i18n/zh.rs b/crates/ui-egui/src/i18n/zh.rs index e71a0df3f..ff671a7c9 100644 --- a/crates/ui-egui/src/i18n/zh.rs +++ b/crates/ui-egui/src/i18n/zh.rs @@ -3,6 +3,8 @@ /// Returns the Chinese translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "图示", + "Decorative Text" => "装饰文字", "Paste" => "粘贴", "Paste Values" => "粘贴值", "Paste Formulas" => "粘贴公式", diff --git a/crates/ui-egui/tests/display_names.rs b/crates/ui-egui/tests/display_names.rs new file mode 100644 index 000000000..250a4fd79 --- /dev/null +++ b/crates/ui-egui/tests/display_names.rs @@ -0,0 +1,215 @@ +//! Guard the production display surfaces without banning compatibility identifiers or notices. +use std::path::{Path, PathBuf}; + +const BANNED: &[&str] = &["SmartArt", "WordArt"]; +const ALLOWED: &[&str] = &[]; +const ROOTS: &[&str] = &["src", "../engine/src/catalog.rs"]; + +fn is_word(c: char) -> bool { + c.is_alphanumeric() || c == '_' +} + +fn banned_display(text: &str) -> bool { + if ALLOWED.contains(&text) { + return false; + } + BANNED.iter().any(|banned| { + text.match_indices(banned) + .any(|(at, _)| !text[..at].chars().next_back().is_some_and(is_word) && !text[at + banned.len()..].chars().next().is_some_and(is_word)) + }) +} + +// Read ordinary and raw Rust string literals, ignoring comments. No compiler dependency is +// needed, so this guard can also run by itself when a full application build is unavailable. +#[allow(clippy::cognitive_complexity)] // This small test-only lexer keeps raw-string/comment state together. +fn literals(source: &str) -> Vec { + let chars: Vec = source.chars().collect(); + let mut out = Vec::new(); + let mut at = 0; + let mut test_module = false; + let mut test_depth = 0_usize; + while at < chars.len() { + if chars.get(at..at + 12) == Some(&['#', '[', 'c', 'f', 'g', '(', 't', 'e', 's', 't', ')', ']']) { + at += 12; + // Any test-only item (module, fn, use, const...) ends at its closing brace or at a + // `;` outside braces, so its literals are skipped and later production is scanned. + test_module = true; + } else if chars.get(at..at + 2) == Some(&['/', '/']) { + while at < chars.len() && chars[at] != '\n' { + at += 1; + } + } else if chars.get(at..at + 2) == Some(&['/', '*']) { + at += 2; + let mut depth = 1; + while at < chars.len() && depth > 0 { + if chars.get(at..at + 2) == Some(&['/', '*']) { + depth += 1; + at += 2; + } else if chars.get(at..at + 2) == Some(&['*', '/']) { + depth -= 1; + at += 2; + } else { + at += 1; + } + } + } else if chars[at] == 'r' && matches!(chars.get(at + 1), Some('"' | '#')) { + let start = at; + at += 1; + while chars.get(at) == Some(&'#') { + at += 1; + } + let hashes = at - start - 1; + if chars.get(at) != Some(&'"') { + continue; + } + at += 1; + let begin = at; + while at < chars.len() { + if chars[at] == '"' && (1..=hashes).all(|n| chars.get(at + n) == Some(&'#')) { + break; + } + at += 1; + } + if !test_module { + out.push(chars[begin..at].iter().collect()); + } + at += 1 + hashes; + } else if chars[at] == '"' { + at += 1; + let mut text = String::new(); + while at < chars.len() { + let c = chars[at]; + at += 1; + if c == '"' { + break; + } + if c == '\\' && at < chars.len() { + text.push(chars[at]); + at += 1; + } else { + text.push(c); + } + } + if !test_module { + out.push(text); + } + } else if chars[at] == '\'' && chars.get(at + 1) == Some(&'\\') { + // An escaped character literal such as '\'', '\"' or '\u{22}': skip to its closing quote. + at += 3; + while at < chars.len() && chars[at] != '\'' && chars[at] != '\n' { + at += 1; + } + at += 1; + } else if chars[at] == '\'' && chars.get(at + 2) == Some(&'\'') { + at += 3; // A character literal such as '"', not the start of a string. + } else { + if test_module { + match chars[at] { + '{' => test_depth += 1, + '}' => { + test_depth = test_depth.saturating_sub(1); + if test_depth == 0 { + test_module = false; + } + } + ';' if test_depth == 0 => test_module = false, + _ => {} + } + } + at += 1; + } + } + out +} + +#[derive(Default)] +struct Scan { + files: usize, + literals: usize, + catalog: bool, + failures: Vec, +} + +fn check_tree(path: &Path, scan: &mut Scan) { + if path.is_dir() { + for entry in std::fs::read_dir(path).expect("source directory") { + let path = entry.expect("source entry").path(); + if path.file_name().is_some_and(|name| name == "tests") { + continue; + } + check_tree(&path, scan); + } + } else if path.extension().is_some_and(|ext| ext == "rs") { + let name = path.file_name().expect("source filename").to_string_lossy(); + if name == "tests.rs" || name.ends_with("_tests.rs") { + return; + } + let source = std::fs::read_to_string(path).expect("UTF-8 Rust source"); + scan.files += 1; + scan.catalog |= name == "catalog.rs"; + // Skip only inline test modules, not later production following test-only imports. + for text in literals(&source) { + scan.literals += 1; + if banned_display(&text) { + scan.failures.push(format!("{}: {text:?}", path.display())); + } + } + } +} + +#[test] +fn production_display_names_stay_neutral() { + let base = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let mut scan = Scan::default(); + for root in ROOTS { + let root = base.join(root); + assert!(root.exists(), "missing display source: {}", root.display()); + check_tree(&root, &mut scan); + } + // A scan that silently finds nothing would pass vacuously. + assert!(scan.catalog, "engine catalog.rs was not scanned"); + assert!(scan.files >= 50, "only {} source files scanned", scan.files); + assert!(scan.literals >= 5_000, "only {} literals scanned", scan.literals); + assert!(scan.failures.is_empty(), "Banned display names:\n{}", scan.failures.join("\n")); +} + +#[test] +fn guard_distinguishes_labels_from_identifiers_and_comments() { + for name in BANNED { + assert!(banned_display(name), "{name}"); + } + for name in ALLOWED { + assert!(!banned_display(name), "{name}"); + } + assert!(!banned_display("filter.cameraRaw")); + assert!(!banned_display("tool.guidedUpright")); + assert!(!banned_display("insert.smartArt")); + for identifier in ["mySmartArt", "SmartArt2", "_SmartArt_", "WordArtStyle"] { + assert!(!banned_display(identifier), "{identifier}"); + } + for label in ["Use SmartArt here", "WordArt…", "(SmartArt)"] { + assert!(banned_display(label), "{label}"); + } + assert_eq!(literals("// \"ignored\"\n/* \"ignored\" */ \"label\""), ["label"]); + assert_eq!(literals("r#\"raw label\"#"), ["raw label"]); + assert_eq!(literals("r##\"a \"# b\"## \"next\""), ["a \"# b", "next"]); + assert_eq!(literals("\"say \\\"hi\\\"\" \"next\""), ["say \"hi\"", "next"]); + assert_eq!(literals("/* outer /* \"inner\" */ \"still\" */ \"label\""), ["label"]); + assert_eq!(literals("let q = '\\''; let d = '\\\"'; let u = '\\u{22}'; \"label\""), ["label"]); + assert_eq!(literals("let d = '\"'; \"label\""), ["label"]); +} + +#[test] +fn guard_scans_production_after_test_only_items() { + let source = r###"#[cfg(test)] mod tests { let _ = "ignored"; } + #[cfg(test)] use fixtures::Helper; + fn production() { let _ = r#"live label"#; } + "###; + assert_eq!(literals(source), ["live label"]); + assert_eq!(literals("#[cfg(test)] mod tests; fn live() { let _ = \"live label\"; }"), ["live label"]); + let items = r#"#[cfg(test)] pub mod a { "x"; } + #[cfg(test)] fn helper() { "y"; } + #[cfg(test)] const C: &str = "z"; + fn live() { "live"; }"#; + assert_eq!(literals(items), ["live"]); +} diff --git a/docs/feature-names.md b/docs/feature-names.md new file mode 100644 index 000000000..800cd3f82 --- /dev/null +++ b/docs/feature-names.md @@ -0,0 +1,28 @@ +# Feature naming + +> **Last reviewed:** 2026-11-10 · **Last updated:** 2026-11-10 · **Change:** minor (Diagram words aligned with craft-i18n#211) + +ArtCraft apps use descriptive names for their own features. Product names used to describe compatibility, file formats, attribution or external software remain accurate. + +## Naming map + +SmartArt → Diagram; WordArt → Decorative Text. These are catalog labels for pending features, not a claim that the features are implemented. + +## Compatibility boundary + +The insert.smartArt and insert.wordArt command IDs and pending keytips remain unchanged. OOXML data, imported names and interoperability references are unchanged. No assets, attribution, dependencies or rendering behavior change. + +## Localization and verification + +Each shipped non-English interface catalog includes the new labels. Diagram uses a word distinct from Chart in every catalog: German (`Schema`), Ukrainian (`Схема`) and Romanian (`Schemă`) avoid their chart word, matching the shared wording in [craft-i18n#211](https://github.com/storytold/craft-i18n/pull/211) for German and Ukrainian. Regression tests cover the labels, every shipped catalog and stable command identities. The translations are AI-assisted and not yet reviewed by native speakers. + +## Regression coverage + +The standard-library-only `display_names` integration tests scan production Rust string literals for the retired display names and preserve explicit compatibility exceptions. They also test identifier, comment and raw-string handling. These focused checks do not replace application or visual testing. + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-11-10 | minor | Diagram no longer repeats the chart word: German `Schema`, Ukrainian `Схема` (as craft-i18n#211), Romanian `Schemă`; a test covers every shipped catalog | +| 2026-11-08 | minor | Created: SmartArt → Diagram and WordArt → Decorative Text as GridCraft's own catalog labels | diff --git a/docs/localization-parity.md b/docs/localization-parity.md index 9f5188948..9dec2d0e7 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-31 · **Change:** minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-10 · **Change:** minor (Diagram and Decorative Text labels, Diagram distinct from Chart); previously minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Per-language status against Excel, which ships its UI in dozens of languages (Windows language packs cover 100+), with localized function names, argument separators, number and date formats, @@ -133,6 +133,7 @@ language (none so far) and of localized function names. | Date | Change | Summary | |---|---|---| +| 2026-11-10 | minor | Diagram and Decorative Text labels in all 18 non-English catalogs; Diagram never repeats the catalog's chart word (German `Schema`, Ukrainian `Схема`, Romanian `Schemă`), see [feature-names.md](feature-names.md) | | 2026-10-31 | minor | System and embedded CJK text fallbacks share the Latin baseline in cells, editors and window titles; other scripts, symbols and emoji keep their alignment | | 2026-10-26 | minor | Shared bidi layout supports Arabic clusters, marks, visual cursor navigation and directional accessibility; cell Context/LTR/RTL is stored separately from sheet RTL and round-trips through XLSX | | 2026-10-26 | minor | Arabic text renders in cells and the interface: a system Arabic face (Tahoma, Arial or Segoe UI on Windows, Geeza Pro on macOS, Noto Sans or Naskh Arabic on Linux) joins the script fallbacks, fontconfig's Arabic choice, when it has Arabic letters, leads the Arabic fallbacks and draws Arabic wherever the UI or cell font has none, such as the default cell fonts (part of #229; a UI font with Arabic of its own, such as DejaVu Sans in the Flatpak, still draws interface Arabic, left for a follow-up), and craft-fonts' Noto Sans Arabic is a required face allowed on the web. Workbook fonts without Arabic keep every script fallback. Shaping and bidirectional layout remain separate work | diff --git a/docs/parity-checklist.md b/docs/parity-checklist.md index 4100ec630..ad7e680f9 100644 --- a/docs/parity-checklist.md +++ b/docs/parity-checklist.md @@ -165,11 +165,11 @@ Implemented: Missing: -- `insert.smartArt` SmartArt (Illustrations) +- `insert.smartArt` Diagram (Illustrations) - `insert.screenshot` Screenshot (Illustrations) - `insert.map` Maps (Charts) - `insert.timeline` Timeline (Filters) -- `insert.wordArt` WordArt (Text) +- `insert.wordArt` Decorative Text (Text) - `insert.signatureLine` Signature Line (Text) - `insert.object` Object (Text) From a36e4987c1b5b53af69a3f3573418e18612d1a4e Mon Sep 17 00:00:00 2001 From: FredApps <153173854+FredApps@users.noreply.github.com> Date: Tue, 10 Nov 2026 17:07:10 +0100 Subject: [PATCH 6/6] =?UTF-8?q?i18n(uk):=20translate=20Chart=20as=20=D0=94?= =?UTF-8?q?=D1=96=D0=B0=D0=B3=D1=80=D0=B0=D0=BC=D0=B0=20(#3703)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Use the shared craft-i18n Ukrainian term, distinct from Diagram (Схема). Also translate the remaining Chart-labelled strings that were still English. Co-authored-by: FredApps --- crates/ui-egui/src/i18n.rs | 8 ++++++++ crates/ui-egui/src/i18n/uk.rs | 30 +++++++++++++++--------------- docs/feature-names.md | 5 +++-- docs/localization-parity.md | 3 ++- 4 files changed, 28 insertions(+), 18 deletions(-) diff --git a/crates/ui-egui/src/i18n.rs b/crates/ui-egui/src/i18n.rs index 3aac97d6e..0ffd0ad09 100644 --- a/crates/ui-egui/src/i18n.rs +++ b/crates/ui-egui/src/i18n.rs @@ -342,6 +342,14 @@ mod tests { } } + /// Ukrainian Chart uses the shared craft-i18n term, which is not the Diagram word. + #[test] + fn ukrainian_chart_uses_the_shared_term() { + assert_eq!(Language::Uk.translated("Chart"), Some("Діаграма")); + assert_eq!(Language::Uk.translated("Insert Chart"), Some("Вставити діаграму")); + assert_eq!(Language::Uk.translated("Diagram"), Some("Схема")); + } + /// Help tables are sorted and unique: lookups binary-search them. #[test] fn function_help_tables_are_sorted_and_unique() { diff --git a/crates/ui-egui/src/i18n/uk.rs b/crates/ui-egui/src/i18n/uk.rs index ddb9368a0..7059a0ff4 100644 --- a/crates/ui-egui/src/i18n/uk.rs +++ b/crates/ui-egui/src/i18n/uk.rs @@ -442,7 +442,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "This workbook" => "This workbook", "Do you want to save the changes you made to {title}?" => "Do you want to save the changes you made to {title}?", "Save a copy as .{format}" => "Save a copy as .{format}", - "Select a chart first." => "Select a chart first.", + "Select a chart first." => "Спочатку виділіть діаграму.", "Text Box" => "Напис", "Select Data" => "Select Data", "Row Height" => "Row Height", @@ -480,13 +480,13 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Search Commands" => "Search Commands", "Agent Control" => "Agent Control", "About GridCraft" => "About GridCraft", - "Insert Chart" => "Insert Chart", + "Insert Chart" => "Вставити діаграму", "Select Data Source" => "Select Data Source", "Action Journal" => "Action Journal", "Macros" => "Macros", "Workbook Statistics" => "Workbook Statistics", "Accessibility" => "Accessibility", - "Chart Title" => "Chart Title", + "Chart Title" => "Назва діаграми", "Place Picture in Cell" => "Place Picture in Cell", "Place Picture over Cells" => "Place Picture over Cells", "Pick From List" => "Pick From List", @@ -604,9 +604,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Sum" => "Sum", "Product" => "Product", "Include forecast statistics" => "Include forecast statistics", - "Chart" => "Chart", - "Line chart" => "Line chart", - "Column chart" => "Column chart", + "Chart" => "Діаграма", + "Line chart" => "Лінійна діаграма", + "Column chart" => "Стовпчикова діаграма", "At each change in (column letter):" => "At each change in (column letter):", "Use function" => "Use function", "Add subtotal to (columns, e.g. C,D):" => "Add subtotal to (columns, e.g. C,D):", @@ -770,8 +770,8 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Scope" => "Scope", "Workbook" => "Workbook", "No names defined" => "No names defined", - "Recommended Charts" => "Recommended Charts", - "All Charts" => "All Charts", + "Recommended Charts" => "Рекомендовані діаграми", + "All Charts" => "Усі діаграми", "Clustered Column" => "Clustered Column", "Stacked Column" => "Stacked Column", "100% Stacked Column" => "100% Stacked Column", @@ -793,7 +793,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Waterfall" => "Waterfall", "Funnel" => "Funnel", "Combo" => "Combo", - "Chart data range:" => "Chart data range:", + "Chart data range:" => "Діапазон даних діаграми:", "Legend Entries (Series)" => "Legend Entries (Series)", "Series{n}" => "Series{n}", "Keep" => "Keep", @@ -801,7 +801,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Horizontal (Category) Axis Labels:" => "Horizontal (Category) Axis Labels:", "There are no conditional formatting rules on this sheet." => "There are no conditional formatting rules on this sheet.", "applies to" => "applies to", - "Type a command, e.g. freeze, chart, bold…" => "Type a command, e.g. freeze, chart, bold…", + "Type a command, e.g. freeze, chart, bold…" => "Введіть команду, наприклад закріпити, діаграма, напівжирний…", "Set cell:" => "Set cell:", "To value:" => "To value:", "By changing cell:" => "By changing cell:", @@ -866,7 +866,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Change" => "Change", "Change All" => "Change All", "$ English (United States)" => "$ English (United States)", - "Add Chart Element" => "Add Chart Element", + "Add Chart Element" => "Додати елемент діаграми", "Align Left" => "Align Left", "Align Right" => "Align Right", "Bottom Align" => "Bottom Align", @@ -874,7 +874,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Calculate Sheet (⇧F9)" => "Calculate Sheet (⇧F9)", "Calculation Options" => "Calculation Options", "Cell Styles" => "Cell Styles", - "Change Chart Type" => "Change Chart Type", + "Change Chart Type" => "Змінити тип діаграми", "Check Accessibility" => "Check Accessibility", "Clear Filter" => "Clear Filter", "Column or Bar" => "Column or Bar", @@ -887,11 +887,11 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Define Name…" => "Define Name…", "Delete Comment" => "Delete Comment", "Delete Sheet" => "Delete Sheet", - "Delete chart" => "Delete chart", + "Delete chart" => "Видалити діаграму", "Delete…" => "Delete…", "Dismiss" => "Dismiss", "Down" => "Down", - "Edit chart title" => "Edit chart title", + "Edit chart title" => "Редагувати назву діаграми", "Erase ink strokes" => "Erase ink strokes", "Every command run in this session (replayable)" => "Every command run in this session (replayable)", "Expand formula bar" => "Expand formula bar", @@ -899,7 +899,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Find & Select" => "Find & Select", "Forecast Sheet" => "Forecast Sheet", "Forecast end (date or number; blank: a third of the history):" => "Forecast end (date or number; blank: a third of the history):", - "Format Chart pane" => "Format Chart pane", + "Format Chart pane" => "Область форматування діаграми", "Format as Table" => "Format as Table", "Hide" => "Hide", "Hide Sheet" => "Hide Sheet", diff --git a/docs/feature-names.md b/docs/feature-names.md index 800cd3f82..267d1f1fb 100644 --- a/docs/feature-names.md +++ b/docs/feature-names.md @@ -1,6 +1,6 @@ # Feature naming -> **Last reviewed:** 2026-11-10 · **Last updated:** 2026-11-10 · **Change:** minor (Diagram words aligned with craft-i18n#211) +> **Last reviewed:** 2026-11-10 · **Last updated:** 2026-11-10 · **Change:** minor (Ukrainian Chart is Діаграма); previously minor (Diagram words aligned with craft-i18n#211) ArtCraft apps use descriptive names for their own features. Product names used to describe compatibility, file formats, attribution or external software remain accurate. @@ -14,7 +14,7 @@ The insert.smartArt and insert.wordArt command IDs and pending keytips remain un ## Localization and verification -Each shipped non-English interface catalog includes the new labels. Diagram uses a word distinct from Chart in every catalog: German (`Schema`), Ukrainian (`Схема`) and Romanian (`Schemă`) avoid their chart word, matching the shared wording in [craft-i18n#211](https://github.com/storytold/craft-i18n/pull/211) for German and Ukrainian. Regression tests cover the labels, every shipped catalog and stable command identities. The translations are AI-assisted and not yet reviewed by native speakers. +Each shipped non-English interface catalog includes the new labels. Diagram uses a word distinct from Chart in every catalog: German (`Schema`), Ukrainian (`Схема`) and Romanian (`Schemă`) avoid their chart word, matching the shared wording in [craft-i18n#211](https://github.com/storytold/craft-i18n/pull/211) for German and Ukrainian. Ukrainian Chart is `Діаграма`, the same term as in that catalog. Regression tests cover the labels, every shipped catalog and stable command identities. The translations are AI-assisted and not yet reviewed by native speakers. ## Regression coverage @@ -24,5 +24,6 @@ The standard-library-only `display_names` integration tests scan production Rust | Date | Change | Summary | |---|---|---| +| 2026-11-10 | minor | Ukrainian Chart is `Діаграма`, matching [craft-i18n#211](https://github.com/storytold/craft-i18n/pull/211) and distinct from Diagram (`Схема`) | | 2026-11-10 | minor | Diagram no longer repeats the chart word: German `Schema`, Ukrainian `Схема` (as craft-i18n#211), Romanian `Schemă`; a test covers every shipped catalog | | 2026-11-08 | minor | Created: SmartArt → Diagram and WordArt → Decorative Text as GridCraft's own catalog labels | diff --git a/docs/localization-parity.md b/docs/localization-parity.md index 9dec2d0e7..1de2927ac 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-10 · **Change:** minor (Diagram and Decorative Text labels, Diagram distinct from Chart); previously minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-10 · **Change:** minor (Ukrainian Chart is Діаграма); previously minor (Diagram and Decorative Text labels, Diagram distinct from Chart); previously minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Per-language status against Excel, which ships its UI in dozens of languages (Windows language packs cover 100+), with localized function names, argument separators, number and date formats, @@ -133,6 +133,7 @@ language (none so far) and of localized function names. | Date | Change | Summary | |---|---|---| +| 2026-11-10 | minor | Ukrainian Chart is `Діаграма`, matching [craft-i18n#211](https://github.com/storytold/craft-i18n/pull/211) and distinct from Diagram (`Схема`) | | 2026-11-10 | minor | Diagram and Decorative Text labels in all 18 non-English catalogs; Diagram never repeats the catalog's chart word (German `Schema`, Ukrainian `Схема`, Romanian `Schemă`), see [feature-names.md](feature-names.md) | | 2026-10-31 | minor | System and embedded CJK text fallbacks share the Latin baseline in cells, editors and window titles; other scripts, symbols and emoji keep their alignment | | 2026-10-26 | minor | Shared bidi layout supports Arabic clusters, marks, visual cursor navigation and directional accessibility; cell Context/LTR/RTL is stored separately from sheet RTL and round-trips through XLSX |