From 74ab5cc96930cad7d05e4553859b96c746dc5919 Mon Sep 17 00:00:00 2001 From: Diogo Soares Rodrigues Date: Tue, 10 Nov 2026 09:54:53 -0300 Subject: [PATCH 1/3] M12: regional format data (Region, 20 regions) (#3048) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * M12: regional format data (Region, 15 regions, Workbook::region) Adds the data model for Excel-style regional formats, with no behaviour change: nothing reads it yet. - core: region.rs with `Region` (separators, grouping, date order, short and long date and time codes, month and day names, AM/PM words, currency) and the 15 regions en-US, en-GB, en-IN, es-ES, de-DE, fr-FR, it-IT, pt-BR, pt-PT, ru-RU, cs-CZ, ja-JP, ko-KR, zh-CN, zh-TW. Values follow Unicode CLDR; the deliberate departures (four-digit short-date year, Windows grouping spaces) are listed in the module doc. `all`, `find` (strict tag), `lookup` (system or browser tag, best effort) and `default_region` (en-US) select one; `EN_US` reproduces today's behaviour exactly. Unit tests check the tables and the lookups. - model: `Workbook::region`, a `#[serde(skip)]` runtime field that is never saved and defaults to en-US. It is here so that the following changes can be reviewed independently of each other. The region is independent of the interface language and of the formula language. Builds on this, in order: typed input (core `parse_input_in`), display (numfmt and engine display), the engine session and `app.region` command, then the Options UI and the app defaults. * M12: region data: review fixes (tag extensions, zh territory, month periods) - lookup stops at an extension or private-use singleton: en-GB-u-ca-gregory is en-GB (ca was read as Canada) and zh-Hant-u-nu-latn is zh-TW. - For Chinese an explicit territory wins over the script, so zh-Hant-CN is China's conventions, consistent with zh-Hans-TW. - In regions whose month abbreviations end in a period (ru-RU, fr-FR), month_from_name accepts any abbreviation typed with one (мар., июн.); elsewhere a period stays outside the name, so en-US is unchanged. * Region: document where dates and month names follow Windows, not CLDR * M12: Swiss regions (en-CH, de-CH, fr-CH, it-CH) and nl-NL The Swiss regions group digits with an apostrophe (CLDR 48) and accept the typographic one glibc uses; lookup maps CH and LI tags to them. Dutch (Netherlands) follows CLDR 48. * Region: table-driven lookup test, under main's cognitive-complexity limit * Region: French and Italian Liechtenstein tags resolve to the Swiss regions * Region: credit the Unicode CLDR data under the Unicode License v3 (NOTICE, ATTRIBUTION.md) * Region: month_from_name takes one word; the currency format has the currency's own decimals --------- Co-authored-by: quantmind-br Co-authored-by: quantforge-br <218010578+quantforge-br@users.noreply.github.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> --- ATTRIBUTION.md | 1 + NOTICE | 49 ++ crates/core/src/lib.rs | 4 +- crates/core/src/region.rs | 883 +++++++++++++++++++++++++++++++++++ crates/model/src/workbook.rs | 6 + 5 files changed, 942 insertions(+), 1 deletion(-) create mode 100644 crates/core/src/region.rs diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index 2a4b5c96e..7dd388ccd 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -12,6 +12,7 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra | `epaint_default_fonts::HACK_REGULAR` (dependency font input) | Source Foundry Authors (Hack; DejaVu and Bitstream Vera foundations) | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Hack](https://github.com/source-foundry/Hack) | MIT + Bitstream Vera License (full upstream texts in `NOTICE`) | Hack-Regular.ttf is egui's bundled monospace face and GridCraft's last-resort symbol fallback. Dependency asset unmodified here, no font file copied into this repository. | | `epaint_default_fonts::NOTO_EMOJI_REGULAR` (dependency font input) | The Noto Project Authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Noto Emoji](https://fonts.google.com/noto/specimen/Noto+Emoji) | SIL Open Font License 1.1 (full upstream text in `NOTICE`) | NotoEmoji-Regular.ttf is enabled by egui's `monochrome_emoji_fonts` feature and used as the monochrome emoji fallback. Dependency asset unmodified here, no font file copied into this repository. | | `epaint_default_fonts::UBUNTU_LIGHT` (dependency font input) | Canonical Ltd. and the Ubuntu Font Family authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Ubuntu font](https://design.ubuntu.com/font) | Ubuntu Font Licence 1.0 (full upstream text in `NOTICE`) | Ubuntu-Light.ttf is egui's default proportional face. Dependency asset unmodified here, no font file copied into this repository. | +| Regional format data in `crates/core/src/region.rs` (data, no file of its own) | Unicode, Inc. (Unicode CLDR) | [cldr-json](https://github.com/unicode-org/cldr-json) (`cldr-numbers-full`, `cldr-dates-full`) and ICU's CLDR 48 data | Unicode License v3 (`Unicode-3.0`; full text in `NOTICE`) | Separators, month and weekday names, AM/PM words, currency symbols and date order of the built-in regions. No CLDR file is copied into this repository. | | `docs/brand/artcraft-logo.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms (`docs/brand/LICENSE-brand.txt`) | Trademark, not open source | | `docs/brand/artcraft-logo.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | | `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark | diff --git a/NOTICE b/NOTICE index d859d99d3..64753f94a 100644 --- a/NOTICE +++ b/NOTICE @@ -275,6 +275,55 @@ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE. + +The regional format data in crates/core/src/region.rs (separators, month and weekday names, AM/PM +words, currency symbols and date order) follows the Unicode Common Locale Data Repository (CLDR), +read from unicode-org/cldr-json and from ICU's CLDR 48 data, under the Unicode License v3. +Source: https://github.com/unicode-org/cldr-json +The following text is reproduced verbatim from cldr-json's LICENSE: + +UNICODE LICENSE V3 + +COPYRIGHT AND PERMISSION NOTICE + +Copyright © 2015-2024 Unicode, Inc. + +NOTICE TO USER: Carefully read the following legal agreement. BY +DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR +SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE +TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT +DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE. + +Permission is hereby granted, free of charge, to any person obtaining a +copy of data files and any associated documentation (the "Data Files") or +software and any associated documentation (the "Software") to deal in the +Data Files or Software without restriction, including without limitation +the rights to use, copy, modify, merge, publish, distribute, and/or sell +copies of the Data Files or Software, and to permit persons to whom the +Data Files or Software are furnished to do so, provided that either (a) +this copyright and permission notice appear with all copies of the Data +Files or Software, or (b) this copyright and permission notice appear in +associated Documentation. + +THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF +THIRD PARTY RIGHTS. + +IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE +BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, +OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, +WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, +ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA +FILES OR SOFTWARE. + +Except as contained in this notice, the name of a copyright holder shall +not be used in advertising or otherwise to promote the sale, use or other +dealings in these Data Files or Software without prior written +authorization of the copyright holder. + +SPDX-License-Identifier: Unicode-3.0 + Linux Wayland window decorations ------------------------------- sctk-adwaita 0.10.1 (https://github.com/PolyMeilex/sctk-adwaita, diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index b3f5dbd67..52b313b25 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -1,5 +1,5 @@ //! GridCraft core types: cell addresses, values and Excel's coercion rules, error values, -//! date serials and typed-input parsing. No dependencies on the rest of the workspace. +//! date serials, regional formats and typed-input parsing. No dependencies on the rest of the workspace. #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] #![forbid(unsafe_code)] @@ -7,6 +7,7 @@ pub mod addr; pub mod color; pub mod date; pub mod parse; +pub mod region; pub mod value; pub use addr::{ @@ -14,4 +15,5 @@ pub use addr::{ split_areas, split_sheet, unquote_sheet, }; pub use date::DateSystem; +pub use region::Region; pub use value::{Array, CellError, Value, compare, compare_numbers, compare_text, number_to_text, sort_compare}; diff --git a/crates/core/src/region.rs b/crates/core/src/region.rs new file mode 100644 index 000000000..30632603c --- /dev/null +++ b/crates/core/src/region.rs @@ -0,0 +1,883 @@ +//! Regional formats: how a region writes numbers, dates, times and money. +//! +//! Excel takes these from the operating system's regional settings; they are independent of the +//! interface language and of the formula language. A [`Region`] changes what people *type* into +//! a cell (`1.234,56`, `25/09/2026`) and what cells *show* (separators, month and day names, +//! AM/PM, the system short and long dates). It never changes what is stored: values are numbers, +//! and the number format codes saved in a file stay in their canonical en-US spelling. +//! +//! [`EN_US`] is the default and reproduces GridCraft's original behaviour exactly. +//! +//! # Where the values come from +//! The separators, month and weekday names, AM/PM words, currency symbols and the order of day, +//! month and year are the Unicode CLDR conventions (`unicode-org/cldr-json`: +//! `cldr-numbers-full/main//numbers.json` and `cldr-dates-full/main//ca-gregorian.json`), +//! which is also what macOS, Android and ICU ship. They were read from those files for en-GB, +//! en-IN, cs and ru, read from ICU's CLDR 48 data for the Swiss regions and nl-NL, and written +//! from the same conventions for the other regions. CLDR data is under the Unicode License v3, +//! reproduced in `NOTICE` (row in `ATTRIBUTION.md`). +//! Deliberate choices where operating systems differ from CLDR: +//! - Short dates use a four-digit year (`dd/mm/yyyy`, where CLDR has `dd/MM/y` or `dd/MM/yy`): +//! Excel's short date is the system's, Windows writes four digits, and a two-digit year would +//! not read back to the same date. +//! - The long date is CLDR's *full* date (with the weekday), like Windows en-US, except where +//! Windows' long date differs: Russian, Japanese and Chinese have no weekday (`23 августа +//! 2005 г.`, `2005年8月23日`), as in Microsoft's National Language Support tables. +//! - Korean writes the short date `yyyy-mm-dd` (CLDR: `yy. M. d.`), also as in Microsoft's +//! National Language Support tables. +//! - Month names for `mmm` are CLDR's abbreviated names (so en-GB has `Sept`), except +//! Portuguese, which drops CLDR's period (`jan`, not `jan.`) as Excel in Brazil shows it. +//! - French and Portuguese (Portugal) group digits with U+00A0 (Windows), not CLDR's U+202F. +//! Parsing accepts every space-like grouping character in those regions. +//! - The Swiss regions (en-CH, de-CH, fr-CH, it-CH; Liechtenstein uses de-CH) group digits with +//! CLDR's ASCII apostrophe (`1'234.50`); parsing also accepts the typographic `’` that glibc +//! uses. French (Switzerland) writes a decimal point like the other Swiss regions and glibc's +//! `fr_CH`, where CLDR has a comma. +//! +//! Regions this module doesn't list (`de-AT`, `es-MX`, `en-CA`…) resolve to the main region of +//! their language through [`lookup`]; their own conventions (a Mexican decimal point) are not +//! modelled. + +/// How digits are grouped. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Grouping { + /// Groups of three (`1,234,567`). + Thousands, + /// The Indian system: three digits, then groups of two (`12,34,567`). + Indian, +} + +/// The order of the parts of a short date. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum DateOrder { + Mdy, + Dmy, + Ymd, +} + +/// One region's conventions. See the module documentation. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Region { + /// BCP 47 tag, `pt-BR`. + pub tag: &'static str, + /// The region's name in its own language. + pub name: &'static str, + /// Windows locale id, used by `[$-409]`-style format tags. + pub lcid: u16, + pub decimal: char, + pub group: char, + pub grouping: Grouping, + pub date_order: DateOrder, + pub date_sep: char, + pub time_sep: char, + /// The system short date as a canonical format code (`dd/mm/yyyy`): built-in format 14. + pub short_date: &'static str, + /// The system long date (`[$-F800]`). + pub long_date: &'static str, + /// The system short time. It contains `AM/PM` in regions that use a 12-hour clock. + pub short_time: &'static str, + /// The system long time (`[$-F400]`). + pub long_time: &'static str, + /// Built-in format 22: the short date and a 24-hour time. + pub datetime: &'static str, + pub months: [&'static str; 12], + pub months_abbr: [&'static str; 12], + /// The month names a date with a day uses (Russian `1 января`, Czech `1. ledna`), where they + /// differ from `months`. + pub months_genitive: Option<[&'static str; 12]>, + /// Sunday first. + pub weekdays: [&'static str; 7], + pub weekdays_abbr: [&'static str; 7], + pub am_pm: [&'static str; 2], + pub currency: &'static str, + /// The currency format with the currency's usual decimals (two, or none for the yen and the + /// won), as a canonical code. Where it starts with a quote the symbol comes before the number, + /// otherwise after. + pub currency_format: &'static str, + /// The same with no decimals (for a typed whole amount). + pub currency_format_int: &'static str, +} + +const EN_MONTHS: [&str; 12] = + ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"]; +const EN_MONTHS_ABBR: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; +const EN_DAYS: [&str; 7] = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; +const EN_DAYS_ABBR: [&str; 7] = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]; +const AM_PM: [&str; 2] = ["AM", "PM"]; + +/// English (United States): the default. Identical to the formats before regions existed. +pub static EN_US: Region = Region { + tag: "en-US", + name: "English (United States)", + lcid: 0x0409, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Mdy, + date_sep: '/', + time_sep: ':', + short_date: "m/d/yyyy", + long_date: "dddd, mmmm dd, yyyy", + short_time: "h:mm AM/PM", + long_time: "h:mm:ss AM/PM", + datetime: "m/d/yyyy h:mm", + months: EN_MONTHS, + months_abbr: EN_MONTHS_ABBR, + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "$", + currency_format: "\"$\"#,##0.00", + currency_format_int: "\"$\"#,##0", +}; + +static EN_GB: Region = Region { + tag: "en-GB", + name: "English (United Kingdom)", + lcid: 0x0809, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: EN_MONTHS, + months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"], + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "£", + currency_format: "\"£\"#,##0.00", + currency_format_int: "\"£\"#,##0", +}; + +static EN_IN: Region = Region { + tag: "en-IN", + name: "English (India)", + lcid: 0x4009, + decimal: '.', + group: ',', + grouping: Grouping::Indian, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "h:mm AM/PM", + long_time: "h:mm:ss AM/PM", + datetime: "dd/mm/yyyy h:mm", + months: EN_MONTHS, + months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"], + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "₹", + currency_format: "\"₹\"#,##0.00", + currency_format_int: "\"₹\"#,##0", +}; + +static EN_CH: Region = Region { + tag: "en-CH", + name: "English (Switzerland)", + // English (Switzerland) has no LCID of its own: Windows reports LOCALE_CUSTOM_UNSPECIFIED. + lcid: 0x1000, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: EN_MONTHS, + months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"], + months_genitive: None, + weekdays: EN_DAYS, + weekdays_abbr: EN_DAYS_ABBR, + am_pm: AM_PM, + currency: "CHF", + currency_format: "\"CHF\" #,##0.00", + currency_format_int: "\"CHF\" #,##0", +}; + +static ES_ES: Region = Region { + tag: "es-ES", + name: "Español (España)", + lcid: 0x0C0A, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d \"de\" mmmm \"de\" yyyy", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "dd/mm/yyyy h:mm", + months: ["enero", "febrero", "marzo", "abril", "mayo", "junio", "julio", "agosto", "septiembre", "octubre", "noviembre", "diciembre"], + months_abbr: ["ene", "feb", "mar", "abr", "may", "jun", "jul", "ago", "sep", "oct", "nov", "dic"], + months_genitive: None, + weekdays: ["domingo", "lunes", "martes", "miércoles", "jueves", "viernes", "sábado"], + weekdays_abbr: ["dom", "lun", "mar", "mié", "jue", "vie", "sáb"], + am_pm: ["a. m.", "p. m."], + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static DE_DE: Region = Region { + tag: "de-DE", + name: "Deutsch (Deutschland)", + lcid: 0x0407, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d. mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"], + months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"], + months_genitive: None, + weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"], + weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static DE_CH: Region = Region { + tag: "de-CH", + name: "Deutsch (Schweiz)", + lcid: 0x0807, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d. mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"], + months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"], + months_genitive: None, + weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"], + weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"], + am_pm: AM_PM, + currency: "CHF", + currency_format: "\"CHF\" #,##0.00", + currency_format_int: "\"CHF\" #,##0", +}; + +static FR_FR: Region = Region { + tag: "fr-FR", + name: "Français (France)", + lcid: 0x040C, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"], + months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."], + months_genitive: None, + weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"], + weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static FR_CH: Region = Region { + tag: "fr-CH", + name: "Français (Suisse)", + lcid: 0x100C, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"], + months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."], + months_genitive: None, + weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"], + weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."], + am_pm: AM_PM, + currency: "CHF", + currency_format: "#,##0.00 \"CHF\"", + currency_format_int: "#,##0 \"CHF\"", +}; + +static IT_IT: Region = Region { + tag: "it-IT", + name: "Italiano (Italia)", + lcid: 0x0410, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"], + months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"], + months_genitive: None, + weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"], + weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static IT_CH: Region = Region { + tag: "it-CH", + name: "Italiano (Svizzera)", + lcid: 0x0810, + decimal: '.', + group: '\'', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd, d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"], + months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"], + months_genitive: None, + weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"], + weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"], + am_pm: AM_PM, + currency: "CHF", + currency_format: "\"CHF\" #,##0.00", + currency_format_int: "\"CHF\" #,##0", +}; + +static NL_NL: Region = Region { + tag: "nl-NL", + name: "Nederlands (Nederland)", + lcid: 0x0413, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '-', + time_sep: ':', + short_date: "dd-mm-yyyy", + long_date: "dddd d mmmm yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd-mm-yyyy hh:mm", + months: ["januari", "februari", "maart", "april", "mei", "juni", "juli", "augustus", "september", "oktober", "november", "december"], + months_abbr: ["jan", "feb", "mrt", "apr", "mei", "jun", "jul", "aug", "sep", "okt", "nov", "dec"], + months_genitive: None, + weekdays: ["zondag", "maandag", "dinsdag", "woensdag", "donderdag", "vrijdag", "zaterdag"], + weekdays_abbr: ["zo", "ma", "di", "wo", "do", "vr", "za"], + am_pm: AM_PM, + currency: "€", + currency_format: "\"€\" #,##0.00", + currency_format_int: "\"€\" #,##0", +}; + +static PT_BR: Region = Region { + tag: "pt-BR", + name: "Português (Brasil)", + lcid: 0x0416, + decimal: ',', + group: '.', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d \"de\" mmmm \"de\" yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"], + months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"], + months_genitive: None, + weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"], + weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"], + am_pm: AM_PM, + currency: "R$", + currency_format: "\"R$\" #,##0.00", + currency_format_int: "\"R$\" #,##0", +}; + +static PT_PT: Region = Region { + tag: "pt-PT", + name: "Português (Portugal)", + lcid: 0x0816, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '/', + time_sep: ':', + short_date: "dd/mm/yyyy", + long_date: "dddd, d \"de\" mmmm \"de\" yyyy", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd/mm/yyyy hh:mm", + months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"], + months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"], + months_genitive: None, + weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"], + weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"], + am_pm: AM_PM, + currency: "€", + currency_format: "#,##0.00 \"€\"", + currency_format_int: "#,##0 \"€\"", +}; + +static RU_RU: Region = Region { + tag: "ru-RU", + name: "Русский (Россия)", + lcid: 0x0419, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "d mmmm yyyy\" г.\"", + short_time: "hh:mm", + long_time: "hh:mm:ss", + datetime: "dd.mm.yyyy hh:mm", + months: ["январь", "февраль", "март", "апрель", "май", "июнь", "июль", "август", "сентябрь", "октябрь", "ноябрь", "декабрь"], + months_abbr: ["янв.", "февр.", "март", "апр.", "май", "июнь", "июль", "авг.", "сент.", "окт.", "нояб.", "дек."], + months_genitive: Some(["января", "февраля", "марта", "апреля", "мая", "июня", "июля", "августа", "сентября", "октября", "ноября", "декабря"]), + weekdays: ["воскресенье", "понедельник", "вторник", "среда", "четверг", "пятница", "суббота"], + weekdays_abbr: ["вс", "пн", "вт", "ср", "чт", "пт", "сб"], + am_pm: AM_PM, + currency: "₽", + currency_format: "#,##0.00\u{a0}\"₽\"", + currency_format_int: "#,##0\u{a0}\"₽\"", +}; + +static CS_CZ: Region = Region { + tag: "cs-CZ", + name: "Čeština (Česko)", + lcid: 0x0405, + decimal: ',', + group: '\u{a0}', + grouping: Grouping::Thousands, + date_order: DateOrder::Dmy, + date_sep: '.', + time_sep: ':', + short_date: "dd.mm.yyyy", + long_date: "dddd d. mmmm yyyy", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "dd.mm.yyyy h:mm", + months: ["leden", "únor", "březen", "duben", "květen", "červen", "červenec", "srpen", "září", "říjen", "listopad", "prosinec"], + months_abbr: ["led", "úno", "bře", "dub", "kvě", "čvn", "čvc", "srp", "zář", "říj", "lis", "pro"], + months_genitive: Some(["ledna", "února", "března", "dubna", "května", "června", "července", "srpna", "září", "října", "listopadu", "prosince"]), + weekdays: ["neděle", "pondělí", "úterý", "středa", "čtvrtek", "pátek", "sobota"], + weekdays_abbr: ["ne", "po", "út", "st", "čt", "pá", "so"], + am_pm: ["dop.", "odp."], + currency: "Kč", + currency_format: "#,##0.00 \"Kč\"", + currency_format_int: "#,##0 \"Kč\"", +}; + +static JA_JP: Region = Region { + tag: "ja-JP", + name: "日本語 (日本)", + lcid: 0x0411, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '/', + time_sep: ':', + short_date: "yyyy/m/d", + long_date: "yyyy\"年\"m\"月\"d\"日\"", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "yyyy/m/d h:mm", + months: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_genitive: None, + weekdays: ["日曜日", "月曜日", "火曜日", "水曜日", "木曜日", "金曜日", "土曜日"], + weekdays_abbr: ["日", "月", "火", "水", "木", "金", "土"], + am_pm: ["午前", "午後"], + currency: "¥", + currency_format: "\"¥\"#,##0", + currency_format_int: "\"¥\"#,##0", +}; + +static KO_KR: Region = Region { + tag: "ko-KR", + name: "한국어 (대한민국)", + lcid: 0x0412, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '-', + time_sep: ':', + short_date: "yyyy-mm-dd", + long_date: "yyyy\"년\" m\"월\" d\"일\" dddd", + short_time: "AM/PM h:mm", + long_time: "AM/PM h:mm:ss", + datetime: "yyyy-mm-dd hh:mm", + months: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"], + months_abbr: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"], + months_genitive: None, + weekdays: ["일요일", "월요일", "화요일", "수요일", "목요일", "금요일", "토요일"], + weekdays_abbr: ["일", "월", "화", "수", "목", "금", "토"], + am_pm: ["오전", "오후"], + currency: "₩", + currency_format: "\"₩\"#,##0", + currency_format_int: "\"₩\"#,##0", +}; + +static ZH_CN: Region = Region { + tag: "zh-CN", + name: "中文 (中国)", + lcid: 0x0804, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '/', + time_sep: ':', + short_date: "yyyy/m/d", + long_date: "yyyy\"年\"m\"月\"d\"日\"", + short_time: "h:mm", + long_time: "h:mm:ss", + datetime: "yyyy/m/d h:mm", + months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"], + months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_genitive: None, + weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"], + weekdays_abbr: ["周日", "周一", "周二", "周三", "周四", "周五", "周六"], + am_pm: ["上午", "下午"], + currency: "¥", + currency_format: "\"¥\"#,##0.00", + currency_format_int: "\"¥\"#,##0", +}; + +static ZH_TW: Region = Region { + tag: "zh-TW", + name: "中文 (台灣)", + lcid: 0x0404, + decimal: '.', + group: ',', + grouping: Grouping::Thousands, + date_order: DateOrder::Ymd, + date_sep: '/', + time_sep: ':', + short_date: "yyyy/m/d", + long_date: "yyyy\"年\"m\"月\"d\"日\"", + short_time: "AM/PM hh:mm", + long_time: "AM/PM hh:mm:ss", + datetime: "yyyy/m/d hh:mm", + months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"], + months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"], + months_genitive: None, + weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"], + weekdays_abbr: ["週日", "週一", "週二", "週三", "週四", "週五", "週六"], + am_pm: ["上午", "下午"], + currency: "NT$", + currency_format: "\"NT$\"#,##0.00", + currency_format_int: "\"NT$\"#,##0", +}; + +static ALL: [&Region; 20] = [ + &EN_US, &EN_GB, &EN_IN, &EN_CH, &ES_ES, &DE_DE, &DE_CH, &FR_FR, &FR_CH, &IT_IT, &IT_CH, &NL_NL, &PT_BR, &PT_PT, &RU_RU, &CS_CZ, &JA_JP, &KO_KR, + &ZH_CN, &ZH_TW, +]; + +/// Every region, English (United States) first. +pub fn all() -> &'static [&'static Region] { + &ALL +} + +/// The default region, English (United States). +pub fn default_region() -> &'static Region { + &EN_US +} + +/// The region with exactly this tag (`pt-BR`; case and `_` don't matter). +pub fn find(tag: &str) -> Option<&'static Region> { + let t = tag.trim().replace('_', "-"); + ALL.iter().copied().find(|r| r.tag.eq_ignore_ascii_case(&t)) +} + +/// The region for a tag as operating systems and browsers write it: `pt-BR`, `pt_BR.UTF-8`, +/// `de`, `en-AU`, `zh-Hant`, `sr_RS@latin`. An exact match wins; otherwise the language's main +/// region (with the usual variants: `en-AU`→`en-GB`, `pt-AO`→`pt-PT`, `zh-Hant`→`zh-TW`). +/// An unknown language is English (United States). +pub fn lookup(tag: &str) -> &'static Region { + if let Some(r) = find(tag) { + return r; + } + let base = tag.split(['.', '@']).next().unwrap_or(""); + let norm = base.trim().replace('_', "-"); + if let Some(r) = find(&norm) { + return r; + } + let mut parts = norm.split('-').filter(|p| !p.is_empty()); + let lang = parts.next().unwrap_or("").to_ascii_lowercase(); + let mut script = String::new(); + let mut country = String::new(); + for p in parts { + // An extension or private-use singleton (`-u-ca-gregory`, `-x-…`) ends the region part. + if p.len() == 1 { + break; + } + if p.len() == 4 && p.chars().all(|c| c.is_ascii_alphabetic()) { + script = p.to_ascii_lowercase(); + } else if (p.len() == 2 && p.chars().all(|c| c.is_ascii_alphabetic())) || (p.len() == 3 && p.chars().all(|c| c.is_ascii_digit())) { + country = p.to_ascii_uppercase(); + } + } + let tag = match lang.as_str() { + "en" => match country.as_str() { + "GB" | "AU" | "NZ" | "IE" => "en-GB", + "IN" => "en-IN", + "CH" | "LI" => "en-CH", + _ => "en-US", + }, + "pt" => match country.as_str() { + "AO" | "CH" | "CV" | "FR" | "GQ" | "GW" | "LU" | "MO" | "MZ" | "ST" | "TL" | "PT" => "pt-PT", + _ => "pt-BR", + }, + "zh" => match country.as_str() { + // An explicit territory wins over the script (`zh-Hant-CN` is China's conventions). + "CN" | "SG" => "zh-CN", + "TW" | "HK" | "MO" => "zh-TW", + _ if script == "hant" => "zh-TW", + _ => "zh-CN", + }, + "es" => "es-ES", + "de" => match country.as_str() { + "CH" | "LI" => "de-CH", + _ => "de-DE", + }, + "fr" => match country.as_str() { + "CH" | "LI" => "fr-CH", + _ => "fr-FR", + }, + "it" => match country.as_str() { + "CH" | "LI" => "it-CH", + _ => "it-IT", + }, + "nl" => "nl-NL", + "ru" => "ru-RU", + "cs" => "cs-CZ", + "ja" => "ja-JP", + "ko" => "ko-KR", + _ => "en-US", + }; + find(tag).unwrap_or(&EN_US) +} + +impl Region { + /// True for the characters that separate groups of digits in this region. A region that + /// groups with a space also accepts the other space-like characters (NBSP, narrow NBSP, thin + /// space), as people type and paste any of them; one that groups with an apostrophe accepts + /// both `'` and `’`. + pub fn is_group(&self, c: char) -> bool { + c == self.group || (is_space_like(self.group) && is_space_like(c)) || (is_apostrophe_like(self.group) && is_apostrophe_like(c)) + } + + /// True when the currency symbol comes before the number. + pub fn currency_first(&self) -> bool { + self.currency_format.starts_with('"') + } + + /// Month number (1–12) for a month name as typed: the full name, the abbreviation (with or + /// without its period) or any prefix of the full name of three or more letters (`Sept`), + /// ignoring case. `word` is one word: the genitive form used after a day (`января` in + /// `1 января`) matches too, and the caller reads the day itself. + pub fn month_from_name(&self, word: &str) -> Option { + let w = word.trim().to_lowercase(); + // Where the abbreviations end in a period (`сент.`, `janv.`), any abbreviation may be typed + // with one (`мар.`, `juin.`). Elsewhere a period is not part of a month name, so en-US + // `Mar. 4, 2026` stays text as before. + let dotted = self.months_abbr.iter().any(|a| a.ends_with('.')); + let stem = if dotted { w.trim_end_matches('.') } else { w.as_str() }; + if stem.is_empty() { + return None; + } + let long_enough = stem.chars().count() >= 3; + for (i, full) in self.months.iter().enumerate() { + let full_l = full.to_lowercase(); + let abbr_l = self.months_abbr.get(i).map(|a| a.to_lowercase()).unwrap_or_default(); + let gen_l = self.months_genitive.as_ref().and_then(|g| g.get(i)).map(|g| g.to_lowercase()); + let hit = w == full_l + || w == abbr_l + || stem == abbr_l.trim_end_matches('.') + || gen_l.as_deref() == Some(w.as_str()) + || (long_enough && full_l.starts_with(stem)) + || (long_enough && gen_l.as_deref().is_some_and(|g| g.starts_with(stem))); + if hit { + return Some(i as u32 + 1); + } + } + None + } + + /// The name of month `month` (1–12) for `mmmm`; `with_day` selects the genitive form where + /// the region has one. + pub fn month_name(&self, month: u32, with_day: bool) -> &'static str { + let i = (month as usize).wrapping_sub(1); + if with_day && let Some(g) = &self.months_genitive { + return g.get(i).copied().unwrap_or(""); + } + self.months.get(i).copied().unwrap_or("") + } + + /// True when the short time uses a 12-hour clock. + pub fn is_12_hour(&self) -> bool { + self.short_time.contains("AM/PM") + } +} + +fn is_space_like(c: char) -> bool { + matches!(c, ' ' | '\u{a0}' | '\u{202f}' | '\u{2009}') +} + +fn is_apostrophe_like(c: char) -> bool { + matches!(c, '\'' | '\u{2019}') +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn lookup_accepts_system_and_browser_tags() { + let cases = [ + ("pt-BR", "pt-BR"), + ("pt_BR.UTF-8", "pt-BR"), + ("pt", "pt-BR"), + ("pt-PT", "pt-PT"), + ("pt-AO", "pt-PT"), + ("de", "de-DE"), + ("de-AT", "de-DE"), + ("de_CH.UTF-8", "de-CH"), + ("de-LI", "de-CH"), + ("fr-CH", "fr-CH"), + ("fr-LI", "fr-CH"), + ("fr-BE", "fr-FR"), + ("it_CH.UTF-8", "it-CH"), + ("it-LI", "it-CH"), + ("en-CH", "en-CH"), + ("nl_NL.UTF-8", "nl-NL"), + ("nl", "nl-NL"), + ("en-AU", "en-GB"), + ("en_GB.UTF-8", "en-GB"), + ("en-IN", "en-IN"), + ("en-CA", "en-US"), + ("zh-Hant", "zh-TW"), + ("zh-HK", "zh-TW"), + ("zh", "zh-CN"), + // Extensions don't replace the region or script; an explicit territory beats the script. + ("en-GB-u-ca-gregory", "en-GB"), + ("zh-Hant-u-nu-latn", "zh-TW"), + ("zh-Hant-CN", "zh-CN"), + ("zh-Hans-TW", "zh-TW"), + ("cs_CZ.UTF-8", "cs-CZ"), + ("C", "en-US"), + ("", "en-US"), + ("xx-YY", "en-US"), + ]; + for (tag, expected) in cases { + assert_eq!(lookup(tag).tag, expected, "{tag}"); + } + assert_eq!(find("DE-de").map(|r| r.tag), Some("de-DE")); + assert!(find("xx").is_none()); + } + + #[test] + fn every_region_is_consistent() { + let mut tags = std::collections::HashSet::new(); + for r in all() { + assert!(tags.insert(r.tag), "duplicate {}", r.tag); + assert_ne!(r.decimal, r.group, "{}", r.tag); + assert!(r.currency_format.contains(r.currency), "{}", r.tag); + assert!(r.months.iter().all(|m| !m.is_empty()) && r.months_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag); + assert!(r.weekdays.iter().all(|m| !m.is_empty()) && r.weekdays_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag); + assert_eq!(find(r.tag).map(|x| x.tag), Some(r.tag)); + } + } + + #[test] + fn month_names_match_loosely() { + let de = find("de-DE").unwrap(); + assert_eq!(de.month_from_name("März"), Some(3)); + assert_eq!(de.month_from_name("mär"), Some(3)); + assert_eq!(de.month_from_name("DEZ"), Some(12)); + assert_eq!(de.month_from_name("de"), None); + let gb = find("en-GB").unwrap(); + assert_eq!(gb.month_from_name("Sept"), Some(9)); + assert_eq!(gb.month_from_name("sep"), Some(9)); + let ru = find("ru-RU").unwrap(); + assert_eq!(ru.month_from_name("сентября"), Some(9)); + assert_eq!(ru.month_from_name("сент."), Some(9)); + // Abbreviations typed with a period the table doesn't carry. + assert_eq!(ru.month_from_name("мар."), Some(3)); + assert_eq!(ru.month_from_name("июн."), Some(6)); + assert_eq!(find("fr-FR").unwrap().month_from_name("juin."), Some(6)); + assert_eq!(EN_US.month_from_name("Mar."), None); + assert_eq!(de.month_from_name("."), None); + assert_eq!(EN_US.month_from_name("Ma"), None); + assert_eq!(EN_US.month_from_name("May"), Some(5)); + } + + #[test] + fn space_like_and_apostrophe_groups_are_interchangeable() { + let fr = find("fr-FR").unwrap(); + assert!(fr.is_group('\u{a0}') && fr.is_group(' ') && fr.is_group('\u{202f}')); + assert!(!find("de-DE").unwrap().is_group(' ')); + let ch = find("de-CH").unwrap(); + assert!(ch.is_group('\'') && ch.is_group('\u{2019}')); + assert!(!ch.is_group(' ') && !find("de-DE").unwrap().is_group('\'')); + } +} diff --git a/crates/model/src/workbook.rs b/crates/model/src/workbook.rs index 665ec323f..ab401828d 100644 --- a/crates/model/src/workbook.rs +++ b/crates/model/src/workbook.rs @@ -268,6 +268,11 @@ pub struct Workbook { /// `preserved` are relative to. Saving into another folder rewrites them from there. Not in JSON. #[serde(skip)] pub source_dir: Option, + /// The region whose formats this workbook is typed and shown in (Options › Region). A runtime + /// setting of the application, not of the file: it is not saved, and the engine session sets + /// it on every document it holds. en-US until then. + #[serde(skip, default = "gridcraft_core::region::default_region")] + pub region: &'static gridcraft_core::Region, /// The file this workbook reports as its own, which is what `CELL("filename")` returns. /// `None` = no file yet, which Excel reports as an empty string until the workbook has been /// saved once. Deliberately separate from `source_dir`: that one names the folder relative @@ -365,6 +370,7 @@ impl Workbook { macros_unread: Arc::default(), preserved: None, source_dir: None, + region: gridcraft_core::region::default_region(), source_path: None, } } From 50a9e9ede44a0c9bd1840fa78e12fcdc204f0c73 Mon Sep 17 00:00:00 2001 From: Harrison Booth <73434116+Harrison-Booth@users.noreply.github.com> Date: Tue, 10 Nov 2026 08:05:02 -0600 Subject: [PATCH 2/3] XLS: formulas open as legacy formulas with Excel's implicit @ (#410) (#1367) .xls formulas are read as legacy formulas with Excel's implicit @ (apply_legacy via read_xls_with), one-cell Ctrl+Shift+Enter arrays keep their range, @ of a 2-D range meets the formula's row and column, and defined-name analyses are shared across the formulas of a pass and replayed exactly. Closes #410 --- crates/calc/src/eval.rs | 5 + crates/calc/src/tests.rs | 19 ++- crates/engine/src/io.rs | 5 +- crates/engine/src/tests.rs | 73 ++++++++++ crates/engine/tests/legacy_array_functions.rs | 33 ++++- crates/xlsx/src/array_formula.rs | 119 +++++++++++++++- crates/xlsx/src/lib.rs | 2 +- crates/xlsx/src/sheet_write.rs | 2 +- crates/xlsx/src/tests/array_export.rs | 132 ++++++++++++++++++ crates/xlsx/src/tests/xls_tests.rs | 113 ++++++++++++++- crates/xlsx/src/write.rs | 3 + crates/xlsx/src/xls.rs | 33 ++++- docs/file-format-parity.md | 3 +- 13 files changed, 518 insertions(+), 24 deletions(-) diff --git a/crates/calc/src/eval.rs b/crates/calc/src/eval.rs index 41ffca9c2..2fda4442b 100644 --- a/crates/calc/src/eval.rs +++ b/crates/calc/src/eval.rs @@ -543,6 +543,11 @@ impl<'h> Evaluator<'h> { if r.height() == 1 && r.start.col <= self.at.col && self.at.col <= r.end.col { return Some(CellRef::new(r.start.row, self.at.col)); } + // A 2-D range intersects on both the formula's row and column (observed in Excel, on + // another sheet too); one that includes the formula's own cell is a circular reference. + if r.start.row <= self.at.row && self.at.row <= r.end.row && r.start.col <= self.at.col && self.at.col <= r.end.col { + return Some(self.at); + } None } diff --git a/crates/calc/src/tests.rs b/crates/calc/src/tests.rs index b4f820f0c..8784a8d69 100644 --- a/crates/calc/src/tests.rs +++ b/crates/calc/src/tests.rs @@ -1923,8 +1923,8 @@ fn offset_extreme_arguments_return_ref() { } /// `@` intersects a reference with the formula's row (a column range) or column (a row range); -/// a 2D range or one the formula does not line up with gives #VALUE!. An array gives its -/// top-left value. Legacy formulas read from files use it where Excel intersects implicitly. +/// a 2D range both (circular when it includes the formula's cell); a range the formula does not +/// line up with gives #VALUE!. An array gives its top-left value. Legacy formulas read from files use it where Excel intersects implicitly. #[test] fn implicit_intersection_operator() { let mut t = T::new(); @@ -1941,9 +1941,22 @@ fn implicit_intersection_operator() { t.set("D3", "=SUM(@A1:A3)"); assert_eq!(t.num("B2"), 2f64.tanh()); assert_eq!(t.num("B7"), 5f64.tanh()); - for at in ["B3", "B9", "E9"] { + for at in ["B9", "E9"] { assert_eq!(t.get(at), Value::Error(CellError::Value), "{at}"); } + // A 2D range that includes the formula's own cell is a circular reference (Excel shows 0). + assert_eq!(t.get("B3"), Value::Error(CellError::Circ)); + // From another sheet a 2D range intersects on both row and column; outside it is #VALUE!. + t.wb.sheets.push(std::sync::Arc::new(gridcraft_model::Sheet::new("Other"))); + for (at, input) in [("B2", "=@Sheet1!A1:C3"), ("C1", "=@Sheet1!A1:C3"), ("B9", "=@Sheet1!A1:C3"), ("E2", "=@Sheet1!A1:C3")] { + t.wb.sheet_mut(1).unwrap().cells.set(c(at), Cell::formula(Formula::new(input))); + } + t.calc.recalc_all(&mut t.wb); + let other = |at: &str| t.wb.sheet(1).unwrap().value(c(at)); + assert_eq!(other("B2"), Value::Number(2f64.tanh())); // Sheet1!B2 + assert_eq!(other("C1"), Value::Number(0.0)); + assert_eq!(other("B9"), Value::Error(CellError::Value)); + assert_eq!(other("E2"), Value::Error(CellError::Value)); assert_eq!(t.num("D1"), 7.0); assert_eq!(t.num("D2"), 2.0); assert_eq!(t.num("D3"), 3.0); diff --git a/crates/engine/src/io.rs b/crates/engine/src/io.rs index 22c236e61..690efec1d 100644 --- a/crates/engine/src/io.rs +++ b/crates/engine/src/io.rs @@ -9,7 +9,7 @@ use crate::{EngineError, Result}; mod sylk; -/// The function library's answers to what the XLSX reader and writer need to know about functions +/// The function library's answers to what the XLSX and XLS readers and the XLSX writer need to know about functions /// (which names are built in, which parameters take a single value). pub const XLSX_FUNCTIONS: gridcraft_xlsx::FunctionInfo = gridcraft_xlsx::FunctionInfo { is_builtin: |name| gridcraft_functions::lookup(name).is_some(), @@ -271,7 +271,8 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, let kind = FileKind::from_path(name); let sniffed = gridcraft_xlsx::sniff(bytes); if sniffed == gridcraft_xlsx::Format::Xls { - let (wb, report) = gridcraft_xlsx::read_xls(bytes).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?; + let (wb, report) = + gridcraft_xlsx::read_xls_with(bytes, XLSX_FUNCTIONS).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?; return Ok((wb, report.warnings)); } if sniffed == gridcraft_xlsx::Format::LegacyBinary { diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index fcaf19c36..08fc9119f 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -3664,6 +3664,79 @@ fn xls_import_keeps_formulas_and_never_saves_back_to_xls() { assert!(s.execute("file.saveBytes", json!({"format": "xlsx"})).is_ok()); } +#[test] +fn xls_formulas_calculate_with_legacy_implicit_intersection() { + // Data!A1 = 11, B1:G1 = 1, A2 = 5, A3 = 7. Report!B1:G1 are filled, so a formula in Report!A1 + // that spilled would show #SPILL!. Every cached result is 0, so the values below are GridCraft's. + fn rec(id: u16, payload: &[u8]) -> Vec { + [&id.to_le_bytes()[..], &(payload.len() as u16).to_le_bytes(), payload].concat() + } + let u16s = |v: &[u16]| v.iter().flat_map(|x| x.to_le_bytes()).collect::>(); + let bof = |dt: u16| rec(0x0809, &[&0x0600u16.to_le_bytes()[..], &dt.to_le_bytes(), &[0; 12]].concat()); + let number = |r: u16, c: u16, n: f64| rec(0x0203, &[&u16s(&[r, c, 0])[..], &n.to_le_bytes()].concat()); + let formula = |r: u16, c: u16, rgce: &[u8]| { + rec(0x0006, &[&u16s(&[r, c, 0])[..], &0f64.to_le_bytes(), &[0; 6], &(rgce.len() as u16).to_le_bytes(), rgce].concat()) + }; + let area3d = |r0: u16, r1: u16, c0: u16, c1: u16| [&[0x3Bu8, 0, 0][..], &u16s(&[r0, r1, c0, c1])].concat(); + let a2a3 = area3d(1, 2, 0, 0); + let mut data = bof(0x0010); + data.extend([number(0, 0, 11.0), number(1, 0, 5.0), number(2, 0, 7.0)].concat()); + (1..7).for_each(|c| data.extend(number(0, c, 1.0))); + data.extend(rec(0x000A, &[])); + let mut report = bof(0x0010); + (1..7).for_each(|c| report.extend(number(0, c, 0.0))); + // A1 = Data!A1:G1; B2 = Data!$A$2:$A$3*10; C2 = SQRT(Data!$A$2:$A$3) (row 2: SQRT(5)); + // H1 = {SUM(Data!$A$2:$A$3*2)}, a one-cell Ctrl+Shift+Enter array. + let cse = [&a2a3[..], &[0x1E, 2, 0, 0x05, 0x22, 1, 4, 0]].concat(); + report.extend(formula(0, 0, &area3d(0, 0, 0xC000, 6 | 0xC000))); + report.extend(formula(1, 1, &[&a2a3[..], &[0x1E, 10, 0, 0x05]].concat())); + report.extend(formula(1, 2, &[&a2a3[..], &[0x21, 20, 0]].concat())); + report.extend(formula(0, 7, &[1, 0, 0, 7, 0])); + report.extend(rec(0x0221, &[&u16s(&[0, 0])[..], &[7, 7], &[0; 6], &(cse.len() as u16).to_le_bytes(), &cse].concat())); + report.extend(rec(0x000A, &[])); + let boundsheet = |pos: u32, name: &str| rec(0x0085, &[&pos.to_le_bytes()[..], &[0, 0, name.len() as u8, 0], name.as_bytes()].concat()); + let globals = |p1: u32, p2: u32| { + [ + bof(0x0005), + boundsheet(p1, "Data"), + boundsheet(p2, "Report"), + rec(0x01AE, &u16s(&[2, 0x0401])), + rec(0x0017, &u16s(&[1, 0, 0, 0])), + rec(0x000A, &[]), + ] + .concat() + }; + let p1 = globals(0, 0).len() as u32; + let stream = [globals(p1, p1 + data.len() as u32), data, report].concat(); + let bytes = gridcraft_xlsx::testing::compound_file(&[("Workbook", &stream)]); + + let mut s = s(); + s.execute("file.open", json!({"name": "legacy.xls", "base64": crate::io::base64_encode(&bytes)})).unwrap(); + s.execute("formulas.calculateNow", json!({})).unwrap(); + let report = |s: &Session, a1: &str| s.doc().unwrap().wb.sheets[1].value(CellRef::parse(a1).unwrap()); + assert_eq!(report(&s, "A1"), Value::Number(11.0)); + assert_eq!(report(&s, "B2"), Value::Number(50.0)); + assert_eq!(report(&s, "C2"), Value::Number(5f64.sqrt())); + assert_eq!(report(&s, "H1"), Value::Number(24.0)); + // Nothing spilled out of A1 or B2. + assert_eq!(report(&s, "B1"), Value::Number(0.0)); + assert_eq!(report(&s, "B3"), Value::Empty); + // Saved as .xlsx, the formulas carry no dynamic-array metadata. + let r = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap(); + let saved = crate::io::base64_decode(r["base64"].as_str().unwrap()).unwrap(); + let (wb, _) = gridcraft_xlsx::read_xlsx(&saved).unwrap(); + let f = |a1: &str| wb.sheets[1].cell(CellRef::parse(a1).unwrap()).and_then(|c| c.formula.clone()).unwrap(); + assert_eq!(f("A1").text, "@Data!A1:G1"); + assert_eq!((f("H1").text.as_str(), f("H1").array), ("SUM(Data!$A$2:$A$3*2)", gridcraft_core::RangeRef::parse("H1"))); + let mut sheet = String::new(); + std::io::Read::read_to_string( + &mut zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap().by_name("xl/worksheets/sheet2.xml").unwrap(), + &mut sheet, + ) + .unwrap(); + assert!(!sheet.contains("cm=") && !sheet.contains("SINGLE"), "{sheet}"); +} + #[test] fn malformed_xlsb_keeps_the_current_workbook() { let mut s = s(); diff --git a/crates/engine/tests/legacy_array_functions.rs b/crates/engine/tests/legacy_array_functions.rs index 1549b6f70..6e86f2e13 100644 --- a/crates/engine/tests/legacy_array_functions.rs +++ b/crates/engine/tests/legacy_array_functions.rs @@ -228,6 +228,28 @@ fn legacy_unions_names_and_parentheses_calculate_and_save_as_read() { assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}"); } +/// A whole column in a scalar argument of a legacy formula meets the formula's row (issue #1636, +/// from PR #1660): `ISBLANK(B:B)` in row 4 tests `B4`, so the formula shows `C4` instead of +/// spilling into `G5`, and in row 6, where `B6` is blank, shows 0. +#[test] +fn whole_column_in_a_legacy_scalar_argument_meets_the_formulas_row() { + let formulas: Vec<(String, String)> = [ + ("B4", r#""Thierry""#), + ("C4", "7"), + ("G4", r#"IF(ISBLANK(B:B),0,IF(B4="Thierry",C4,0))"#), + ("G5", "1"), + ("G6", r#"IF(ISBLANK(B:B),0,IF(B6="Thierry",C6,0))"#), + ] + .iter() + .map(|(a, f)| (a.to_string(), f.to_string())) + .collect(); + let s = open(&legacy_xlsx(&formulas, "")); + let at = |a: &str| CellRef::parse(a).unwrap(); + assert_eq!(value(&s, at("G4")), Value::Number(7.0), "B:B meets row 4 instead of spilling"); + assert_eq!(value(&s, at("G5")), Value::Number(1.0), "nothing spills below G4"); + assert_eq!(one_value(&s, at("G6"), "G6").unwrap(), Value::Number(0.0), "B6 is blank"); +} + /// One-cell ranges written as ranges (`B1:B1`) outside any `@`, and formulas whose reading /// reaches the parser's nesting limit (254 nested functions, far past Excel's 64): saved as read, /// without `@`. Only a one-cell range under an `@` counts as one an edit may have shrunk. All but @@ -302,9 +324,18 @@ fn legacy_xlsx(formulas: &[(String, String)], names: &str) -> Vec { let esc = |s: &str| s.replace('&', "&").replace('<', "<").replace('>', ">").replace('"', """); let mut rows = String::from(r#"1234"#); + // Cells in one row share its ``, in the order given. + let mut by_row: Vec<(String, String)> = Vec::new(); for (a, f) in formulas { let r: String = a.chars().filter(char::is_ascii_digit).collect(); - rows.push_str(&format!(r#"{}"#, esc(f))); + let c = format!(r#"{}"#, esc(f)); + match by_row.iter_mut().find(|(row, _)| *row == r) { + Some((_, cells)) => cells.push_str(&c), + None => by_row.push((r, c)), + } + } + for (r, cells) in by_row { + rows.push_str(&format!(r#"{cells}"#)); } let names = if names.is_empty() { String::new() } else { format!("{names}") }; let parts: [(&str, String); 5] = [ diff --git a/crates/xlsx/src/array_formula.rs b/crates/xlsx/src/array_formula.rs index 8d871dd75..804192969 100644 --- a/crates/xlsx/src/array_formula.rs +++ b/crates/xlsx/src/array_formula.rs @@ -88,11 +88,11 @@ pub(crate) enum Export { Dynamic, } -pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo) -> Export { +pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo, names: &mut NameCache) -> Export { let Some(expr) = formula.expr_arc() else { return Export::Unchanged }; let mut explicit = false; expr.walk(&mut |e| explicit |= matches!(e, Expr::Unary(UnOp::At, _))); - let mut legacy = Legacy::new(wb, at, functions); + let mut legacy = Legacy::new(wb, at, functions, names); if !explicit { legacy.formula(expr.as_ref().clone(), sheet); return if !legacy.certain { @@ -119,7 +119,7 @@ pub(crate) fn export(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula candidates.insert(0, unwrap_at(x.as_ref().clone())); } for text in candidates { - let read = self::legacy(wb, sheet, at, &Formula::from_expr(text.clone()), functions); + let read = self::legacy(wb, sheet, at, &Formula::from_expr(text.clone()), functions, names); if read.map_or(text == *expr, |f| f.expr_arc().is_some_and(|e| *e == *expr)) { return reads_back(text).map_or(Export::Dynamic, Export::Legacy); } @@ -140,12 +140,14 @@ fn remove_at(e: Expr) -> Expr { /// Rewrites the legacy cell formulas at `cells` (model sheet index, positions) to their /// dynamic-array reading, with `@` where Excel intersects. Runs once names and tables are loaded. pub(crate) fn apply_legacy(wb: &mut Workbook, cells: Vec<(usize, Vec)>, functions: FunctionInfo) { + // Defined names do not change during the pass: each is analysed once for all the cells. + let mut names = NameCache::default(); for (si, positions) in cells { let Some(sheet) = wb.sheet(si) else { continue }; let mut changed = Vec::new(); for at in positions { let Some(formula) = sheet.cell(at).and_then(|c| c.formula.as_deref()) else { continue }; - if let Some(f) = legacy(wb, si, at, formula, functions) { + if let Some(f) = legacy(wb, si, at, formula, functions, &mut names) { changed.push((at, f)); } } @@ -162,12 +164,12 @@ pub(crate) fn apply_legacy(wb: &mut Workbook, cells: Vec<(usize, Vec)>, } /// The dynamic-array reading of a legacy formula, when it differs. -fn legacy(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo) -> Option { +fn legacy(wb: &Workbook, sheet: usize, at: CellRef, formula: &Formula, functions: FunctionInfo, names: &mut NameCache) -> Option { if formula.array.is_some() { return None; } let expr = formula.expr_arc()?; - let mut legacy = Legacy::new(wb, at, functions); + let mut legacy = Legacy::new(wb, at, functions, names); let out = legacy.formula(expr.as_ref().clone(), sheet); if legacy.certain && legacy.complete { if legacy.inserted == 0 { @@ -228,8 +230,56 @@ fn atomic(e: &Expr) -> bool { /// (see [`Legacy::binary`]). const MAX_DEPTH: usize = 256; +/// What the analysis of one defined name leaves for a later formula that refers to it. +#[derive(Clone)] +struct NameInfo { + kind: Kind, + /// The definition reached a one-cell reference written as a range. + one_cell_range: bool, + /// The analysis covered only established legacy semantics. + certain: bool, + /// Nodes of the per-formula budget the analysis used. + cost: usize, + /// Nesting below the reference that the analysis reached. + depth: usize, + /// Every name the analysis read, with its result (itself last): a hit records them all, as a + /// fresh analysis does, so a later reference to one of them in the formula costs nothing. + closure: Vec<(NameKey, Kind, bool)>, +} + +/// A defined name as [`Legacy::names`] keys it: upper-case name, scope, collapsed. +type NameKey = (String, usize, bool); + +/// Defined names analysed so far, shared by the formulas of one pass (reading a workbook, or +/// writing one) so that a name is parsed and analysed once, not once per formula that refers +/// to it. Keyed like [`Legacy::names`]. Only analyses that finished within their limits, did not +/// depend on the formula's own position and did not reuse a name the formula had already read +/// (or meet a cycle) are kept. A hit charges the budget and nesting the analysis used and records +/// every name it read, and is taken only when the formula can afford it, so a formula reads the +/// same with the cache as without it. +#[derive(Default)] +pub(crate) struct NameCache { + map: HashMap, + /// How many names were analysed (not found in the cache). + pub(crate) analyses: usize, +} + +/// Names kept at most: the cache is bounded even when a workbook defines a great many. +const MAX_CACHED_NAMES: usize = 65_536; + struct Legacy<'a> { wb: &'a Workbook, + /// Name analyses shared with the other formulas of this pass. + shared: &'a mut NameCache, + /// The analysis read a structured reference without a table name, which depends on `at`. + position_dependent: bool, + /// The analysis reused a name this formula had already read, or met a cycle: its result + /// leaves out what that name cost. + reused: bool, + /// The names read so far in this formula, in order, with their results. + read: Vec<(NameKey, Kind, bool)>, + /// The deepest nesting reached. + deepest: usize, at: CellRef, functions: FunctionInfo, remaining: usize, @@ -262,9 +312,14 @@ struct Legacy<'a> { } impl<'a> Legacy<'a> { - fn new(wb: &'a Workbook, at: CellRef, functions: FunctionInfo) -> Self { + fn new(wb: &'a Workbook, at: CellRef, functions: FunctionInfo, shared: &'a mut NameCache) -> Self { Legacy { wb, + shared, + position_dependent: false, + reused: false, + read: Vec::new(), + deepest: 0, at, functions, remaining: 16384, @@ -364,6 +419,7 @@ impl<'a> Legacy<'a> { fn expr(&mut self, e: Expr, sheet: usize, ctx: Ctx, depth: usize) -> (Expr, Kind) { // Names can be cyclic or nest deeply; their kinds are remembered, so repeated names // cost one analysis each. + self.deepest = self.deepest.max(depth); if depth >= MAX_DEPTH || self.remaining == 0 { self.complete = false; return (e, Kind::Array); @@ -407,6 +463,7 @@ impl<'a> Legacy<'a> { } Expr::Struct(r) => { let table = if r.table.is_empty() { + self.position_dependent = true; self.wb.sheet(sheet).and_then(|s| s.table_at(self.at)) } else { self.wb.table(&r.table).and_then(|(si, ti)| self.wb.sheet(si).and_then(|s| s.tables.get(ti))) @@ -459,25 +516,73 @@ impl<'a> Legacy<'a> { let key = (bare.to_ascii_uppercase(), scope, self.collapsed); match self.names.get(&key) { Some(Some((kind, one_cell_range))) => { + self.reused = true; self.one_cell_range |= *one_cell_range; return *kind; } Some(None) => { // A name defined in terms of itself has no legacy value to keep. self.certain = false; + self.reused = true; return Kind::Array; } None => {} } + let reached = depth.saturating_add(1); + // Taken only when the formula can afford it and has read none of the names it read: + // otherwise the fresh analysis below differs (cut short, or reusing those names). + let hit = self.shared.map.get(&key).filter(|i| { + reached + i.depth < MAX_DEPTH && i.cost <= self.remaining && i.closure.iter().all(|(k, _, _)| !self.names.contains_key(k)) + }); + if let Some(info) = hit { + // Charged and recorded as the analysis was: the formula reads as a fresh one does. + self.deepest = self.deepest.max(reached + info.depth); + self.certain &= info.certain; + self.one_cell_range |= info.one_cell_range; + self.remaining -= info.cost; + for (k, kind, one_cell_range) in &info.closure { + self.names.insert(k.clone(), Some((*kind, *one_cell_range))); + } + self.read.extend(info.closure.iter().cloned()); + return info.kind; + } let Ok(expr) = gridcraft_formula::parse_name(def.formula.trim_start_matches('=')) else { return Kind::Array }; + self.shared.analyses += 1; self.names.insert(key.clone(), None); // A name's own formula does not see the formula's LET variables. let locals = std::mem::take(&mut self.locals); let seen = std::mem::replace(&mut self.one_cell_range, false); + let before = self.remaining; + let first_read = self.read.len(); + let outer = ( + std::mem::replace(&mut self.certain, true), + std::mem::replace(&mut self.complete, true), + std::mem::replace(&mut self.position_dependent, false), + std::mem::replace(&mut self.reused, false), + std::mem::replace(&mut self.deepest, 0), + ); let kind = self.kind_of(expr, def.scope.unwrap_or(scope), depth + 1); let one_cell_range = std::mem::replace(&mut self.one_cell_range, seen); + self.read.push((key.clone(), kind, one_cell_range)); + let cacheable = self.complete && !self.position_dependent && !self.reused; + let info = cacheable.then(|| NameInfo { + kind, + one_cell_range, + certain: self.certain, + cost: before.saturating_sub(self.remaining), + depth: self.deepest.saturating_sub(depth + 1), + closure: self.read[first_read..].to_vec(), + }); + self.certain &= outer.0; + self.complete &= outer.1; + self.position_dependent |= outer.2; + self.reused |= outer.3; + self.deepest = self.deepest.max(outer.4); self.one_cell_range |= one_cell_range; self.locals = locals; + if let Some(info) = info.filter(|_| self.shared.map.len() < MAX_CACHED_NAMES) { + self.shared.map.insert(key.clone(), info); + } self.names.insert(key, Some((kind, one_cell_range))); return kind; } diff --git a/crates/xlsx/src/lib.rs b/crates/xlsx/src/lib.rs index 259a1a1a6..773e2895f 100644 --- a/crates/xlsx/src/lib.rs +++ b/crates/xlsx/src/lib.rs @@ -52,7 +52,7 @@ pub use csv::{MAX_CSV_BYTES, decode_text, read_csv, read_csv_with_report, write_ pub use ods::read_ods; pub use package::MAX_PART; pub use passthrough::rebase_links; -pub use xls::read_xls; +pub use xls::{read_xls, read_xls_with}; pub use xlsb::read_xlsb; /// Errors from reading or writing files. diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index d3ba1426b..c405667b1 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -652,7 +652,7 @@ fn write_sheet_data(s: &mut String, data: SheetCellData<'_>, sid: &dyn Fn(gridcr if f.dynamic { return Some(RangeRef::cell(cell_ref)); } - match crate::array_formula::export(workbook, index, cell_ref, f, out.functions) { + match crate::array_formula::export(workbook, index, cell_ref, f, out.functions, &mut out.names) { crate::array_formula::Export::Unchanged => None, crate::array_formula::Export::Dynamic => Some(RangeRef::cell(cell_ref)), // Every `@` is one Excel implies in a legacy formula: write it without them. diff --git a/crates/xlsx/src/tests/array_export.rs b/crates/xlsx/src/tests/array_export.rs index 9c7f0700a..9e60f0373 100644 --- a/crates/xlsx/src/tests/array_export.rs +++ b/crates/xlsx/src/tests/array_export.rs @@ -467,6 +467,138 @@ fn implicit_intersection_metadata_distinguishes_table_rows_and_columns() { ); } +/// How `export` saved a formula, as comparable text. +fn export_text(wb: &Workbook, at: CellRef, text: &str, names: &mut crate::array_formula::NameCache) -> String { + use crate::array_formula::{Export, export}; + match export(wb, 0, at, &Formula::new(text), crate::FunctionInfo::default(), names) { + Export::Unchanged => "unchanged".into(), + Export::Dynamic => "dynamic".into(), + Export::Legacy(e) => format!("legacy {}", gridcraft_formula::print(&e)), + } +} + +#[test] +fn defined_names_are_analysed_once_for_many_formulas() { + use crate::array_formula::NameCache; + let mut wb = Workbook::new(); + // A large definition: many operations, ending in a range. + let big = (1..=500).map(|i| format!("Sheet1!$A${i}")).collect::>().join("+"); + for (name, text) in [("Big", format!("({big})*Sheet1!$B$1:$B$3")), ("Rng", "Sheet1!$A$1:$A$3".to_string()), ("Alias", "Rng".to_string())] { + wb.names.push(DefinedName { name: name.into(), scope: None, formula: text, comment: String::new(), hidden: false }); + } + let texts = ["SUM(Big)", "SQRT(Big)", "Big*2", "SQRT(Alias)", "SUM(Alias)", "Alias", "SUM(Rng*2)", "Missing*2"]; + let mut shared = NameCache::default(); + for i in 0..400u32 { + for text in texts { + let at = CellRef::new(i + 10, 5); + let with_shared = export_text(&wb, at, text, &mut shared); + let fresh = export_text(&wb, at, text, &mut NameCache::default()); + assert_eq!(with_shared, fresh, "{text}"); + } + } + assert!(gridcraft_formula::parse(&wb.names[0].formula).is_ok(), "the large name must parse"); + // 3 names, each analysed once for all 3200 formulas (a fresh cache analyses a name per formula). + assert_eq!(shared.analyses, 3); +} + +#[test] +fn shared_name_analysis_keeps_each_formulas_limits_and_position() { + use crate::array_formula::NameCache; + let mut wb = Workbook::new(); + let terms = |n: usize| (1..=n).map(|i| format!("Sheet1!$A${i}")).collect::>().join("+"); + // Twenty names of about a thousand nodes each: a formula using all of them needs more than + // its 16,384-node budget, one using a few does not. + for i in 0..20 { + let text = format!("({})*Sheet1!$B$1:$B$3", terms(500)); + wb.names.push(DefinedName { name: format!("Big_{i}"), scope: None, formula: text, comment: String::new(), hidden: false }); + } + for (name, text) in [ + ("Cyc1", "Cyc2".to_string()), + ("Cyc2", "Cyc1".to_string()), + ("Deep", format!("{}1{}", "(".repeat(100), ")".repeat(100))), + ("Tbl", "Table1[Col]".to_string()), + ] { + wb.names.push(DefinedName { name: name.into(), scope: None, formula: text, comment: String::new(), hidden: false }); + } + assert!(wb.names.iter().all(|n| gridcraft_formula::parse(&n.formula).is_ok()), "every name must parse"); + let uses = |n: usize| (0..n).map(|i| format!("SQRT(Big_{i})")).collect::>().join("+"); + let mut texts: Vec = [1, 5, 10, 15, 17, 20].iter().map(|n| uses(*n)).collect(); + texts.extend(["SQRT(Big_3)", "Cyc1", "SUM(Cyc1)", "SUM(Deep)", "SQRT(Tbl)", "((((((((SQRT(Deep))))))))"].map(String::from)); + // The limit is really reached: a few uses analyse fully, all twenty stop at the budget. + let probe = |n: usize| export_text(&wb, CellRef::new(5, 5), &uses(n), &mut NameCache::default()); + assert_eq!(probe(1), "dynamic"); + assert_eq!(probe(20), "unchanged"); + let mut shared = NameCache::default(); + for round in 0..3 { + for text in &texts { + for at in [CellRef::new(5, 5), CellRef::new(7, 9)] { + let with_shared = export_text(&wb, at, text, &mut shared); + let fresh = export_text(&wb, at, text, &mut NameCache::default()); + assert_eq!(with_shared, fresh, "round {round}: {}", &text[..text.len().min(60)]); + } + } + } +} + +/// A name defined as another name reads the same from the shared cache in either order: after +/// the other name (whose cost and uncertainty the formula already paid) and before it (a hit +/// records the other name too, so the formula does not pay for it twice). +#[test] +fn shared_name_analysis_of_names_defined_by_names_matches_fresh_in_both_orders() { + use crate::array_formula::NameCache; + let mut wb = Workbook::new(); + let terms = (1..=500).map(|i| format!("Sheet1!$A${i}")).collect::>().join("+"); + // `Unsure` is outside the established legacy semantics (ISOMITTED), and `Alias` is it by name. + let mut defs = vec![ + ("Unsure".to_string(), "IF(ISOMITTED(Sheet1!$C$1),Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)".to_string()), + ("Alias".to_string(), "Unsure".to_string()), + ]; + for i in 0..10 { + defs.push((format!("Big_{i}"), format!("({terms})*Sheet1!$B$1:$B$3"))); + defs.push((format!("Via_{i}"), format!("Big_{i}"))); + } + for (name, formula) in defs { + wb.names.push(DefinedName { name, scope: None, formula, comment: String::new(), hidden: false }); + } + // Ten names of about a thousand nodes, each reached twice: within the budget only if the + // second reference is free. + let both = |first: &str, second: &str| (0..10).map(|i| format!("SQRT({first}_{i})+SQRT({second}_{i})")).collect::>().join("+"); + let orders = [ + ["Unsure+SQRT(Alias)", "SQRT(Alias)+Unsure", "SQRT(Alias)+SQRT(Sheet1!$A$1:$A$3)", "@SQRT(Alias)"].map(String::from).to_vec(), + ["SQRT(Alias)", "SQRT(Alias)+Unsure", "Unsure+SQRT(Alias)", "SQRT(Alias)+SQRT(Sheet1!$A$1:$A$3)", "@SQRT(Alias)"].map(String::from).to_vec(), + vec![both("Big", "Via"), both("Via", "Big"), "SQRT(Via_0)".to_string(), both("Via", "Big")], + vec!["SQRT(Via_0)".to_string(), both("Via", "Big"), both("Big", "Via"), both("Via", "Big")], + ]; + let at = CellRef::new(5, 5); + assert_eq!(export_text(&wb, at, &both("Via", "Big"), &mut NameCache::default()), "dynamic", "fresh fits the budget"); + for texts in orders { + let mut shared = NameCache::default(); + for text in &texts { + let with_shared = export_text(&wb, at, text, &mut shared); + let fresh = export_text(&wb, at, text, &mut NameCache::default()); + assert_eq!(with_shared, fresh, "{}", &text[..text.len().min(60)]); + } + // Reading: one pass over the formulas, one row each, against each formula read alone. + let read = |cells: &[(CellRef, &String)], one_pass: bool| -> Vec { + let mut wb = wb.clone(); + for (c, text) in cells { + Arc::make_mut(&mut wb.sheets[0]).cells.set(*c, Cell::formula(Formula::new(text))); + } + let all: Vec = cells.iter().map(|(c, _)| *c).collect(); + if one_pass { + crate::array_formula::apply_legacy(&mut wb, vec![(0, all.clone())], crate::FunctionInfo::default()); + } else { + for c in &all { + crate::array_formula::apply_legacy(&mut wb, vec![(0, vec![*c])], crate::FunctionInfo::default()); + } + } + all.iter().map(|c| gridcraft_formula::print(&wb.sheets[0].cells.get(*c).unwrap().formula.as_ref().unwrap().expr_arc().unwrap())).collect() + }; + let cells: Vec<(CellRef, &String)> = texts.iter().enumerate().map(|(i, t)| (CellRef::new(i as u32 + 5, 5), t)).collect(); + assert_eq!(read(&cells, true), read(&cells, false)); + } +} + #[test] fn long_flat_chains_are_scalar_formulas() { // `=A1+A2+…` nests one level per operator. Past the scan's depth limit of 64, a plain sum diff --git a/crates/xlsx/src/tests/xls_tests.rs b/crates/xlsx/src/tests/xls_tests.rs index bd21b4939..762e512e0 100644 --- a/crates/xlsx/src/tests/xls_tests.rs +++ b/crates/xlsx/src/tests/xls_tests.rs @@ -508,8 +508,11 @@ fn sheets_share_one_record_budget() { stream.extend(eof()); let file = compound("Workbook", &stream); // About n × 2,000 records in all: over a 100,000 shared allowance, though each sheet alone fits. - assert!(matches!(crate::xls::read_xls_with(&file, crate::xls::Budget::with_max_records(100_000)), Err(IoError::TooLarge(_)))); - assert!(crate::xls::read_xls_with(&file, crate::xls::Budget::with_max_records(10_000_000)).is_ok()); + assert!(matches!( + crate::xls::read_xls_budgeted(&file, crate::xls::Budget::with_max_records(100_000), crate::FunctionInfo::default()), + Err(IoError::TooLarge(_)) + )); + assert!(crate::xls::read_xls_budgeted(&file, crate::xls::Budget::with_max_records(10_000_000), crate::FunctionInfo::default()).is_ok()); // Two sheets at the same offset are refused outright. let mut dup = bof(0x0005); @@ -625,6 +628,112 @@ fn a_format_shorter_than_its_count_keeps_its_characters() { assert!(report.warnings.iter().any(|w| w.contains("declared length")), "{:?}", report.warnings); } +/// A workbook whose "Report" sheet holds legacy formulas over ranges of "Data": Data!A1 = 11, +/// B1:G1 = 1, A2 = 5, A3 = 7, and Report!B1:G1 filled, so a formula in Report!A1 that spilled +/// would be blocked. +fn legacy_formulas_book() -> Vec { + let area3d = |r0: u16, r1: u16, c0: u16, c1: u16| cat(&[&[0x3B], &0u16.to_le_bytes(), &u16s(&[r0, r1, c0, c1])]); + let a2a3 = area3d(1, 2, 0, 0); + let sum = |inner: &[u8]| cat(&[inner, &[0x22, 1], &4u16.to_le_bytes()]); + let boundsheet = |pos: u32, name: &str| rec(0x0085, &cat(&[&pos.to_le_bytes(), &[0, 0, name.len() as u8, 0], name.as_bytes()])); + let globals = |p1: u32, p2: u32| { + cat(&[ + &bof(0x0005), + &boundsheet(p1, "Data"), + &boundsheet(p2, "Report"), + &rec(0x01AE, &u16s(&[2, 0x0401])), + &rec(0x0017, &cat(&[&u16s(&[1]), &u16s(&[0, 0, 0])])), + &eof(), + ]) + }; + let number = |r: u16, c: u16, n: f64| rec(0x0203, &cat(&[&cell(r, c, 0), &n.to_le_bytes()])); + let mut data = bof(0x0010); + data.extend(number(0, 0, 11.0)); + for c in 1..7 { + data.extend(number(0, c, 1.0)); + } + data.extend(number(1, 0, 5.0)); + data.extend(number(2, 0, 7.0)); + data.extend(eof()); + let shared = cat(&[&a2a3, &ptg_int(10), &[0x05]]); + let array = cat(&[&a2a3, &ptg_int(2), &[0x05]]); + let single_array = sum(&cat(&[&a2a3, &ptg_int(2), &[0x05]])); + let mut report = bof(0x0010); + for c in 1..7 { + report.extend(number(0, c, 0.0)); + } + report.extend(cat(&[ + // A1 = Data!A1:G1, A2 = +Data!A1:G1 (a title row repeated from another sheet). + &formula(0, 0, num(11.0), &area3d(0, 0, 0xC000, 6 | 0xC000)), + &formula(1, 0, num(11.0), &cat(&[&area3d(0, 0, 0xC000, 6 | 0xC000), &[0x12]])), + // C2:C3 share Data!$A$2:$A$3*10. + &formula(1, 2, num(50.0), &exp(1, 2)), + &rec(0x04BC, &cat(&[&u16s(&[1, 2]), &[2, 2, 0, 2], &(shared.len() as u16).to_le_bytes(), &shared])), + &formula(2, 2, num(70.0), &exp(1, 2)), + // D2:D3 = {Data!$A$2:$A$3*2}, entered with Ctrl+Shift+Enter. + &formula(1, 3, num(10.0), &exp(1, 3)), + &rec(0x0221, &cat(&[&u16s(&[1, 2]), &[3, 3], &[0; 6], &(array.len() as u16).to_le_bytes(), &array])), + &formula(2, 3, num(14.0), &exp(1, 3)), + // H1 = {SUM(Data!$A$2:$A$3*2)}, a one-cell Ctrl+Shift+Enter array. + &formula(0, 7, num(24.0), &exp(0, 7)), + &rec(0x0221, &cat(&[&u16s(&[0, 0]), &[7, 7], &[0; 6], &(single_array.len() as u16).to_le_bytes(), &single_array])), + // H2 = MAX(SUM(Data!$A$2:$A$3/12),0), H3 = SUM(Data!$A$2:$A$3), H4 = SQRT(Data!$A$2:$A$3). + &formula(1, 7, num(5.0 / 12.0), &cat(&[&sum(&cat(&[&a2a3, &ptg_int(12), &[0x06]])), &ptg_int(0), &[0x22, 2], &7u16.to_le_bytes()])), + &formula(2, 7, num(12.0), &sum(&a2a3)), + &formula(3, 7, special(2, 0x0F), &cat(&[&a2a3, &[0x21], &20u16.to_le_bytes()])), + &eof(), + ])); + let p1 = globals(0, 0).len() as u32; + let p2 = p1 + data.len() as u32; + cat(&[&globals(p1, p2), &data, &report]) +} + +#[test] +fn legacy_formulas_read_with_implicit_intersection() { + let bytes = compound("Workbook", &legacy_formulas_book()); + let formula = |wb: &gridcraft_model::Workbook, a1: &str| { + let f = wb.sheets[1].cell(CellRef::parse(a1).unwrap()).and_then(|c| c.formula.clone()).unwrap(); + (f.text.clone(), f.array) + }; + let range = |a1: &str| RangeRef::parse(a1); + let (wb, _) = read_xls(&bytes).unwrap(); + // Every .xls formula is legacy: a range where one value is expected is intersected. + assert_eq!(formula(&wb, "A1"), ("@Data!A1:G1".into(), None)); + assert_eq!(formula(&wb, "A2"), ("+@Data!A1:G1".into(), None)); + // Shared formulas too, each cell on its own. + assert_eq!(formula(&wb, "C2"), ("@Data!$A$2:$A$3*10".into(), None)); + assert_eq!(formula(&wb, "C3"), ("@Data!$A$2:$A$3*10".into(), None)); + // Ctrl+Shift+Enter arrays, one cell or several, stay arrays and are not intersected. + assert_eq!(formula(&wb, "D2"), ("Data!$A$2:$A$3*2".into(), range("D2:D3"))); + assert_eq!(formula(&wb, "H1"), ("SUM(Data!$A$2:$A$3*2)".into(), range("H1"))); + // An operation inside SUM takes values; SUM of the range itself is left alone. + assert_eq!(formula(&wb, "H2"), ("MAX(SUM(@Data!$A$2:$A$3/12),0)".into(), None)); + assert_eq!(formula(&wb, "H3"), ("SUM(Data!$A$2:$A$3)".into(), None)); + assert_eq!(formula(&wb, "H4"), ("SQRT(@Data!$A$2:$A$3)".into(), None)); + + // The caller's function facts decide: here SQRT takes its argument as a whole. + let library = crate::FunctionInfo { is_builtin: |_| true, is_scalar_arg: |n, _| n != "SQRT" }; + let (custom, _) = crate::read_xls_with(&bytes, library).unwrap(); + assert_eq!(formula(&custom, "H4"), ("SQRT(Data!$A$2:$A$3)".into(), None)); + assert_eq!(formula(&custom, "A1"), ("@Data!A1:G1".into(), None)); + + // Saved as .xlsx, they stay legacy: no dynamic-array metadata, no _xlfn.SINGLE, and they read + // back as they were opened. + let saved = write_xlsx(&wb).unwrap(); + let mut sheet = String::new(); + std::io::Read::read_to_string( + &mut zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap().by_name("xl/worksheets/sheet2.xml").unwrap(), + &mut sheet, + ) + .unwrap(); + assert!(!sheet.contains("cm=") && !sheet.contains("SINGLE"), "{sheet}"); + assert!(sheet.contains("Data!A1:G1") && sheet.contains("MAX(SUM(Data!$A$2:$A$3/12),0)"), "{sheet}"); + let (back, _) = read_xlsx(&saved).unwrap(); + for a1 in ["A1", "A2", "C2", "C3", "D2", "H1", "H2", "H3", "H4"] { + assert_eq!(formula(&back, a1), formula(&wb, a1), "{a1}"); + } +} + fn one_sheet_with(sheet_records: &[u8]) -> Vec { let globals = |pos: u32| cat(&[&bof(0x0005), &rec(0x0085, &cat(&[&pos.to_le_bytes(), &[0, 0, 6, 0], b"Sheet1"])), &eof()]); let sheet = cat(&[&bof(0x0010), sheet_records, &rec(0x0203, &cat(&[&cell(0, 0, 0), &7.0f64.to_le_bytes()])), &eof()]); diff --git a/crates/xlsx/src/write.rs b/crates/xlsx/src/write.rs index 5d06d54a0..512144980 100644 --- a/crates/xlsx/src/write.rs +++ b/crates/xlsx/src/write.rs @@ -134,6 +134,8 @@ pub struct Out { pub kept: crate::passthrough::KeptOut, /// Decides which formulas need dynamic-array metadata. pub functions: FunctionInfo, + /// Defined names analysed for implicit intersection, shared by every formula written. + pub(crate) names: crate::array_formula::NameCache, /// The code of the Python cells written so far, in order: script `i` of the Python part. pub python_scripts: Vec, } @@ -271,6 +273,7 @@ pub fn write_xlsx_named_with(wb: &Workbook, kind: XlsxKind, book_name: Option<&s cell_pictures: Default::default(), kept: Default::default(), functions, + names: Default::default(), python_scripts: vec![], }; out.kept.reserve_vml_blocks(wb); diff --git a/crates/xlsx/src/xls.rs b/crates/xlsx/src/xls.rs index 89f4a31f7..1c9285bae 100644 --- a/crates/xlsx/src/xls.rs +++ b/crates/xlsx/src/xls.rs @@ -10,6 +10,9 @@ //! (charts, pictures, comments, conditional formats, validation, macros) is skipped with a //! warning. //! Encrypted workbooks and pre-BIFF8 versions are refused. +//! +//! Every formula in the file is a legacy (pre-dynamic-array) one, so it is read with Excel's +//! implicit `@` (see `array_formula.rs`), like an `.xlsx` formula saved without `cm`. mod formula; mod ftab; @@ -24,7 +27,7 @@ use gridcraft_model::{CalcMode, CalcSettings, Cell, DefinedName, Formula, LineIn use crate::cfb::Compound; use crate::tables::{col_width_to_px, pt_to_px}; -use crate::{IoError, ReadReport}; +use crate::{FunctionInfo, IoError, ReadReport}; use formula::{Links, Mode, SupBook, Unsupported, Xti, cell_error, decode}; use records::{Cursor, Record, Records}; @@ -141,13 +144,20 @@ struct Globals { precision_as_displayed: bool, } -/// Reads an Excel 97-2003 workbook. Inspect the report's warnings for what was skipped. +/// Reads an Excel 97-2003 workbook, with the default [`FunctionInfo`]. Inspect the report's +/// warnings for what was skipped. pub fn read_xls(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { - read_xls_with(bytes, Budget::default()) + read_xls_budgeted(bytes, Budget::default(), FunctionInfo::default()) } -/// [`read_xls`] with an explicit resource budget. -pub(crate) fn read_xls_with(bytes: &[u8], mut budget: Budget) -> Result<(Workbook, ReadReport), IoError> { +/// [`read_xls`] with the function library's facts, which decide where the legacy formulas get +/// Excel's implicit `@`. +pub fn read_xls_with(bytes: &[u8], functions: FunctionInfo) -> Result<(Workbook, ReadReport), IoError> { + read_xls_budgeted(bytes, Budget::default(), functions) +} + +/// [`read_xls_with`] with an explicit resource budget. +pub(crate) fn read_xls_budgeted(bytes: &[u8], mut budget: Budget, functions: FunctionInfo) -> Result<(Workbook, ReadReport), IoError> { let cf = Compound::open(bytes).map_err(invalid)?; let stream = match cf.root_stream("Workbook").map_err(invalid)? { Some(s) => s, @@ -201,6 +211,10 @@ pub(crate) fn read_xls_with(bytes: &[u8], mut budget: Budget) -> Result<(Workboo } wb.calc.precision_as_displayed = g.precision_as_displayed; defined_names(&g, &model_index, &mut wb, &mut report); + // Every .xls formula is a legacy (pre-dynamic-array) one: read with Excel's implicit `@`, like + // an .xlsx formula without `cm`. Ctrl+Shift+Enter arrays are read as they are. + let legacy = wb.sheets.iter().enumerate().map(|(si, s)| (si, legacy_cells(s))).collect(); + crate::array_formula::apply_legacy(&mut wb, legacy, functions); wb.set_active_sheet(wb.sheets.iter().position(|s| s.visibility == Visibility::Visible).unwrap_or(0)); // The overview comes first. report.warnings.insert( @@ -210,6 +224,11 @@ pub(crate) fn read_xls_with(bytes: &[u8], mut budget: Budget) -> Result<(Workboo Ok((wb, report)) } +/// The cells holding a formula that is not a Ctrl+Shift+Enter array. +fn legacy_cells(sheet: &Sheet) -> Vec { + sheet.cells.iter().filter(|(_, c)| c.formula.as_ref().is_some_and(|f| f.array.is_none())).map(|(at, _)| at).collect() +} + fn bof(rec: &Record<'_>) -> Result<(u16, u16), IoError> { if rec.id != BOF { return Err(invalid("a substream does not start with a BOF record")); @@ -463,7 +482,9 @@ impl SheetState<'_> { self.unsupported("syntax GridCraft can't parse yet"); return; } - f.array = array.filter(|r| !r.is_single()); + // A one-cell Ctrl+Shift+Enter array keeps its range, as when an .xlsx is read: + // it evaluates as an array, not with implicit intersection. + f.array = array; if let Some(cell) = self.sheet.cells.get_mut(pos) { cell.formula = Some(Arc::new(f)); } else { diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 12597f313..40942a539 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -92,7 +92,7 @@ past the parser's limit, and a wrapper inside the text. **Saving writes the real | `.xlsm` Macro-Enabled Workbook | R/W | yes | yes, macros kept, not run | The VBA project is kept byte-for-byte and written back only into `.xlsm`/`.xltm` (#6); macros never run. Saved with the macro-enabled workbook content type | `roundtrip.rs` (`vba_project_roundtrip`, `package_kind_sets_workbook_content_type`), engine `io.rs` (`macros_are_saved_only_into_xlsm`) | | `.xltx` / `.xltm` templates | R/W | yes | partial | Saved with the template (and macro-enabled template) content type, chosen by extension; `.xltm` keeps a VBA project, `.xltx` drops it; "new from template" behaviour missing | `roundtrip.rs` (`package_kind_sets_workbook_content_type`), engine `io.rs` | | `.xlsb` Binary Workbook | R/W | **values only** (#101) | no | Names, visibility, numbers, text, booleans, errors, merges, cached formula results; no formulas, formatting, charts, names | `xlsb_tests.rs` | -| `.xls` Excel 97-2003 (BIFF8) | R/W | **yes, partial** (#121) | no (Save As `.xlsx`) | Clean-room from [MS-CFB] and [MS-XLS]. Read: worksheets and visibility, shared strings, numbers, text, booleans, errors, formulas (text for the common tokens, shared and array formulas, 3-D refs, names, newer functions called by name) with cached results, number formats (dates display as dates), cell formatting (fonts, solid and pattern fills, borders with diagonals, alignment with rotation, indent, wrap and shrink, locked and hidden, the custom colour palette, the Normal font as the workbook default, user-defined cell styles), 1904 date system, merges, column widths, row heights, defined names, print area and titles, calculation settings (mode, iteration with its maximum iterations and change, precision as displayed), sheet display options (right-to-left, gridlines, headings, zero values, formulas; a truncated WINDOW2 is ignored). Not read: exact colours beyond the 56-colour palette (XFExt: theme colours and tints show as the palette colour Excel stored next to them), rich-text runs inside a cell, charts, pictures, comments, CF, validation, hyperlinks, freeze panes, external links (formula kept as its value). Chart and macro sheets skipped with a warning; encrypted (FILEPASS) and BIFF5 (Excel 5.0/95) files refused with a message. Opens as a new unsaved workbook. Unverified on a real-world corpus | `xls_tests.rs`, `xls_style_tests.rs` | +| `.xls` Excel 97-2003 (BIFF8) | R/W | **yes, partial** (#121) | no (Save As `.xlsx`) | Clean-room from [MS-CFB] and [MS-XLS]. Read: worksheets and visibility, shared strings, numbers, text, booleans, errors, formulas (text for the common tokens, shared and array formulas, 3-D refs, names, newer functions called by name) with cached results, number formats (dates display as dates), cell formatting (fonts, solid and pattern fills, borders with diagonals, alignment with rotation, indent, wrap and shrink, locked and hidden, the custom colour palette, the Normal font as the workbook default, user-defined cell styles), 1904 date system, merges, column widths, row heights, defined names, print area and titles, calculation settings (mode, iteration with its maximum iterations and change, precision as displayed), sheet display options (right-to-left, gridlines, headings, zero values, formulas; a truncated WINDOW2 is ignored). Every formula except a Ctrl+Shift+Enter array is a legacy one and opens with Excel's implicit `@`, as an `.xlsx` formula without `cm` does (#410); Ctrl+Shift+Enter arrays, one cell or several, stay arrays, and saving as `.xlsx` writes the formulas without `cm`. Not read: exact colours beyond the 56-colour palette (XFExt: theme colours and tints show as the palette colour Excel stored next to them), rich-text runs inside a cell, charts, pictures, comments, CF, validation, hyperlinks, freeze panes, external links (formula kept as its value). Chart and macro sheets skipped with a warning; encrypted (FILEPASS) and BIFF5 (Excel 5.0/95) files refused with a message. Opens as a new unsaved workbook. Unverified on a real-world corpus | `xls_tests.rs`, `xls_style_tests.rs` | | `.xlt` Excel 97-2003 template | R/W | yes (read as `.xls`) | no | | — | | `.xlam` / `.xla` add-ins | R/W | no | no | VBA; out of scope | — | | `.xml` XML Spreadsheet 2003 | R/W | no | no | Simple to support | — | @@ -161,6 +161,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | Date | Change | Summary | |---|---|---| +| 2026-11-10 | minor | `.xls` import: every formula except a Ctrl+Shift+Enter array is read as a legacy one with Excel's implicit `@` (a title row `=Other!A1:G1` shows its first cell instead of `#SPILL!`), one-cell Ctrl+Shift+Enter arrays stay arrays, and `@` of a 2-D range meets the formula's row and column (#410, PR #1367) | | 2026-11-02 | minor | XLSX: buttons, scroll bars and spin buttons read and written (control properties parts, ``, VML `x:ClientData`); other form controls stay with pass-through; a button's command, fixed input or output cell travels in a hidden defined name and comes back only while the file still assigns the button's macro (or none, for a command) | | 2026-11-03 | minor | SYLK (`.slk`) import: values, R1C1 formulas as A1 (shared formulas too) with cached results, number formats and column widths; import only (#1481) | | 2026-11-03 | minor | Opening a web page (HTML) reads its first table as a sheet, whatever the file is called: a web app's "Export to Excel" saved as `.xls` no longer opens as lines of markup. Values read as in a CSV (or as text with `csvAsText`), `` cells are bold and centred, `` names the sheet, and a page not named `.htm`/`.html` opens with a warning (#3166) | From 457bc7e374afbcd9a99c4f4c52c0017149ec8bb9 Mon Sep 17 00:00:00 2001 From: FredApps <153173854+FredApps@users.noreply.github.com> Date: Tue, 10 Nov 2026 16:13:13 +0100 Subject: [PATCH 3/3] Naming: use Diagram and Decorative Text in the feature catalog (#3622) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Naming: use Diagram and Decorative Text in the feature catalog * Naming: align the Chinese decorative text term * test(naming): guard neutral display labels * Naming: Diagram no longer repeats the chart word in de, uk and ro German Schema and Ukrainian Схема match craft-i18n#211; Romanian Schemă avoids GridCraft's own chart word Diagramă. A catalog test checks every shipped language translates both labels and that Diagram differs from Chart/Charts. The display-name guard now handles escaped char literals and any #[cfg(test)] item, asserts it scanned catalog.rs and a minimum of files and literals, and tests word boundaries. The catalog test asserts each ID appears once. Docs carry timestamps and revision history. --------- Co-authored-by: FredApps --- crates/engine/src/catalog.rs | 16 +- crates/ui-egui/src/i18n.rs | 27 +++- crates/ui-egui/src/i18n/cs.rs | 2 + crates/ui-egui/src/i18n/de.rs | 2 + crates/ui-egui/src/i18n/el.rs | 2 + crates/ui-egui/src/i18n/es.rs | 2 + crates/ui-egui/src/i18n/fr.rs | 2 + crates/ui-egui/src/i18n/it.rs | 2 + crates/ui-egui/src/i18n/ja.rs | 2 + crates/ui-egui/src/i18n/ko.rs | 2 + crates/ui-egui/src/i18n/nl.rs | 2 + crates/ui-egui/src/i18n/pl.rs | 2 + crates/ui-egui/src/i18n/pt.rs | 2 + crates/ui-egui/src/i18n/ro.rs | 2 + crates/ui-egui/src/i18n/ru.rs | 2 + crates/ui-egui/src/i18n/sr.rs | 2 + crates/ui-egui/src/i18n/tr.rs | 2 + crates/ui-egui/src/i18n/uk.rs | 2 + crates/ui-egui/src/i18n/vi.rs | 2 + crates/ui-egui/src/i18n/zh.rs | 2 + crates/ui-egui/tests/display_names.rs | 215 ++++++++++++++++++++++++++ docs/feature-names.md | 28 ++++ docs/localization-parity.md | 3 +- docs/parity-checklist.md | 4 +- 24 files changed, 321 insertions(+), 8 deletions(-) create mode 100644 crates/ui-egui/tests/display_names.rs create mode 100644 docs/feature-names.md diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index 234e92e58..41d1157f7 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -118,7 +118,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("insert.picture", "Insert", "Illustrations", "Pictures"), ("insert.shape", "Insert", "Illustrations", "Shapes"), ("insert.icons", "Insert", "Illustrations", "Icons"), - ("insert.smartArt", "Insert", "Illustrations", "SmartArt"), + ("insert.smartArt", "Insert", "Illustrations", "Diagram"), ("insert.screenshot", "Insert", "Illustrations", "Screenshot"), ("insert.recommendedCharts", "Insert", "Charts", "Recommended Charts"), ("insert.chart", "Insert", "Charts", "Charts"), @@ -131,7 +131,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("review.newComment", "Insert", "Comments", "Comment"), ("insert.textBox", "Insert", "Text", "Text Box"), ("pageLayout.headerFooter", "Insert", "Text", "Header & Footer"), - ("insert.wordArt", "Insert", "Text", "WordArt"), + ("insert.wordArt", "Insert", "Text", "Decorative Text"), ("insert.signatureLine", "Insert", "Text", "Signature Line"), ("insert.object", "Insert", "Text", "Object"), ("insert.equation", "Insert", "Symbols", "Equation"), @@ -340,3 +340,15 @@ pub fn parity() -> (usize, usize) { let done = ids.iter().filter(|id| crate::find_command(id).is_some()).count(); (done, ids.len()) } + +#[cfg(test)] +mod feature_name_tests { + #[test] + fn own_feature_labels_keep_command_ids() { + for (id, expected) in [("insert.smartArt", "Diagram"), ("insert.wordArt", "Decorative Text")] { + let entries: Vec<_> = super::CATALOG.iter().filter(|entry| entry.0 == id).collect(); + assert_eq!(entries.len(), 1, "{id} appears once"); + assert_eq!(entries[0].3, expected); + } + } +} diff --git a/crates/ui-egui/src/i18n.rs b/crates/ui-egui/src/i18n.rs index 45c336f87..3aac97d6e 100644 --- a/crates/ui-egui/src/i18n.rs +++ b/crates/ui-egui/src/i18n.rs @@ -194,7 +194,12 @@ impl Language { /// A label that needs two translations (Home's text "Orientation" vs Page Layout's print /// "Orientation") is keyed `context|label`: English shows only the part after the `|`. pub fn tr(self, text: &str) -> &str { - let translated = match self { + self.translated(text).unwrap_or_else(|| text.rsplit_once('|').map_or(text, |(_, label)| label)) + } + + /// The catalog's own translation of `text`, or `None` when it falls back to English. + fn translated(self, text: &str) -> Option<&'static str> { + match self { Self::En => None, Self::Es => es::translate(text), Self::El => el::translate(text), @@ -214,8 +219,7 @@ impl Language { Self::Vi => vi::translate(text), Self::Nl => nl::translate(text), Self::Ro => ro::translate(text), - }; - translated.unwrap_or_else(|| text.rsplit_once('|').map_or(text, |(_, label)| label)) + } } pub(crate) fn tr_error(self, message: &str) -> String { @@ -321,6 +325,23 @@ fn name_sort_key(name: &str) -> String { mod tests { use super::*; + /// Every shipped catalog names GridCraft's own Diagram and Decorative Text features, and + /// Diagram never repeats that catalog's chart word (both sit on the Insert tab). + #[test] + fn every_catalog_has_distinct_diagram_and_decorative_text() { + for language in Language::ALL.into_iter().filter(|l| *l != Language::En) { + let code = language.code(); + let diagram = language.translated("Diagram"); + assert!(diagram.is_some(), "{code}: Diagram"); + assert!(language.translated("Decorative Text").is_some(), "{code}: Decorative Text"); + for chart in ["Chart", "Charts"] { + if let Some(chart_text) = language.translated(chart) { + assert_ne!(diagram, Some(chart_text), "{code}: Diagram equals {chart}"); + } + } + } + } + /// Help tables are sorted and unique: lookups binary-search them. #[test] fn function_help_tables_are_sorted_and_unique() { diff --git a/crates/ui-egui/src/i18n/cs.rs b/crates/ui-egui/src/i18n/cs.rs index 39e6337dd..eaee085cf 100644 --- a/crates/ui-egui/src/i18n/cs.rs +++ b/crates/ui-egui/src/i18n/cs.rs @@ -3,6 +3,8 @@ /// Returns the Czech translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagram", + "Decorative Text" => "Ozdobný text", "Paste" => "Vložit", "Paste Values" => "Vložit hodnoty", "Paste Formulas" => "Vložit vzorce", diff --git a/crates/ui-egui/src/i18n/de.rs b/crates/ui-egui/src/i18n/de.rs index f136e483b..07fdb6710 100644 --- a/crates/ui-egui/src/i18n/de.rs +++ b/crates/ui-egui/src/i18n/de.rs @@ -4,6 +4,8 @@ /// Returns the German translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Schema", + "Decorative Text" => "Dekorativer Text", "Paste" => "Einfügen", "Paste Values" => "Werte einfügen", "Paste Formulas" => "Formeln einfügen", diff --git a/crates/ui-egui/src/i18n/el.rs b/crates/ui-egui/src/i18n/el.rs index dc57982c7..ec42b3473 100644 --- a/crates/ui-egui/src/i18n/el.rs +++ b/crates/ui-egui/src/i18n/el.rs @@ -5,6 +5,8 @@ /// Returns the Greek translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Διάγραμμα", + "Decorative Text" => "Διακοσμητικό κείμενο", "Paste" => "Επικόλληση", "Paste Values" => "Επικόλληση τιμών", "Paste Formulas" => "Επικόλληση τύπων", diff --git a/crates/ui-egui/src/i18n/es.rs b/crates/ui-egui/src/i18n/es.rs index 3c15a16b1..db78d8105 100644 --- a/crates/ui-egui/src/i18n/es.rs +++ b/crates/ui-egui/src/i18n/es.rs @@ -5,6 +5,8 @@ /// Returns the Spanish translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagrama", + "Decorative Text" => "Texto decorativo", "Paste" => "Pegar", "Paste Values" => "Pegar valores", "Paste Formulas" => "Pegar fórmulas", diff --git a/crates/ui-egui/src/i18n/fr.rs b/crates/ui-egui/src/i18n/fr.rs index 3e95d4b72..fd5e23b5b 100644 --- a/crates/ui-egui/src/i18n/fr.rs +++ b/crates/ui-egui/src/i18n/fr.rs @@ -3,6 +3,8 @@ /// Returns the French translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagramme", + "Decorative Text" => "Texte décoratif", "Different First Page" => "Première page différente", "Different Odd & Even Pages" => "Pages paires et impaires différentes", "Scale with Document" => "Mettre à l'échelle avec le document", diff --git a/crates/ui-egui/src/i18n/it.rs b/crates/ui-egui/src/i18n/it.rs index 3ad0ff328..dcede52cc 100644 --- a/crates/ui-egui/src/i18n/it.rs +++ b/crates/ui-egui/src/i18n/it.rs @@ -5,6 +5,8 @@ /// Returns the Italian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagramma", + "Decorative Text" => "Testo decorativo", "Paste" => "Incolla", "Paste Values" => "Incolla valori", "Paste Formulas" => "Incolla formule", diff --git a/crates/ui-egui/src/i18n/ja.rs b/crates/ui-egui/src/i18n/ja.rs index 19fc85f02..d558be6ec 100644 --- a/crates/ui-egui/src/i18n/ja.rs +++ b/crates/ui-egui/src/i18n/ja.rs @@ -8,6 +8,8 @@ /// English label → Japanese, the shared reference set. pub(super) const CORE: &[(&str, &str)] = &[ + ("Diagram", "図解"), + ("Decorative Text", "装飾文字"), ("Paste", "貼り付け"), ("Paste Values", "値の貼り付け"), ("Paste Formulas", "数式の貼り付け"), diff --git a/crates/ui-egui/src/i18n/ko.rs b/crates/ui-egui/src/i18n/ko.rs index 8c28be26e..ca2710272 100644 --- a/crates/ui-egui/src/i18n/ko.rs +++ b/crates/ui-egui/src/i18n/ko.rs @@ -3,6 +3,8 @@ /// Returns the Korean translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "다이어그램", + "Decorative Text" => "장식 텍스트", "Paste" => "붙여넣기", "Paste Values" => "값 붙여넣기", "Paste Formulas" => "수식 붙여넣기", diff --git a/crates/ui-egui/src/i18n/nl.rs b/crates/ui-egui/src/i18n/nl.rs index b5cddb88d..64c24b1f2 100644 --- a/crates/ui-egui/src/i18n/nl.rs +++ b/crates/ui-egui/src/i18n/nl.rs @@ -3,6 +3,8 @@ /// Returns the Dutch translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagram", + "Decorative Text" => "Decoratieve tekst", "Paste" => "Plakken", "Paste Values" => "Waarden plakken", "Paste Formulas" => "Formules plakken", diff --git a/crates/ui-egui/src/i18n/pl.rs b/crates/ui-egui/src/i18n/pl.rs index f2bfd72c7..29264ce91 100644 --- a/crates/ui-egui/src/i18n/pl.rs +++ b/crates/ui-egui/src/i18n/pl.rs @@ -1,6 +1,8 @@ /// Returns the Polish translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagram", + "Decorative Text" => "Tekst ozdobny", "Paste" => "Wklej", "Paste Values" => "Wklej wartości", "Paste Formulas" => "Wklej formuły", diff --git a/crates/ui-egui/src/i18n/pt.rs b/crates/ui-egui/src/i18n/pt.rs index d525f027e..3cc4faf76 100644 --- a/crates/ui-egui/src/i18n/pt.rs +++ b/crates/ui-egui/src/i18n/pt.rs @@ -4,6 +4,8 @@ /// Returns the Portuguese translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diagrama", + "Decorative Text" => "Texto decorativo", "Paste" => "Colar", "Paste Values" => "Colar Valores", "Paste Formulas" => "Colar Fórmulas", diff --git a/crates/ui-egui/src/i18n/ro.rs b/crates/ui-egui/src/i18n/ro.rs index 66eb854b7..3d5a193d1 100644 --- a/crates/ui-egui/src/i18n/ro.rs +++ b/crates/ui-egui/src/i18n/ro.rs @@ -8,6 +8,8 @@ /// Returns the Romanian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Schemă", + "Decorative Text" => "Text decorativ", "Paste" => "Lipire", "Paste Values" => "Lipire valori", "Paste Formulas" => "Lipire formule", diff --git a/crates/ui-egui/src/i18n/ru.rs b/crates/ui-egui/src/i18n/ru.rs index 00a3062bd..2d4d3fe1f 100644 --- a/crates/ui-egui/src/i18n/ru.rs +++ b/crates/ui-egui/src/i18n/ru.rs @@ -3,6 +3,8 @@ /// Returns the Russian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Схема", + "Decorative Text" => "Декоративный текст", "Paste" => "Вставить", "Paste Values" => "Вставить значения", "Paste Formulas" => "Вставить формулы", diff --git a/crates/ui-egui/src/i18n/sr.rs b/crates/ui-egui/src/i18n/sr.rs index 9254dc938..a0eac0a4f 100644 --- a/crates/ui-egui/src/i18n/sr.rs +++ b/crates/ui-egui/src/i18n/sr.rs @@ -3,6 +3,8 @@ /// Returns the Serbian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Dijagram", + "Decorative Text" => "Ukrasni tekst", "Paste" => "Nalepi", "Paste Values" => "Nalepi vrednosti", "Paste Formulas" => "Nalepi formule", diff --git a/crates/ui-egui/src/i18n/tr.rs b/crates/ui-egui/src/i18n/tr.rs index c8da06cb1..ed3d0bf01 100644 --- a/crates/ui-egui/src/i18n/tr.rs +++ b/crates/ui-egui/src/i18n/tr.rs @@ -4,6 +4,8 @@ /// Returns the Turkish translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Diyagram", + "Decorative Text" => "Dekoratif Metin", "Paste" => "Yapıştır", "Paste Values" => "Değerleri Yapıştır", "Paste Formulas" => "Formülleri Yapıştır", diff --git a/crates/ui-egui/src/i18n/uk.rs b/crates/ui-egui/src/i18n/uk.rs index 35fc9f0f7..ddb9368a0 100644 --- a/crates/ui-egui/src/i18n/uk.rs +++ b/crates/ui-egui/src/i18n/uk.rs @@ -3,6 +3,8 @@ /// Returns the Ukrainian translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Схема", + "Decorative Text" => "Декоративний текст", "Paste" => "Вставити", "Paste Values" => "Вставити значення", "Paste Formulas" => "Вставити формули", diff --git a/crates/ui-egui/src/i18n/vi.rs b/crates/ui-egui/src/i18n/vi.rs index 8cb5a32ac..260f9572d 100644 --- a/crates/ui-egui/src/i18n/vi.rs +++ b/crates/ui-egui/src/i18n/vi.rs @@ -4,6 +4,8 @@ /// Returns the Vietnamese translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "Sơ đồ", + "Decorative Text" => "Chữ trang trí", "Paste" => "Dán", "Paste Values" => "Dán giá trị", "Paste Formulas" => "Dán công thức", diff --git a/crates/ui-egui/src/i18n/zh.rs b/crates/ui-egui/src/i18n/zh.rs index e71a0df3f..ff671a7c9 100644 --- a/crates/ui-egui/src/i18n/zh.rs +++ b/crates/ui-egui/src/i18n/zh.rs @@ -3,6 +3,8 @@ /// Returns the Chinese translation for `text`, or `None` so callers fall back to English. pub(super) fn translate(text: &str) -> Option<&'static str> { Some(match text { + "Diagram" => "图示", + "Decorative Text" => "装饰文字", "Paste" => "粘贴", "Paste Values" => "粘贴值", "Paste Formulas" => "粘贴公式", diff --git a/crates/ui-egui/tests/display_names.rs b/crates/ui-egui/tests/display_names.rs new file mode 100644 index 000000000..250a4fd79 --- /dev/null +++ b/crates/ui-egui/tests/display_names.rs @@ -0,0 +1,215 @@ +//! Guard the production display surfaces without banning compatibility identifiers or notices. +use std::path::{Path, PathBuf}; + +const BANNED: &[&str] = &["SmartArt", "WordArt"]; +const ALLOWED: &[&str] = &[]; +const ROOTS: &[&str] = &["src", "../engine/src/catalog.rs"]; + +fn is_word(c: char) -> bool { + c.is_alphanumeric() || c == '_' +} + +fn banned_display(text: &str) -> bool { + if ALLOWED.contains(&text) { + return false; + } + BANNED.iter().any(|banned| { + text.match_indices(banned) + .any(|(at, _)| !text[..at].chars().next_back().is_some_and(is_word) && !text[at + banned.len()..].chars().next().is_some_and(is_word)) + }) +} + +// Read ordinary and raw Rust string literals, ignoring comments. No compiler dependency is +// needed, so this guard can also run by itself when a full application build is unavailable. +#[allow(clippy::cognitive_complexity)] // This small test-only lexer keeps raw-string/comment state together. +fn literals(source: &str) -> Vec { + let chars: Vec = source.chars().collect(); + let mut out = Vec::new(); + let mut at = 0; + let mut test_module = false; + let mut test_depth = 0_usize; + while at < chars.len() { + if chars.get(at..at + 12) == Some(&['#', '[', 'c', 'f', 'g', '(', 't', 'e', 's', 't', ')', ']']) { + at += 12; + // Any test-only item (module, fn, use, const...) ends at its closing brace or at a + // `;` outside braces, so its literals are skipped and later production is scanned. + test_module = true; + } else if chars.get(at..at + 2) == Some(&['/', '/']) { + while at < chars.len() && chars[at] != '\n' { + at += 1; + } + } else if chars.get(at..at + 2) == Some(&['/', '*']) { + at += 2; + let mut depth = 1; + while at < chars.len() && depth > 0 { + if chars.get(at..at + 2) == Some(&['/', '*']) { + depth += 1; + at += 2; + } else if chars.get(at..at + 2) == Some(&['*', '/']) { + depth -= 1; + at += 2; + } else { + at += 1; + } + } + } else if chars[at] == 'r' && matches!(chars.get(at + 1), Some('"' | '#')) { + let start = at; + at += 1; + while chars.get(at) == Some(&'#') { + at += 1; + } + let hashes = at - start - 1; + if chars.get(at) != Some(&'"') { + continue; + } + at += 1; + let begin = at; + while at < chars.len() { + if chars[at] == '"' && (1..=hashes).all(|n| chars.get(at + n) == Some(&'#')) { + break; + } + at += 1; + } + if !test_module { + out.push(chars[begin..at].iter().collect()); + } + at += 1 + hashes; + } else if chars[at] == '"' { + at += 1; + let mut text = String::new(); + while at < chars.len() { + let c = chars[at]; + at += 1; + if c == '"' { + break; + } + if c == '\\' && at < chars.len() { + text.push(chars[at]); + at += 1; + } else { + text.push(c); + } + } + if !test_module { + out.push(text); + } + } else if chars[at] == '\'' && chars.get(at + 1) == Some(&'\\') { + // An escaped character literal such as '\'', '\"' or '\u{22}': skip to its closing quote. + at += 3; + while at < chars.len() && chars[at] != '\'' && chars[at] != '\n' { + at += 1; + } + at += 1; + } else if chars[at] == '\'' && chars.get(at + 2) == Some(&'\'') { + at += 3; // A character literal such as '"', not the start of a string. + } else { + if test_module { + match chars[at] { + '{' => test_depth += 1, + '}' => { + test_depth = test_depth.saturating_sub(1); + if test_depth == 0 { + test_module = false; + } + } + ';' if test_depth == 0 => test_module = false, + _ => {} + } + } + at += 1; + } + } + out +} + +#[derive(Default)] +struct Scan { + files: usize, + literals: usize, + catalog: bool, + failures: Vec, +} + +fn check_tree(path: &Path, scan: &mut Scan) { + if path.is_dir() { + for entry in std::fs::read_dir(path).expect("source directory") { + let path = entry.expect("source entry").path(); + if path.file_name().is_some_and(|name| name == "tests") { + continue; + } + check_tree(&path, scan); + } + } else if path.extension().is_some_and(|ext| ext == "rs") { + let name = path.file_name().expect("source filename").to_string_lossy(); + if name == "tests.rs" || name.ends_with("_tests.rs") { + return; + } + let source = std::fs::read_to_string(path).expect("UTF-8 Rust source"); + scan.files += 1; + scan.catalog |= name == "catalog.rs"; + // Skip only inline test modules, not later production following test-only imports. + for text in literals(&source) { + scan.literals += 1; + if banned_display(&text) { + scan.failures.push(format!("{}: {text:?}", path.display())); + } + } + } +} + +#[test] +fn production_display_names_stay_neutral() { + let base = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let mut scan = Scan::default(); + for root in ROOTS { + let root = base.join(root); + assert!(root.exists(), "missing display source: {}", root.display()); + check_tree(&root, &mut scan); + } + // A scan that silently finds nothing would pass vacuously. + assert!(scan.catalog, "engine catalog.rs was not scanned"); + assert!(scan.files >= 50, "only {} source files scanned", scan.files); + assert!(scan.literals >= 5_000, "only {} literals scanned", scan.literals); + assert!(scan.failures.is_empty(), "Banned display names:\n{}", scan.failures.join("\n")); +} + +#[test] +fn guard_distinguishes_labels_from_identifiers_and_comments() { + for name in BANNED { + assert!(banned_display(name), "{name}"); + } + for name in ALLOWED { + assert!(!banned_display(name), "{name}"); + } + assert!(!banned_display("filter.cameraRaw")); + assert!(!banned_display("tool.guidedUpright")); + assert!(!banned_display("insert.smartArt")); + for identifier in ["mySmartArt", "SmartArt2", "_SmartArt_", "WordArtStyle"] { + assert!(!banned_display(identifier), "{identifier}"); + } + for label in ["Use SmartArt here", "WordArt…", "(SmartArt)"] { + assert!(banned_display(label), "{label}"); + } + assert_eq!(literals("// \"ignored\"\n/* \"ignored\" */ \"label\""), ["label"]); + assert_eq!(literals("r#\"raw label\"#"), ["raw label"]); + assert_eq!(literals("r##\"a \"# b\"## \"next\""), ["a \"# b", "next"]); + assert_eq!(literals("\"say \\\"hi\\\"\" \"next\""), ["say \"hi\"", "next"]); + assert_eq!(literals("/* outer /* \"inner\" */ \"still\" */ \"label\""), ["label"]); + assert_eq!(literals("let q = '\\''; let d = '\\\"'; let u = '\\u{22}'; \"label\""), ["label"]); + assert_eq!(literals("let d = '\"'; \"label\""), ["label"]); +} + +#[test] +fn guard_scans_production_after_test_only_items() { + let source = r###"#[cfg(test)] mod tests { let _ = "ignored"; } + #[cfg(test)] use fixtures::Helper; + fn production() { let _ = r#"live label"#; } + "###; + assert_eq!(literals(source), ["live label"]); + assert_eq!(literals("#[cfg(test)] mod tests; fn live() { let _ = \"live label\"; }"), ["live label"]); + let items = r#"#[cfg(test)] pub mod a { "x"; } + #[cfg(test)] fn helper() { "y"; } + #[cfg(test)] const C: &str = "z"; + fn live() { "live"; }"#; + assert_eq!(literals(items), ["live"]); +} diff --git a/docs/feature-names.md b/docs/feature-names.md new file mode 100644 index 000000000..800cd3f82 --- /dev/null +++ b/docs/feature-names.md @@ -0,0 +1,28 @@ +# Feature naming + +> **Last reviewed:** 2026-11-10 · **Last updated:** 2026-11-10 · **Change:** minor (Diagram words aligned with craft-i18n#211) + +ArtCraft apps use descriptive names for their own features. Product names used to describe compatibility, file formats, attribution or external software remain accurate. + +## Naming map + +SmartArt → Diagram; WordArt → Decorative Text. These are catalog labels for pending features, not a claim that the features are implemented. + +## Compatibility boundary + +The insert.smartArt and insert.wordArt command IDs and pending keytips remain unchanged. OOXML data, imported names and interoperability references are unchanged. No assets, attribution, dependencies or rendering behavior change. + +## Localization and verification + +Each shipped non-English interface catalog includes the new labels. Diagram uses a word distinct from Chart in every catalog: German (`Schema`), Ukrainian (`Схема`) and Romanian (`Schemă`) avoid their chart word, matching the shared wording in [craft-i18n#211](https://github.com/storytold/craft-i18n/pull/211) for German and Ukrainian. Regression tests cover the labels, every shipped catalog and stable command identities. The translations are AI-assisted and not yet reviewed by native speakers. + +## Regression coverage + +The standard-library-only `display_names` integration tests scan production Rust string literals for the retired display names and preserve explicit compatibility exceptions. They also test identifier, comment and raw-string handling. These focused checks do not replace application or visual testing. + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-11-10 | minor | Diagram no longer repeats the chart word: German `Schema`, Ukrainian `Схема` (as craft-i18n#211), Romanian `Schemă`; a test covers every shipped catalog | +| 2026-11-08 | minor | Created: SmartArt → Diagram and WordArt → Decorative Text as GridCraft's own catalog labels | diff --git a/docs/localization-parity.md b/docs/localization-parity.md index 9f5188948..9dec2d0e7 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-31 · **Change:** minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-10 · **Change:** minor (Diagram and Decorative Text labels, Diagram distinct from Chart); previously minor (CJK fallback baseline alignment); previously minor (Arabic bidi display, editing and cell reading direction; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (Arabic font fallback; missing translations are listed by `cargo xtask i18n-report` and never fail a check); previously minor (number formats honour the `B1`/`B2` calendar prefixes, checked against Excel; earlier: missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks); previously minor (AutoFit estimates Arabic text width from measured glyph widths); previously minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check; Arabic text sorts by its letters, not its vowel marks) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Per-language status against Excel, which ships its UI in dozens of languages (Windows language packs cover 100+), with localized function names, argument separators, number and date formats, @@ -133,6 +133,7 @@ language (none so far) and of localized function names. | Date | Change | Summary | |---|---|---| +| 2026-11-10 | minor | Diagram and Decorative Text labels in all 18 non-English catalogs; Diagram never repeats the catalog's chart word (German `Schema`, Ukrainian `Схема`, Romanian `Schemă`), see [feature-names.md](feature-names.md) | | 2026-10-31 | minor | System and embedded CJK text fallbacks share the Latin baseline in cells, editors and window titles; other scripts, symbols and emoji keep their alignment | | 2026-10-26 | minor | Shared bidi layout supports Arabic clusters, marks, visual cursor navigation and directional accessibility; cell Context/LTR/RTL is stored separately from sheet RTL and round-trips through XLSX | | 2026-10-26 | minor | Arabic text renders in cells and the interface: a system Arabic face (Tahoma, Arial or Segoe UI on Windows, Geeza Pro on macOS, Noto Sans or Naskh Arabic on Linux) joins the script fallbacks, fontconfig's Arabic choice, when it has Arabic letters, leads the Arabic fallbacks and draws Arabic wherever the UI or cell font has none, such as the default cell fonts (part of #229; a UI font with Arabic of its own, such as DejaVu Sans in the Flatpak, still draws interface Arabic, left for a follow-up), and craft-fonts' Noto Sans Arabic is a required face allowed on the web. Workbook fonts without Arabic keep every script fallback. Shaping and bidirectional layout remain separate work | diff --git a/docs/parity-checklist.md b/docs/parity-checklist.md index 4100ec630..ad7e680f9 100644 --- a/docs/parity-checklist.md +++ b/docs/parity-checklist.md @@ -165,11 +165,11 @@ Implemented: Missing: -- `insert.smartArt` SmartArt (Illustrations) +- `insert.smartArt` Diagram (Illustrations) - `insert.screenshot` Screenshot (Illustrations) - `insert.map` Maps (Charts) - `insert.timeline` Timeline (Filters) -- `insert.wordArt` WordArt (Text) +- `insert.wordArt` Decorative Text (Text) - `insert.signatureLine` Signature Line (Text) - `insert.object` Object (Text)