craft-xls: read data validation (port of gridcraft#3636) (#999)

* Add craft-sheet: the cell positions, values and tables the spreadsheet formats share

craft-xls carried its own CellRef, Range, Value, CellError, Visibility, sheet-name
quoting, built-in number format table and indexed palette. craft-xlsx and craft-ods
need the same ones, so they move to a crate with no dependencies and craft-xls
re-exports them: an application gets one Value and one CellRef whichever file it read.

Added on the way: A1 parsing (CellRef::parse, Range::parse), Range::count and a1,
CellError::parse and the newer error values SpreadsheetML stores as text,
builtin_format_id for writers.

* craft-xls: read data validation (DVal and Dv records)

Each Dv after a DVal becomes a DataValidation in SpreadsheetML's terms: type,
operator, formulas as A1 text from the top left cell of all the rule's ranges (a
literal list quoted, items separated by commas), the four messages, alert style
and flags. Exactly the DVal's count of rules is read, as Excel does; a rule Excel
would refuse is skipped with a warning, and the rest are kept. A sheet keeps at
most 65,536 rules (the count was not capped on its own).

Ported from storytold/gridcraft#3636.

---------

Co-authored-by: kiwi4boy <kiwi4boy@gmail.com>

crates/xls/README.md

@@ -29,8 +29,12 @@ No third-party implementation was consulted.
 - Merged cells, ColInfo and Row records (width in 1/256 character, height in twips, hidden,
   outline, default format), the default row height, the Window2 display flags.
 - Defined names with their scope, print areas and print titles.
+- Data validation (DVal and Dv records): type, operator, formulas as text (relative references
+  from the top left cell of the rule's ranges, a literal list as `"a,b"`), messages, alert style
+  and flags, in SpreadsheetML's terms. Exactly the DVal's count of rules is read, as Excel does;
+  a rule Excel would refuse is skipped with a warning, and at most 65,536 are kept per sheet.
 
-Not read: charts, pictures, comments, conditional formats, data validation, hyperlinks,
+Not read: charts, pictures, comments, conditional formats, hyperlinks,
 PivotTables, macros. Each is named once in `Read::warnings`. Encrypted workbooks
 (`Error::Encrypted`) and BIFF5 and older (`Error::Biff5`) are refused. There is no writer
 (`craft_xls::testing` builds small fixtures for tests).

crates/xls/src/dv.rs

@@ -0,0 +1,212 @@
+//! Data validation: a DVal record ([MS-XLS] 2.4.96) starts a sheet's rules and says how many
+//! follow, and each Dv record after it (2.4.95) is one rule: its type, operator, alert style,
+//! messages, up to two formulas (DVParsedFormula, 2.5.198.9) and the ranges it covers (SqRefU,
+//! 2.5.247).
+//!
+//! Excel reads exactly the DVal's count of Dv records, and none without a DVal. It opens an
+//! unknown rule type as "any value", but refuses the whole file for an unknown alert style or
+//! operator, or for a formula or range list that runs past its record; here such a rule is
+//! skipped instead and the rest are kept.
+
+use craft_sheet::{CellRef, MAX_COLS, MAX_ROWS, Range};
+
+use crate::formula::{Links, Mode, decode};
+use crate::model::{DataValidation, ErrorStyle, ValidationOperator, ValidationType};
+use crate::records::Cursor;
+use crate::{Error, expr};
+
+/// Rules kept per sheet; more are skipped with a warning. Each rule's messages are charged to
+/// the text allowance, but its ranges are not, so the count is capped on its own.
+pub(crate) const MAX_RULES: usize = 65_536;
+
+/// The last row and column of the BIFF8 grid (65,536 × 256).
+const LAST_ROW: u16 = 0xFFFF;
+const LAST_COL: u16 = 0x00FF;
+
+/// PtgStr: a string constant ([MS-XLS] 2.5.198.89).
+const PTG_STR: u8 = 0x17;
+
+fn invalid(msg: &str) -> Error {
+    Error::Invalid(msg.into())
+}
+
+/// The number of Dv records a DVal says follow it: flags (2 bytes), the input window's
+/// position (4 + 4), the drop-down arrow's object id (4), then `idvMac` (4).
+pub(crate) fn dval(c: &mut Cursor<'_, '_>) -> Result<u32, Error> {
+    c.skip(14)?;
+    c.u32()
+}
+
+/// One Dv record as stored, before its formulas are decoded.
+pub(crate) struct Dv {
+    flags: u32,
+    prompt_title: String,
+    error_title: String,
+    prompt: String,
+    error: String,
+    rgce1: Vec<u8>,
+    rgce2: Vec<u8>,
+    ranges: Vec<Range>,
+}
+
+impl Dv {
+    /// Bytes held in the rule's four messages.
+    pub(crate) fn text_len(&self) -> usize {
+        [&self.prompt_title, &self.error_title, &self.prompt, &self.error].iter().map(|s| s.len()).sum()
+    }
+}
+
+/// Reads a Dv record: flags, the four messages, the two formulas and the ranges.
+pub(crate) fn dv(c: &mut Cursor<'_, '_>) -> Result<Dv, Error> {
+    let flags = c.u32()?;
+    let prompt_title = message(c)?;
+    let error_title = message(c)?;
+    let prompt = message(c)?;
+    let error = message(c)?;
+    let rgce1 = formula(c)?;
+    let rgce2 = formula(c)?;
+    let cref = usize::from(c.u16()?);
+    // Each Ref8U is 8 bytes; a count the record can't hold makes Excel refuse the file.
+    if cref.saturating_mul(8) > c.remaining() {
+        return Err(invalid("data validation ranges run past the end of the record"));
+    }
+    let mut ranges = Vec::with_capacity(cref);
+    for _ in 0..cref {
+        let (r0, r1, c0, c1) = (c.u16()?, c.u16()?, c.u16()?, c.u16()?);
+        ranges.extend(range(r0, r1, c0, c1));
+    }
+    Ok(Dv { flags, prompt_title, error_title, prompt, error, rgce1, rgce2, ranges })
+}
+
+/// A title or message. Excel stores an empty one as a single NUL character.
+fn message(c: &mut Cursor<'_, '_>) -> Result<String, Error> {
+    let s = c.string()?;
+    Ok(if s == "\0" { String::new() } else { s })
+}
+
+/// A DVParsedFormula's tokens: their size, 2 unused bytes (which hold leftover data in Excel's
+/// files, so they are never read), then the tokens.
+fn formula(c: &mut Cursor<'_, '_>) -> Result<Vec<u8>, Error> {
+    let cce = usize::from(c.u16()?);
+    c.skip(2)?;
+    if cce > c.remaining() {
+        return Err(invalid("data validation formula runs past the end of the record"));
+    }
+    (0..cce).map(|_| c.u8()).collect()
+}
+
+/// A Ref8U as a range. Columns past the BIFF8 grid are clamped to its last column, as Excel
+/// does; a reversed range is dropped (Excel drops it too). A range over every row (column) of
+/// the BIFF8 grid is a whole column (row), as SpreadsheetML stores the same rule.
+fn range(r0: u16, r1: u16, c0: u16, c1: u16) -> Option<Range> {
+    let (c0, c1) = (c0.min(LAST_COL), c1.min(LAST_COL));
+    if r0 > r1 || c0 > c1 {
+        return None;
+    }
+    let (r0, r1) = if r0 == 0 && r1 == LAST_ROW { (0, MAX_ROWS - 1) } else { (u32::from(r0), u32::from(r1)) };
+    let (c0, c1) = if c0 == 0 && c1 == LAST_COL { (0, MAX_COLS - 1) } else { (u32::from(c0), u32::from(c1)) };
+    Some(Range::new(CellRef::new(r0, c0), CellRef::new(r1, c1)))
+}
+
+/// The rule of a Dv, or `None` when it has no ranges, when Excel would refuse it (an alert
+/// style or operator outside the spec's values), or when a formula can't be decoded.
+pub(crate) fn validation(dv: Dv, links: &Links) -> Option<DataValidation> {
+    // Relative references are offsets from the top left cell of all the ranges, whatever order
+    // they are listed in.
+    let base = CellRef::new(dv.ranges.iter().map(|r| r.first.row).min()?, dv.ranges.iter().map(|r| r.first.col).min()?);
+    let f = dv.flags;
+    let bit = |n: u32| f & (1 << n) != 0;
+    let kind = match f & 0xF {
+        1 => ValidationType::Whole,
+        2 => ValidationType::Decimal,
+        3 => ValidationType::List,
+        4 => ValidationType::Date,
+        5 => ValidationType::Time,
+        6 => ValidationType::TextLength,
+        7 => ValidationType::Custom,
+        // 0, and the undefined 8 to 15, which Excel opens as "any value".
+        _ => ValidationType::Any,
+    };
+    let error_style = match (f >> 4) & 0x7 {
+        0 => ErrorStyle::Stop,
+        1 => ErrorStyle::Warning,
+        2 => ErrorStyle::Information,
+        _ => return None,
+    };
+    // The operator is undefined for "any value", lists and custom formulas, whatever it holds.
+    let compares = !matches!(kind, ValidationType::Any | ValidationType::List | ValidationType::Custom);
+    let operator = if compares {
+        match (f >> 20) & 0xF {
+            0 => ValidationOperator::Between,
+            1 => ValidationOperator::NotBetween,
+            2 => ValidationOperator::Equal,
+            3 => ValidationOperator::NotEqual,
+            4 => ValidationOperator::Greater,
+            5 => ValidationOperator::Less,
+            6 => ValidationOperator::GreaterOrEqual,
+            7 => ValidationOperator::LessOrEqual,
+            _ => return None,
+        }
+    } else {
+        ValidationOperator::Between
+    };
+    let text = |rgce: &[u8]| decode(rgce, &[], base, Mode::Cell, links).ok().map(|e| expr::print(&e));
+    let formula1 = match kind {
+        ValidationType::Any => String::new(),
+        _ if dv.rgce1.is_empty() => String::new(),
+        ValidationType::List if bit(7) => match literal_list(&dv.rgce1) {
+            Some(list) => list,
+            None => text(&dv.rgce1)?,
+        },
+        _ => text(&dv.rgce1)?,
+    };
+    let formula2 = if compares && matches!(operator, ValidationOperator::Between | ValidationOperator::NotBetween) && !dv.rgce2.is_empty() {
+        Some(text(&dv.rgce2)?)
+    } else {
+        None
+    };
+    Some(DataValidation {
+        ranges: dv.ranges,
+        kind,
+        operator,
+        formula1,
+        formula2,
+        allow_blank: bit(8),
+        // fSuppressCombo hides a list's in-cell arrow; it means nothing for other kinds.
+        in_cell_dropdown: !(kind == ValidationType::List && bit(9)),
+        prompt_title: dv.prompt_title,
+        prompt: dv.prompt,
+        show_prompt: bit(18),
+        error_title: dv.error_title,
+        error: dv.error,
+        error_style,
+        show_error: bit(19),
+    })
+}
+
+/// A literal list: one PtgStr holding the items separated by NUL characters, written as
+/// SpreadsheetML writes it: in quotes, the items separated by commas, each `"` doubled. `None`
+/// when the formula isn't a lone PtgStr.
+fn literal_list(rgce: &[u8]) -> Option<String> {
+    let (&ptg, rest) = rgce.split_first()?;
+    let (&cch, rest) = rest.split_first()?;
+    let (&flags, chars) = rest.split_first()?;
+    if ptg != PTG_STR {
+        return None;
+    }
+    let cch = usize::from(cch);
+    let s = if flags & 1 != 0 {
+        if chars.len() != cch * 2 {
+            return None;
+        }
+        let units: Vec<u16> = chars.as_chunks::<2>().0.iter().map(|u| u16::from_le_bytes(*u)).collect();
+        String::from_utf16_lossy(&units)
+    } else {
+        if chars.len() != cch {
+            return None;
+        }
+        chars.iter().map(|&b| char::from(b)).collect()
+    };
+    let items: Vec<String> = s.split('\0').map(|item| item.replace('"', "\"\"")).collect();
+    Some(format!("\"{}\"", items.join(",")))
+}

crates/xls/src/lib.rs

@@ -7,10 +7,10 @@
 //! results, and formulas as A1 text, including shared and array formulas), the cell formats
 //! (number formats, fonts, fills, borders, alignment, protection, with palette colours
 //! resolved), user-defined cell styles, merged cells, column and row records, sheet display
-//! flags, calculation settings, defined names, print areas and print titles, and the 1904 date
-//! flag. Widths and heights stay in the file's units.
+//! flags, calculation settings, defined names, print areas and print titles, data validation
+//! rules, and the 1904 date flag. Widths and heights stay in the file's units.
 //!
-//! Everything else (charts, pictures, comments, conditional formats, validation, hyperlinks,
+//! Everything else (charts, pictures, comments, conditional formats, hyperlinks,
 //! PivotTables, macros) is skipped and named in [`Read::warnings`]. Encrypted workbooks and
 //! versions before BIFF8 are refused ([`Error::Encrypted`], [`Error::Biff5`]). A formula that
 //! cannot be put back into text (links to other workbooks, macro commands, unknown tokens)
@@ -30,6 +30,7 @@
 //! # }
 //! ```
 
+mod dv;
 mod expr;
 mod formula;
 mod ftab;
@@ -42,8 +43,9 @@ pub mod testing;
 
 pub use craft_sheet::{DEFAULT_PALETTE, quote_sheet, sheet_needs_quotes};
 pub use model::{
-    Alignment, BorderLine, BorderStyle, Borders, Calc, CalcMode, Cell, CellError, CellFormat, CellRef, Color, Column, DefinedName, Fill, Font,
-    Formula, HAlign, Pattern, Range, Row, Script, Sheet, Underline, VAlign, Value, View, Visibility, Workbook,
+    Alignment, BorderLine, BorderStyle, Borders, Calc, CalcMode, Cell, CellError, CellFormat, CellRef, Color, Column, DataValidation, DefinedName,
+    ErrorStyle, Fill, Font, Formula, HAlign, Pattern, Range, Row, Script, Sheet, Underline, VAlign, ValidationOperator, ValidationType, Value, View,
+    Visibility, Workbook,
 };
 pub use read::{is_xls, read, read_with};
 
@@ -91,8 +93,15 @@ pub enum Warning {
     Drawings,
     #[error("conditional formatting was not read")]
     ConditionalFormatting,
-    #[error("data validation was not read")]
-    DataValidation,
+    /// Some Dv records could not be read: Excel would refuse them (an unknown alert style or
+    /// operator, a formula or range list past the record), no range was left, a formula could not
+    /// be decoded, or the sheet had more rules than the reader keeps.
+    #[error("some data validation rules could not be read and were skipped")]
+    DataValidationSkipped,
+    /// Dv records with no DVal record before them, or past the number of rules it gives: Excel
+    /// does not read them either.
+    #[error("some data validation records were ignored, as Excel ignores them: no DVal record before them, or past the number of rules it gives")]
+    DataValidationIgnored,
     #[error("hyperlinks were not read")]
     Hyperlinks,
     #[error("PivotTables were not read; their values were kept")]

crates/xls/src/model.rs

@@ -103,6 +103,75 @@ pub struct Sheet {
     pub print_area: Option<String>,
     /// The sheet's `Print_Titles` name as formula text (`Sheet1!$1:$2`).
     pub print_titles: Option<String>,
+    /// Data validation rules (Dv records), in file order.
+    pub validations: Vec<DataValidation>,
+}
+
+/// What a data validation rule allows ([MS-XLS] 2.4.95 `typOperator`'s sibling `valType`).
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
+pub enum ValidationType {
+    /// Any value; also the undefined types 8 to 15, which Excel opens as "any value".
+    #[default]
+    Any,
+    Whole,
+    Decimal,
+    List,
+    Date,
+    Time,
+    TextLength,
+    Custom,
+}
+
+/// How a rule compares the value with its formulas. Only whole, decimal, date, time and text
+/// length rules have one.
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
+pub enum ValidationOperator {
+    #[default]
+    Between,
+    NotBetween,
+    Equal,
+    NotEqual,
+    Greater,
+    Less,
+    GreaterOrEqual,
+    LessOrEqual,
+}
+
+/// The alert shown for a value the rule refuses.
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
+pub enum ErrorStyle {
+    #[default]
+    Stop,
+    Warning,
+    Information,
+}
+
+/// A data validation rule (Dv, [MS-XLS] 2.4.95), in the terms SpreadsheetML uses for the same
+/// rule (`dataValidation`).
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub struct DataValidation {
+    /// The ranges it covers. A range over every BIFF8 row (column) is a whole column (row) of
+    /// the 1,048,576 × 16,384 grid, as SpreadsheetML stores it.
+    pub ranges: Vec<Range>,
+    pub kind: ValidationType,
+    /// `Between` for the types without an operator.
+    pub operator: ValidationOperator,
+    /// The first formula as A1 text without `=`, relative references from the top left cell of
+    /// all the ranges. A literal list is quoted, its items separated by commas and each `"`
+    /// doubled (`"Red,Green"`), as SpreadsheetML's `formula1`. Empty for "any value".
+    pub formula1: String,
+    /// The upper bound of `Between` and `NotBetween`.
+    pub formula2: Option<String>,
+    pub allow_blank: bool,
+    /// A list shows its drop-down arrow in the cell.
+    pub in_cell_dropdown: bool,
+    pub prompt_title: String,
+    pub prompt: String,
+    pub show_prompt: bool,
+    pub error_title: String,
+    pub error: String,
+    pub error_style: ErrorStyle,
+    pub show_error: bool,
 }
 
 /// A user-defined name.

crates/xls/src/read.rs

@@ -438,6 +438,9 @@ struct SheetState<'g> {
     pending_string: Option<(CellRef, Option<u16>)>,
     dropped: usize,
     unsupported: Vec<&'static str>,
+    /// Dv records still to read: the last DVal's count, less those read since. Excel reads
+    /// exactly that many after a DVal, and none without one.
+    dv_left: u32,
 }
 
 impl SheetState<'_> {
@@ -590,6 +593,7 @@ fn worksheet<'g>(
         pending_string: None,
         dropped: 0,
         unsupported: vec![],
+        dv_left: 0,
     };
     let mut depth = 1usize;
     while let Some(rec) = recs.next_record()? {
@@ -798,7 +802,26 @@ fn sheet_record(rec: &Record<'_>, st: &mut SheetState<'_>, budget: &mut Budget,
         0x001C => report.warn(Warning::Comments),
         0x00EC => report.warn(Warning::Drawings),
         0x01B0 | 0x0879 => report.warn(Warning::ConditionalFormatting),
-        0x01B2 => report.warn(Warning::DataValidation),
+        // A short DVal starts no rules.
+        0x01B2 => st.dv_left = crate::dv::dval(&mut c).unwrap_or(0),
+        0x01BE => {
+            if st.dv_left == 0 {
+                report.warn(Warning::DataValidationIgnored);
+                return Ok(());
+            }
+            st.dv_left -= 1;
+            let rule = match crate::dv::dv(&mut c) {
+                Ok(raw) => {
+                    budget.text(raw.text_len())?;
+                    crate::dv::validation(raw, &st.g.links)
+                }
+                Err(_) => None,
+            };
+            match rule {
+                Some(v) if st.sheet.validations.len() < crate::dv::MAX_RULES => st.sheet.validations.push(v),
+                _ => report.warn(Warning::DataValidationSkipped),
+            }
+        }
         0x01B8 => report.warn(Warning::Hyperlinks),
         0x00B0 => report.warn(Warning::PivotTables),
         _ => {}

crates/xls/tests/validation.rs

@@ -0,0 +1,339 @@
+//! Data validation: DVal and Dv records ([MS-XLS] 2.4.96, 2.4.95), each assembled here from the
+//! spec's structures; no Excel-produced file is used. The expected rules are what a
+//! SpreadsheetML reader gives for the same rule. Ported from storytold/gridcraft#3636.
+
+mod common;
+
+use common::*;
+use craft_xls::{CellRef, DataValidation, ErrorStyle, Range, ValidationOperator, ValidationType, Workbook};
+
+/// A workbook of one worksheet, "Sheet1", holding `records`, with defined names (Lbl records,
+/// each a name and its formula) in the globals.
+fn workbook(names: &[(&str, &[u8])], records: &[Vec<u8>]) -> (Workbook, Report) {
+    let lbl = |name: &str, rgce: &[u8]| {
+        rec(0x0018, &cat(&[&u16s(&[0]), &[0, name.len() as u8], &u16s(&[rgce.len() as u16, 0, 0]), &[0; 4], &[0], name.as_bytes(), rgce]))
+    };
+    let lbls: Vec<u8> = names.iter().flat_map(|(n, f)| lbl(n, f)).collect();
+    let globals = |pos: u32| cat(&[&bof(0x0005), &rec(0x0085, &cat(&[&pos.to_le_bytes(), &[0, 0, 6, 0], b"Sheet1"])), &lbls, &eof()]);
+    let sheet = cat(&[&bof(0x0010), &records.concat(), &eof()]);
+    let stream = cat(&[&globals(globals(0).len() as u32), &sheet]);
+    read_xls(&compound("Workbook", &stream)).unwrap()
+}
+
+/// DVal: flags, the input window's position, no drop-down object (-1), then `count` rules.
+fn dval(count: u32) -> Vec<u8> {
+    rec(0x01B2, &cat(&[&[0, 0], &[0; 8], &(-1i32).to_le_bytes(), &count.to_le_bytes()]))
+}
+
+// Dv flag bits ([MS-XLS] 2.4.95).
+const STR_LOOKUP: u32 = 1 << 7;
+const ALLOW_BLANK: u32 = 1 << 8;
+const SUPPRESS_COMBO: u32 = 1 << 9;
+const SHOW_INPUT: u32 = 1 << 18;
+const SHOW_ERROR: u32 = 1 << 19;
+/// What Excel sets on a new rule: blanks allowed, input message and error alert shown.
+const DEFAULTS: u32 = ALLOW_BLANK | SHOW_INPUT | SHOW_ERROR;
+
+/// Dv flags: rule type, alert style, operator, plus `bits`.
+fn flags(kind: u32, alert: u32, op: u32, bits: u32) -> u32 {
+    kind | (alert << 4) | (op << 20) | bits
+}
+
+/// XLUnicodeString with compressed characters; "" is stored as Excel stores an empty message,
+/// a single NUL character.
+fn msg(s: &str) -> Vec<u8> {
+    xlstr(if s.is_empty() { "\0" } else { s })
+}
+
+/// XLUnicodeString with UTF-16 characters.
+fn wide(s: &str) -> Vec<u8> {
+    let units: Vec<u16> = s.encode_utf16().collect();
+    cat(&[&(units.len() as u16).to_le_bytes(), &[1], &u16s(&units)])
+}
+
+/// DVParsedFormula: size, 2 unused bytes (Excel leaves stray data there), the tokens.
+fn formula(rgce: &[u8]) -> Vec<u8> {
+    cat(&[&(rgce.len() as u16).to_le_bytes(), &[0x6E, 0x74], rgce])
+}
+
+/// SqRefU: a count, then each Ref8U as rows (first, last) and columns (first, last).
+fn sqref(ranges: &[[u16; 4]]) -> Vec<u8> {
+    let list: Vec<u8> = ranges.iter().flat_map(|r| u16s(r)).collect();
+    cat(&[&(ranges.len() as u16).to_le_bytes(), &list])
+}
+
+/// A Dv record with its four messages (prompt title, error title, prompt, error).
+fn dv(flags: u32, msgs: [Vec<u8>; 4], f1: &[u8], f2: &[u8], ranges: &[[u16; 4]]) -> Vec<u8> {
+    let [pt, et, p, e] = msgs;
+    rec(0x01BE, &cat(&[&flags.to_le_bytes(), &pt, &et, &p, &e, &formula(f1), &formula(f2), &sqref(ranges)]))
+}
+
+fn no_msgs() -> [Vec<u8>; 4] {
+    [msg(""), msg(""), msg(""), msg("")]
+}
+
+/// A Dv with Excel's default flags and no messages.
+fn rule(kind: u32, alert: u32, op: u32, f1: &[u8], f2: &[u8], ranges: &[[u16; 4]]) -> Vec<u8> {
+    dv(flags(kind, alert, op, DEFAULTS), no_msgs(), f1, f2, ranges)
+}
+
+fn range(a1: &str) -> Range {
+    Range::parse(a1).unwrap()
+}
+
+/// A rule with Excel's default flags and no messages.
+fn expected(ranges: &[&str], kind: ValidationType, operator: ValidationOperator, f1: &str, f2: Option<&str>) -> DataValidation {
+    DataValidation {
+        ranges: ranges.iter().map(|r| range(r)).collect(),
+        kind,
+        operator,
+        formula1: f1.into(),
+        formula2: f2.map(Into::into),
+        allow_blank: true,
+        in_cell_dropdown: true,
+        prompt_title: String::new(),
+        prompt: String::new(),
+        show_prompt: true,
+        error_title: String::new(),
+        error: String::new(),
+        error_style: ErrorStyle::Stop,
+        show_error: true,
+    }
+}
+
+fn hex(s: &str) -> Vec<u8> {
+    (0..s.len()).step_by(2).map(|i| u8::from_str_radix(&s[i..i + 2], 16).unwrap()).collect()
+}
+
+const SKIPPED: &str = "some data validation rules could not be read and were skipped";
+const IGNORED: &str =
+    "some data validation records were ignored, as Excel ignores them: no DVal record before them, or past the number of rules it gives";
+
+#[test]
+fn reads_every_kind_of_rule() {
+    use ValidationOperator::*;
+    use ValidationType::*;
+    let rules = [
+        // Whole number between 1 and 10, with all four messages and the stop alert.
+        dv(
+            flags(1, 0, 0, DEFAULTS),
+            [msg("Whole"), msg("Oops"), msg("Enter 1 to 10"), msg("Not 1..10")],
+            &hex("1e0100"),
+            &hex("1e0a00"),
+            &[[1, 5, 1, 1]],
+        ),
+        // Decimal >= 0.5, warning alert.
+        rule(2, 1, 6, &hex("1f000000000000e03f"), &[], &[[1, 5, 2, 2]]),
+        // Literal list (NUL-separated items), information alert.
+        dv(flags(3, 2, 0, DEFAULTS | STR_LOOKUP), no_msgs(), &hex("170e0052656400477265656e00426c7565"), &[], &[[1, 5, 3, 3]]),
+        // List from an absolute range ($H$2:$H$5).
+        rule(3, 0, 0, &hex("250100040007000700"), &[], &[[1, 5, 4, 4]]),
+        // Date between two serials, time < 0.5, text length <= 5, whole not between 5 and 7.
+        rule(4, 0, 0, &hex("1e37ab"), &hex("1ea4ac"), &[[1, 5, 5, 5]]),
+        rule(5, 0, 5, &hex("1f000000000000e03f"), &[], &[[1, 5, 6, 6]]),
+        rule(6, 0, 7, &hex("1e0500"), &[], &[[1, 5, 8, 8]]),
+        rule(1, 0, 1, &hex("1e0500"), &hex("1e0700"), &[[1, 5, 9, 9]]),
+        // Custom formula K2>A2 on K2:K6 (relative PtgRefN offsets 0 and -10 columns).
+        rule(7, 0, 0, &hex("4c000000c04c0000f6c00d"), &[], &[[1, 5, 10, 10]]),
+        // Any value, with an input message only.
+        dv(flags(0, 0, 0, DEFAULTS), [msg("Note"), msg(""), msg("Anything goes"), msg("")], &[], &[], &[[1, 5, 13, 13]]),
+        // A list from a defined name.
+        rule(3, 0, 0, &hex("2301000000"), &[], &[[1, 5, 16, 16]]),
+        // A dependent list: INDIRECT($S2) on T2:T6.
+        rule(3, 0, 0, &hex("190100004c0000128022019400"), &[], &[[1, 5, 19, 19]]),
+        // Decimal between two absolute cells.
+        rule(2, 0, 0, &hex("4406000700"), &hex("4407000700"), &[[1, 5, 22, 22]]),
+    ];
+    let mut records = vec![dval(rules.len() as u32)];
+    records.extend(rules);
+    let (wb, report) = workbook(&[("Colors", &[0x1E, 1, 0])], &records);
+    let got = &wb.sheets[0].validations;
+    let mut whole = expected(&["B2:B6"], Whole, Between, "1", Some("10"));
+    (whole.prompt_title, whole.error_title, whole.prompt, whole.error) = ("Whole".into(), "Oops".into(), "Enter 1 to 10".into(), "Not 1..10".into());
+    let mut decimal = expected(&["C2:C6"], Decimal, GreaterOrEqual, "0.5", None);
+    decimal.error_style = ErrorStyle::Warning;
+    let mut literal = expected(&["D2:D6"], List, Between, "\"Red,Green,Blue\"", None);
+    literal.error_style = ErrorStyle::Information;
+    let mut any = expected(&["N2:N6"], Any, Between, "", None);
+    (any.prompt_title, any.prompt) = ("Note".into(), "Anything goes".into());
+    let want = vec![
+        whole,
+        decimal,
+        literal,
+        expected(&["E2:E6"], List, Between, "$H$2:$H$5", None),
+        expected(&["F2:F6"], Date, Between, "43831", Some("44196")),
+        expected(&["G2:G6"], Time, Less, "0.5", None),
+        expected(&["I2:I6"], TextLength, LessOrEqual, "5", None),
+        expected(&["J2:J6"], Whole, NotBetween, "5", Some("7")),
+        expected(&["K2:K6"], Custom, Between, "K2>A2", None),
+        any,
+        expected(&["Q2:Q6"], List, Between, "Colors", None),
+        expected(&["T2:T6"], List, Between, "INDIRECT($S2)", None),
+        expected(&["W2:W6"], Decimal, Between, "$H$7", Some("$H$8")),
+    ];
+    assert_eq!(got.len(), want.len(), "{:?}", report.warnings);
+    for (g, w) in got.iter().zip(&want) {
+        assert_eq!(g, w);
+    }
+    assert!(report.warnings.is_empty(), "{:?}", report.warnings);
+}
+
+#[test]
+fn reads_flags_messages_lists_and_whole_columns() {
+    let off = flags(3, 0, 0, STR_LOOKUP | SUPPRESS_COMBO);
+    let records = [
+        dval(4),
+        dv(off, no_msgs(), &hex("170300610062"), &[], &[[1, 5, 14, 14]]),
+        dv(
+            flags(1, 0, 2, DEFAULTS),
+            [wide("Café"), wide("Ошибка"), wide("日本語のテスト é ü"), wide("Nur 3 ✓")],
+            &hex("1e0300"),
+            &[],
+            &[[1, 5, 20, 20]],
+        ),
+        // typOperator 6 on a list, as found in real files: ignored.
+        dv(flags(3, 0, 6, DEFAULTS | STR_LOOKUP), no_msgs(), &hex("170c00736179202268692200627965"), &[], &[[1, 5, 23, 23]]),
+        // Rows 0 to 65535: every row of the BIFF8 grid.
+        dv(flags(3, 0, 0, DEFAULTS | STR_LOOKUP), no_msgs(), &hex("170500780079007a"), &[], &[[0, 0xFFFF, 21, 21]]),
+    ];
+    let (wb, report) = workbook(&[], &records);
+    let got = &wb.sheets[0].validations;
+    assert_eq!(got.len(), 4, "{:?}", report.warnings);
+    let mut flags_off = expected(&["O2:O6"], ValidationType::List, ValidationOperator::Between, "\"a,b\"", None);
+    (flags_off.allow_blank, flags_off.in_cell_dropdown, flags_off.show_prompt, flags_off.show_error) = (false, false, false, false);
+    assert_eq!(got[0], flags_off);
+    let mut unicode = expected(&["U2:U6"], ValidationType::Whole, ValidationOperator::Equal, "3", None);
+    (unicode.prompt_title, unicode.error_title, unicode.prompt, unicode.error) =
+        ("Café".into(), "Ошибка".into(), "日本語のテスト é ü".into(), "Nur 3 ✓".into());
+    assert_eq!(got[1], unicode);
+    // The item `say "hi"` keeps its quotes, doubled; the operator is ignored.
+    assert_eq!(got[2], expected(&["X2:X6"], ValidationType::List, ValidationOperator::Between, "\"say \"\"hi\"\",bye\"", None));
+    assert_eq!(got[3].ranges, vec![range("V:V")]);
+}
+
+#[test]
+fn relative_references_start_from_the_top_left_of_all_ranges() {
+    let records = [
+        dval(3),
+        // =AE1>0 for AF5:AF9 AG1:AG3 (column offset -1 from AF1).
+        rule(7, 0, 0, &hex("4c0000ffc01e00000d"), &[], &[[4, 8, 31, 31], [0, 2, 32, 32]]),
+        // =AB1>0 for AC1:AC3 AA5:AA9 (column offset +1 from AA1).
+        rule(7, 0, 0, &hex("4c000001c01e00000d"), &[], &[[0, 2, 28, 28], [4, 8, 26, 26]]),
+        // PtgRefErr: a reference that fell off the grid.
+        rule(7, 0, 0, &hex("2a0000f4c01e00000d"), &[], &[[4, 8, 12, 12], [0, 2, 11, 11]]),
+    ];
+    let (wb, report) = workbook(&[], &records);
+    let got = &wb.sheets[0].validations;
+    assert_eq!(got.len(), 3, "{:?}", report.warnings);
+    assert_eq!(got[0].ranges, vec![range("AF5:AF9"), range("AG1:AG3")]);
+    assert_eq!(got[0].formula1, "AE1>0");
+    assert_eq!(got[1].formula1, "AB1>0");
+    assert_eq!(got[2].formula1, "#REF!>0");
+}
+
+#[test]
+fn only_the_dvals_count_of_rules_is_read() {
+    let a = || rule(1, 0, 4, &hex("1e0000"), &[], &[[0, 0, 0, 0]]);
+    let b = || rule(1, 0, 4, &hex("1e0000"), &[], &[[1, 1, 0, 0]]);
+    let (wb, report) = workbook(&[], &[dval(1), a(), b(), b()]);
+    assert_eq!(wb.sheets[0].validations.iter().map(|v| v.ranges.clone()).collect::<Vec<_>>(), vec![vec![range("A1")]]);
+    assert_eq!(report.warnings, [IGNORED]);
+    let (wb, report) = workbook(&[], &[a(), b()]);
+    assert!(wb.sheets[0].validations.is_empty());
+    assert_eq!(report.warnings, [IGNORED]);
+}
+
+#[test]
+fn malformed_rules_are_repaired_or_skipped() {
+    let ok = |row: u16| rule(1, 0, 4, &hex("1e0000"), &[], &[[row, row, 0, 0]]);
+    let cases: Vec<(&str, Vec<u8>, Option<&str>)> = vec![
+        // Excel opens an unknown type as "any value".
+        ("unknown type", rule(9, 0, 0, &hex("1e0100"), &[], &[[1, 5, 1, 1]]), Some("B2:B6")),
+        // Excel refuses an unknown alert style, or an unknown operator on a rule that uses one.
+        ("unknown alert", rule(1, 5, 0, &hex("1e0100"), &hex("1e0a00"), &[[1, 5, 1, 1]]), None),
+        ("unknown operator", rule(1, 0, 9, &hex("1e0100"), &[], &[[1, 5, 1, 1]]), None),
+        // Excel drops a reversed range, and clamps columns past IV.
+        ("reversed range", rule(1, 0, 4, &hex("1e0000"), &[], &[[5, 1, 2, 2]]), None),
+        ("past column IV", rule(1, 0, 4, &hex("1e0000"), &[], &[[1, 5, 3, 0x100]]), Some("D2:IV6")),
+        // Excel refuses a range count or a formula size that runs past the record.
+        (
+            "too many ranges",
+            rec(
+                0x01BE,
+                &cat(&[
+                    &flags(1, 0, 4, DEFAULTS).to_le_bytes(),
+                    &no_msgs().concat(),
+                    &formula(&hex("1e0000")),
+                    &formula(&[]),
+                    &u16s(&[2, 1, 5, 4, 4]),
+                ]),
+            ),
+            None,
+        ),
+        (
+            "formula past the record",
+            rec(0x01BE, &cat(&[&flags(1, 0, 4, DEFAULTS).to_le_bytes(), &no_msgs().concat(), &u16s(&[200, 0]), &hex("1e0000")])),
+            None,
+        ),
+        ("truncated message", rec(0x01BE, &cat(&[&flags(1, 0, 4, DEFAULTS).to_le_bytes(), &u16s(&[40]), &[0], b"short"])), None),
+        ("empty record", rec(0x01BE, &[]), None),
+        // A formula that can't be decoded (an unknown token).
+        ("unknown token", rule(7, 0, 0, &[0xFF], &[], &[[1, 5, 1, 1]]), None),
+        ("no ranges", rule(1, 0, 4, &hex("1e0000"), &[], &[]), None),
+    ];
+    for (what, bad, want) in cases {
+        let (wb, report) = workbook(&[], &[dval(3), ok(10), bad, ok(11)]);
+        let got = &wb.sheets[0].validations;
+        let ranges: Vec<Range> = got.iter().flat_map(|v| v.ranges.clone()).collect();
+        let mut expect = vec![range("A11")];
+        expect.extend(want.map(range));
+        expect.push(range("A12"));
+        assert_eq!(ranges, expect, "{what}: {:?}", report.warnings);
+        assert_eq!(report.warnings.iter().any(|w| w == SKIPPED), want.is_none(), "{what}: {:?}", report.warnings);
+        if what == "unknown type" {
+            assert_eq!((got[1].kind, got[1].formula1.as_str(), got[1].formula2.as_deref()), (ValidationType::Any, "", None));
+        }
+    }
+}
+
+#[test]
+fn hostile_validation_records_never_panic() {
+    let f = flags(1, 0, 4, DEFAULTS).to_le_bytes();
+    let big = [
+        rec(0x01B2, &[0xFF; 18]),
+        rec(0x01BE, &cat(&[&f, &u16s(&[0xFFFF]), &[1]])),
+        rec(0x01BE, &cat(&[&f, &no_msgs().concat(), &u16s(&[0xFFFF, 0xFFFF])])),
+        rec(0x01BE, &cat(&[&f, &no_msgs().concat(), &formula(&[]), &formula(&[]), &u16s(&[0xFFFF])])),
+        // A literal list whose PtgStr claims 255 UTF-16 characters and holds none.
+        rec(
+            0x01BE,
+            &cat(&[
+                &flags(3, 0, 0, DEFAULTS | STR_LOOKUP).to_le_bytes(),
+                &no_msgs().concat(),
+                &formula(&[0x17, 0xFF, 1]),
+                &formula(&[]),
+                &sqref(&[[0, 0, 0, 0]]),
+            ]),
+        ),
+        // Every flag bit set: type 15, alert 7, operator 15.
+        rec(0x01BE, &cat(&[&[0xFF; 4], &no_msgs().concat(), &formula(&[]), &formula(&[]), &sqref(&[[0, 0, 0, 0]])])),
+    ];
+    let (wb, _) = workbook(&[], &big);
+    assert!(wb.sheets[0].validations.is_empty());
+    let mut many = vec![dval(u32::MAX)];
+    many.extend((0..200u16).map(|r| rule(1, 0, 4, &hex("1e0000"), &[], &[[r, r, 0, 0]])));
+    let (wb, _) = workbook(&[], &many);
+    assert_eq!(wb.sheets[0].validations.len(), 200);
+    assert_eq!(wb.sheets[0].validations[199].ranges, vec![Range::new(CellRef::new(199, 0), CellRef::new(199, 0))]);
+}
+
+#[test]
+fn rules_past_the_cap_are_skipped() {
+    // A sheet keeps 65,536 rules; one more is skipped with a warning, not kept.
+    let one = rule(1, 0, 4, &hex("1e0000"), &[], &[[0, 0, 0, 0]]);
+    let mut records = vec![dval(65_537)];
+    records.extend(std::iter::repeat_n(one, 65_537));
+    let (wb, report) = workbook(&[], &records);
+    assert_eq!(wb.sheets[0].validations.len(), 65_536);
+    assert_eq!(report.warnings, [SKIPPED]);
+}