mirror of
https://github.com/storytold/gridcraft.git
synced 2026-11-16 15:23:57 +00:00
* Functions: GETPIVOTDATA (#284) The engine records each PivotTable report value cell (value field, row/column items, value) on refresh and when a workbook is opened; GETPIVOTDATA is an evaluator special form that looks the value up by value field and field/item pairs, #REF! when the report doesn't show it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * GETPIVOTDATA: shared lazy index, report values, hash lookup (review of #290) - PivotTable::data_index is an Arc-shared PivotLookup (serde-skipped), so the per-command workbook clone no longer copies the index; refresh sets it, clear resets it. - No index work on open: the first GETPIVOTDATA on a pivot without one builds it from the source via the engine-registered PIVOT_INDEXER; a failure is #REF!, not an empty index. - The result is the value the sheet shows at the cell's report position, so formula and report cell agree (test with a source edited after the last refresh). - Lookup by a pre-lowercased hash key; numbers/booleans fall back to a scan on a miss. - GETPIVOTDATA is no longer volatile: it depends on its pivot's whole report area. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * GETPIVOTDATA: keep the index of the stored report; collision-free keys; align estimates The lookup index describes the report as last refreshed and is saved with the pivot. A pivot loaded without one (XLSX) only builds it when the recomputed layout matches the stored labels. Lookup keys are length-prefixed and distinguish Boolean from text items. Function-parity estimates match. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * pivot: import PivotDataCell, PivotItemRef and PivotLookup explicitly after upstream dropped the glob import Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * pivot: add an ignored measurement of edits on a sheet with a large report Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * pivot: keep a failed lazy GETPIVOTDATA build retryable, never save an empty index A failed lazy index build stored unwrap_or_default() into the shared, first-set-wins PivotLookup, so a valid state coming later (undo, a source fix, a reopen) could never rebuild it, and the empty list was saved as a restored index. Cache only validated builds, treat an empty list as no index in serialization, reject reports with more value cells than the index cap instead of truncating, register the pivot indexer in gridcraft-cli check, and add two regression tests. * GETPIVOTDATA: move cached offsets with edits inside the report; exact numeric items first Inserting or deleting rows or columns inside a report remapped the cells and last_range but kept the cached value-cell offsets, so a lookup could read another item's cell. shift_sheet_features now maps each indexed cell through the edit into a new index (the shared one stays with the undo snapshots); a value cell deleted with its line drops out (#REF!). A numeric item was accepted on a formatted-label hash hit, so 0.6 and 1 both shown as "1" returned the first. A number now prefers the cell whose item has exactly that value, before any label match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix pivot chart test for the new Output.data field Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
19f07f3a25
commit
ef99f1e51f
21 files changed
+846
-37
No files matched your search
@@ -205,7 +205,7 @@ egui frontend is a separate crate, so the UI can be swapped without touching the
|
||||
|
||||
GridCraft is in **alpha**: the core workflow works end to end, but depth, fidelity and file
|
||||
compatibility are still rough. It covers 89% of standard ribbon commands in our catalog
|
||||
([`docs/parity-checklist.md`](docs/parity-checklist.md)) and 513 standard worksheet functions
|
||||
([`docs/parity-checklist.md`](docs/parity-checklist.md)) and 514 standard worksheet functions
|
||||
([`docs/function-parity.md`](docs/function-parity.md)); this is a parity inventory, not a count of working implementations. Weighted by what spreadsheet users rely on, we
|
||||
estimate it is about 50% ready for real work. Next up: XLSX fidelity on real-world files, very large
|
||||
workbooks and multi-threaded recalc, chart depth, text in every script, and `.xls` import. Stage,
|
||||
|
||||
+4
-2
@@ -22,7 +22,7 @@ This page is the summary. The detail is in:
|
||||
| Number | Value | Kind |
|
||||
|---|---|---|
|
||||
| 2026-10-14 | Editing a cell with text that would read back as a number, boolean, date or formula (entered with a leading apostrophe) shows the apostrophe in the editor and formula bar, so Enter unchanged keeps it text (#1362) |
|
||||
| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 281 / 296 (94.9%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) |
|
||||
| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 281 / 296 (94.9%), worksheet functions 514 / 523 (98.3%; 505 working, 96.6%) |
|
||||
| **Ready for real work** | **~50%** | estimated from written weights (51.6); [target-app-parity.md](docs/target-app-parity.md) |
|
||||
| **Mainstream practitioner** | **~50%** | estimated: typical analyst/finance user, weekly features, × 0.90 interaction × 0.92 stability × 0.88 file exchange; [method](docs/target-app-parity.md#mainstream-practitioner) |
|
||||
| **Essentials user** | **~63%** | estimated: core features only, × 0.92 launch/stability × 0.95 clarity × 0.90 opening sent files; [method](docs/target-app-parity.md#essentials-user) |
|
||||
@@ -74,7 +74,7 @@ collaboration, mobile, AI).
|
||||
| Area | Breadth | Ready | Hours | Doc |
|
||||
|---|---|---|---|---|
|
||||
| Formula language and calculation | ~92% | ~72% | 20–30 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) |
|
||||
| Worksheet functions | 96.6% measured | ~82% | 16–26 | [function-parity.md](docs/function-parity.md) |
|
||||
| Worksheet functions | 96.6% measured | ~82% | 15–25 | [function-parity.md](docs/function-parity.md) |
|
||||
| Editing, navigation, clipboard, fill | ~90% | ~70% | 15–25 | [ui-parity.md](docs/ui-parity.md) |
|
||||
| Cell formatting, styles, conditional formatting | ~95% | ~80% | 8–14 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) |
|
||||
| Sort, filter, tables, validation, outline | ~82% | ~70% | 10–16 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) |
|
||||
@@ -202,6 +202,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
|
||||
| 2026-10-13 | Conditional formatting: a rule gathers range statistics only when it reads them (top/bottom, above average, duplicates, scales, bars, icons) and borrows them instead of copying them per cell, so the per-frame lookup of an expression or cell-value rule no longer visits its whole range (#1098, part 1) |
|
||||
| 2026-10-13 | Grid: resizing several selected columns or rows previews on the dragged one and applies the size to all on release; Esc cancels (#1082) |
|
||||
| 2026-10-12 | Next/previous sheet navigation stops at the first/last visible worksheet instead of activating a hidden boundary sheet; the current selection is retained |
|
||||
| 2026-10-12 | GETPIVOTDATA reads PivotTable values by field and items (#284) |
|
||||
| 2026-10-12 | PDF writer: objects are written straight to the output, identical pictures are stored once, style operators are emitted only when they change, `q` nesting is limited to 28 and polyline joins no longer leak into later strokes (#816) |
|
||||
| 2026-10-12 | Insert › Text › WordArt (`insert.wordArt`, `wordArt.set`): floating decorative text with 10 original styles, outline and shadow, in-place editing, XLSX and print (#784) |
|
||||
| 2026-10-12 | Number formatting: one table of built-in formats (xlsx); numbers, dates and General render with one scratch buffer instead of per-token strings; output unchanged, checked against digests recorded from the old code (#817) |
|
||||
@@ -321,6 +322,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
|
||||
| 2026-10-13 | trivial | Progress log: shared pictures written once on XLSX export (#985) |
|
||||
| 2026-10-13 | minor | Progress log: conditional-format lookups no longer scan the range for rules without statistics (#1098) |
|
||||
| 2026-10-12 | minor | Progress log: hidden-sheet navigation at workbook boundaries |
|
||||
| 2026-10-12 | minor | Worksheet functions 514 / 523 (GETPIVOTDATA); progress log row |
|
||||
| 2026-10-13 | trivial | Progress log: charts cached between frames (#1139) |
|
||||
| 2026-10-12 | minor | Progress log: PDF writer cleanup (#816) |
|
||||
| 2026-10-12 | minor | Catalog 276 → 277 / 292: Insert › WordArt (#784) |
|
||||
|
||||
@@ -368,6 +368,8 @@ pub fn check_bytes(path: &str, bytes: &[u8], opts: &Options) -> FileReport {
|
||||
}
|
||||
|
||||
fn check_inner(path: &str, bytes: &[u8], opts: &Options) -> FileReport {
|
||||
// GETPIVOTDATA needs the engine's pivot indexer; without it every such cell reads #REF!.
|
||||
let _ = gridcraft_engine::model::PIVOT_INDEXER.set(gridcraft_engine::pivot::index_cells);
|
||||
let mut report = FileReport { path: path.to_string(), ..Default::default() };
|
||||
let (cached, warnings) = match gridcraft_engine::io::open_bytes(path, bytes) {
|
||||
Ok(x) => x,
|
||||
|
||||
@@ -72,6 +72,7 @@ impl<'h> Evaluator<'h> {
|
||||
"XLOOKUP" => self.xlookup(args),
|
||||
"CELL" => self.cell_info(args),
|
||||
"TRIMRANGE" => self.trimrange(args),
|
||||
"GETPIVOTDATA" => self.getpivotdata(args),
|
||||
"SUBTOTAL" | "AGGREGATE" => self.subtotal(name, args),
|
||||
"TEXT" => {
|
||||
if args.len() != 2 {
|
||||
|
||||
@@ -136,6 +136,25 @@ impl Precedents<'_> {
|
||||
(_, Some(Expr::Number(_) | Expr::Text(_) | Expr::Bool(_) | Expr::Error(_) | Expr::Array(_) | Expr::Missing)) => {}
|
||||
_ => self.dynamic = true,
|
||||
},
|
||||
// GETPIVOTDATA reads the report cells, so it depends on the whole report its reference
|
||||
// points into (pivot commands recalculate everything). Any other reference is dynamic.
|
||||
Expr::Call(n, args) if n == "GETPIVOTDATA" => match args.get(1) {
|
||||
Some(Expr::Ref(r)) => {
|
||||
let si = match &r.sheet {
|
||||
SheetSel::Current => Some(self.sheet),
|
||||
SheetSel::Named(name) => self.wb.sheet_index(name),
|
||||
SheetSel::Span(..) => None,
|
||||
};
|
||||
let report = si
|
||||
.and_then(|i| self.wb.sheet(i))
|
||||
.and_then(|sh| sh.pivots.iter().find_map(|p| p.last_range.filter(|lr| lr.intersects(&r.range()))));
|
||||
match report {
|
||||
Some(lr) => self.push(r, lr, boxes),
|
||||
None => self.dynamic = true,
|
||||
}
|
||||
}
|
||||
_ => self.dynamic = true,
|
||||
},
|
||||
Expr::Call(n, _)
|
||||
if matches!(n.as_str(), "INDIRECT" | "OFFSET" | "NOW" | "TODAY" | "RAND" | "RANDBETWEEN" | "RANDARRAY" | "CELL" | "INFO") =>
|
||||
{
|
||||
@@ -199,6 +218,7 @@ fn is_special(n: &str) -> bool {
|
||||
| "INDEX"
|
||||
| "CELL"
|
||||
| "TRIMRANGE"
|
||||
| "GETPIVOTDATA"
|
||||
| "SUBTOTAL"
|
||||
| "AGGREGATE"
|
||||
| "TEXT"
|
||||
@@ -237,6 +257,7 @@ pub const SPECIAL_FUNCTIONS: &[&str] = &[
|
||||
"INDEX",
|
||||
"CELL",
|
||||
"TRIMRANGE",
|
||||
"GETPIVOTDATA",
|
||||
"SUBTOTAL",
|
||||
"AGGREGATE",
|
||||
"TEXT",
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
#![forbid(unsafe_code)]
|
||||
|
||||
pub mod eval;
|
||||
mod pivotdata;
|
||||
mod precision;
|
||||
pub mod recalc;
|
||||
mod spill_index;
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
//! GETPIVOTDATA: a value from a PivotTable report, found by its value field and the row and
|
||||
//! column items that locate it (Excel's documented semantics). Each PivotTable's value cells are
|
||||
//! indexed in `PivotTable::data_index` (set on refresh and saved with the report; built on first
|
||||
//! use only for a pivot loaded without one, and only when it matches the stored report); the
|
||||
//! result is read from the report cell the sheet shows.
|
||||
|
||||
use gridcraft_core::{CellError, CellRef, Value};
|
||||
use gridcraft_formula::Expr;
|
||||
use gridcraft_model::{PIVOT_INDEXER, PivotDataCell, PivotItemRef, PivotLookup, PivotTable};
|
||||
|
||||
use crate::eval::{Ev, Evaluator};
|
||||
|
||||
/// Excel allows 126 field/item pairs.
|
||||
const MAX_PAIRS: usize = 126;
|
||||
|
||||
fn same_text(a: &str, b: &str) -> bool {
|
||||
a.trim().to_lowercase() == b.trim().to_lowercase()
|
||||
}
|
||||
|
||||
/// A number matches an item with that value (dates as serials) or with that label; anything
|
||||
/// else matches the label as text. With `exact`, a number matches only an item of that value.
|
||||
fn item_matches(item: &PivotItemRef, v: &Value, exact: bool) -> bool {
|
||||
if let Value::Number(n) = v {
|
||||
if let Value::Number(x) = &item.value
|
||||
&& n == x
|
||||
{
|
||||
return true;
|
||||
}
|
||||
if exact {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if let Value::Bool(b) = v {
|
||||
return item.value == Value::Bool(*b);
|
||||
}
|
||||
v.to_text().is_ok_and(|t| same_text(&t, &item.label))
|
||||
}
|
||||
|
||||
/// Whether `cell` is the value field asked for with exactly these items.
|
||||
fn cell_matches(cell: &PivotDataCell, data_field: &str, pairs: &[(String, Value)], exact: bool) -> bool {
|
||||
(same_text(&cell.data_field, data_field) || same_text(&cell.source_col, data_field))
|
||||
&& cell.items.len() == pairs.len()
|
||||
&& pairs.iter().all(|(f, v)| cell.items.iter().any(|it| same_text(&it.field, f) && item_matches(it, v, exact)))
|
||||
}
|
||||
|
||||
fn is_axis_field(pt: &PivotTable, field: &str) -> bool {
|
||||
pt.rows.iter().chain(pt.columns.iter()).any(|f| same_text(&f.source_col, field))
|
||||
}
|
||||
|
||||
/// The value cell whose value field and items are exactly those asked for. Text items hit the
|
||||
/// hash index; numbers and booleans also match by value, which a miss falls back to scanning for.
|
||||
/// A number prefers the item of that value: two numeric items can show the same label (0.6 and 1
|
||||
/// formatted as `0`), so a label hit counts only when no item has the exact value.
|
||||
fn find<'a>(cells: &'a PivotLookup, data_field: &str, pairs: &[(String, Value)]) -> Option<&'a PivotDataCell> {
|
||||
let labels: Option<Vec<(&str, String, bool)>> =
|
||||
pairs.iter().map(|(f, v)| v.to_text().ok().map(|t| (f.as_str(), t, matches!(v, Value::Bool(_))))).collect();
|
||||
let hit = labels.as_ref().and_then(|labels| {
|
||||
let refs: Vec<(&str, &str, bool)> = labels.iter().map(|(f, t, b)| (*f, t.as_str(), *b)).collect();
|
||||
cells.find(data_field, &refs)
|
||||
});
|
||||
if pairs.iter().all(|(_, v)| matches!(v, Value::Text(_))) {
|
||||
return hit;
|
||||
}
|
||||
if let Some(c) = hit.filter(|c| cell_matches(c, data_field, pairs, true)) {
|
||||
return Some(c);
|
||||
}
|
||||
let all = cells.cells()?;
|
||||
all.iter()
|
||||
.find(|cell| cell_matches(cell, data_field, pairs, true))
|
||||
.or(hit)
|
||||
.or_else(|| all.iter().find(|cell| cell_matches(cell, data_field, pairs, false)))
|
||||
}
|
||||
|
||||
impl Evaluator<'_> {
|
||||
pub(crate) fn getpivotdata(&mut self, args: &[Expr]) -> Ev {
|
||||
match self.pivot_value(args) {
|
||||
Ok(v) => Ev::V(v),
|
||||
Err(e) => Ev::V(Value::Error(e)),
|
||||
}
|
||||
}
|
||||
|
||||
fn scalar_arg(&mut self, e: &Expr) -> Result<Value, CellError> {
|
||||
match self.value(e).scalar() {
|
||||
Value::Error(e) => Err(e),
|
||||
v => Ok(v),
|
||||
}
|
||||
}
|
||||
|
||||
fn pivot_value(&mut self, args: &[Expr]) -> Result<Value, CellError> {
|
||||
let (Some(field_expr), Some(table_expr)) = (args.first(), args.get(1)) else { return Err(CellError::Value) };
|
||||
let rest = args.get(2..).unwrap_or_default();
|
||||
if rest.len() % 2 != 0 || rest.len() / 2 > MAX_PAIRS {
|
||||
return Err(CellError::Ref);
|
||||
}
|
||||
let data_field = self.scalar_arg(field_expr)?.to_text()?;
|
||||
let area = match self.eval(table_expr) {
|
||||
Ev::R(areas) => areas.first().copied().ok_or(CellError::Ref)?,
|
||||
Ev::V(Value::Error(e)) => return Err(e),
|
||||
_ => return Err(CellError::Ref),
|
||||
};
|
||||
let mut pairs: Vec<(String, Value)> = Vec::with_capacity(rest.len() / 2);
|
||||
for pair in rest.chunks(2) {
|
||||
let (Some(f), Some(i)) = (pair.first(), pair.get(1)) else { return Err(CellError::Ref) };
|
||||
let field = self.scalar_arg(f)?.to_text()?;
|
||||
let item = self.scalar_arg(i)?;
|
||||
if pairs.iter().any(|(g, _)| same_text(g, &field)) {
|
||||
return Err(CellError::Ref);
|
||||
}
|
||||
pairs.push((field, item));
|
||||
}
|
||||
let wb = self.host.workbook();
|
||||
let sheet = wb.sheet(area.sheet).ok_or(CellError::Ref)?;
|
||||
let pt = sheet.pivots.iter().find(|p| p.last_range.is_some_and(|r| r.intersects(&area.range))).ok_or(CellError::Ref)?;
|
||||
// A report filter field may be named when the report shows exactly that one item.
|
||||
let mut axis_pairs = Vec::with_capacity(pairs.len());
|
||||
for (field, item) in pairs {
|
||||
if is_axis_field(pt, &field) {
|
||||
axis_pairs.push((field, item));
|
||||
continue;
|
||||
}
|
||||
let filter = pt.filters.iter().find(|f| same_text(&f.source_col, &field)).ok_or(CellError::Ref)?;
|
||||
let shown = match filter.selected.as_deref() {
|
||||
Some([one]) => item.to_text().is_ok_and(|t| same_text(&t, one)),
|
||||
_ => false,
|
||||
};
|
||||
if !shown {
|
||||
return Err(CellError::Ref);
|
||||
}
|
||||
}
|
||||
// The report's value cells. A pivot loaded without them builds them once; a stored report
|
||||
// that no longer matches its source gets no index at all, so the lookup is #REF! until a
|
||||
// refresh — never an index cached from a failed build, because the shared lookup keeps
|
||||
// the first one set and undo or reopen could then never rebuild it.
|
||||
if pt.data_index.cells().is_none() {
|
||||
let build = PIVOT_INDEXER.get().ok_or(CellError::Ref)?;
|
||||
if let Ok(cells) = build(wb, area.sheet, pt) {
|
||||
pt.data_index.set_cells(cells);
|
||||
}
|
||||
}
|
||||
let cell = find(&pt.data_index, &data_field, &axis_pairs).ok_or(CellError::Ref)?;
|
||||
// The value is the one the sheet shows, so the formula and the report cell always agree.
|
||||
let report = pt.last_range.ok_or(CellError::Ref)?;
|
||||
let at = CellRef::new(report.start.row.saturating_add(cell.row), report.start.col.saturating_add(cell.col));
|
||||
if !report.contains(at) {
|
||||
return Err(CellError::Ref);
|
||||
}
|
||||
Ok(sheet.value(at))
|
||||
}
|
||||
}
|
||||
@@ -290,6 +290,12 @@ pub fn function_list() -> Vec<Json> {
|
||||
("SHEETS", "Information", "SHEETS([reference])", "Number of sheets."),
|
||||
("AREAS", "Lookup", "AREAS(reference)", "Number of areas in a reference."),
|
||||
("TRIMRANGE", "Lookup", "TRIMRANGE(range, [trim_rows], [trim_cols])", "A range or array without its blank outer rows and columns."),
|
||||
(
|
||||
"GETPIVOTDATA",
|
||||
"Lookup",
|
||||
"GETPIVOTDATA(data_field, pivot_table, [field1, item1], ...)",
|
||||
"A value from a PivotTable report, by value field and items.",
|
||||
),
|
||||
("ISOMITTED", "Information", "ISOMITTED(argument)", "TRUE when a LAMBDA argument was left out."),
|
||||
];
|
||||
for (n, c, sig, d) in specials {
|
||||
|
||||
@@ -628,6 +628,312 @@ fn calculated_field_errors() {
|
||||
assert!(matches!(s.ui_requests.last(), Some(crate::UiRequest::Dialog(n, _)) if n == "pivotCalculatedField"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_reads_the_report() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales"]})).unwrap();
|
||||
assert_eq!(pt(&s).last_range.unwrap().a1(), "A3:E8");
|
||||
let gp = |s: &mut Session, cell: &str, f: &str| {
|
||||
s.execute("cell.set", json!({"cell": cell, "input": f})).unwrap();
|
||||
val(s, cell)
|
||||
};
|
||||
// Item cell, row subtotal by any cell of the report, grand total; names and items ignore case.
|
||||
assert_eq!(gp(&mut s, "G1", r#"=GETPIVOTDATA("Sales",$A$3,"Region","East","Product","Apples")"#), Value::Number(160.0));
|
||||
assert_eq!(gp(&mut s, "G2", r#"=GETPIVOTDATA("sales",C6,"region","east")"#), Value::Number(415.0));
|
||||
assert_eq!(gp(&mut s, "G3", r#"=GETPIVOTDATA("Sum of Sales",A3)"#), Value::Number(TOTAL));
|
||||
assert_eq!(gp(&mut s, "G4", r#"=GETPIVOTDATA("Sales",A3,"Product","Pears")"#), Value::Number(270.0));
|
||||
// Not in the report: unknown item, value field, field, unpaired field, not a pivot cell.
|
||||
for (cell, f) in [
|
||||
("H1", r#"=GETPIVOTDATA("Sales",A3,"Region","South")"#),
|
||||
("H2", r#"=GETPIVOTDATA("Units",A3)"#),
|
||||
("H3", r#"=GETPIVOTDATA("Sales",A3,"Date","East")"#),
|
||||
("H4", r#"=GETPIVOTDATA("Sales",A3,"Region")"#),
|
||||
("H5", r#"=GETPIVOTDATA("Sales",Z99)"#),
|
||||
("H6", r#"=GETPIVOTDATA("Sales",A3,"Region","East","Region","East")"#),
|
||||
] {
|
||||
assert_eq!(gp(&mut s, cell, f), Value::Error(CellError::Ref), "{f}");
|
||||
}
|
||||
assert_eq!(gp(&mut s, "H7", r#"=GETPIVOTDATA("Sales",A3,"Region",NA())"#), Value::Error(CellError::NA));
|
||||
// A refresh after a source edit updates the result.
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "E2", "input": "1100"})).unwrap();
|
||||
s.execute("data.refreshAll", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(val(&s, "G1"), Value::Number(1160.0));
|
||||
// Opening a saved workbook starts without an index (files don't carry it); the first
|
||||
// GETPIVOTDATA builds it.
|
||||
let wb = (*s.doc().unwrap().wb).clone();
|
||||
let (back, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(&wb).unwrap()).unwrap();
|
||||
s.add_document(crate::DocState::new(back, None, "Reopened".into(), crate::Trust::FromFile));
|
||||
let d = s.doc().unwrap();
|
||||
let sh = d.wb.sheet(d.wb.sheet_index("Sheet2").unwrap()).unwrap();
|
||||
assert!(sh.pivots.first().unwrap().data_index.cells().is_some());
|
||||
assert_eq!(sh.value(CellRef::parse("G1").unwrap()), Value::Number(1160.0));
|
||||
assert_eq!(sh.value(CellRef::parse("G3").unwrap()), Value::Number(TOTAL + 1000.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_reads_the_displayed_report() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
// The source changes after the last refresh, then the workbook is saved and reopened:
|
||||
// the report still shows the old numbers, and the formula agrees with the cell.
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "E2", "input": "1100"})).unwrap();
|
||||
let wb = (*s.doc().unwrap().wb).clone();
|
||||
let (back, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(&wb).unwrap()).unwrap();
|
||||
s.add_document(crate::DocState::new(back, None, "Reopened".into(), crate::Trust::FromFile));
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
let r = pt(&s).last_range.unwrap();
|
||||
let at = r.start.a1();
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": format!(r#"=GETPIVOTDATA("Sales",{at},"Region","East")"#)})).unwrap();
|
||||
let d = s.doc().unwrap();
|
||||
let sh = d.wb.sheet(d.wb.sheet_index("Sheet2").unwrap()).unwrap();
|
||||
let east = (r.start.row..=r.end.row).find(|row| sh.value(CellRef::new(*row, r.start.col)) == Value::Text("East".into())).unwrap();
|
||||
let shown = sh.value(CellRef::new(east, r.start.col + 1));
|
||||
assert_eq!(shown, Value::Number(415.0));
|
||||
assert_eq!(val(&s, "K1"), shown);
|
||||
}
|
||||
|
||||
/// `=GETPIVOTDATA("Sales", <report>, "Region", <region>)` evaluated in K1 of the active sheet.
|
||||
fn region_sales(s: &mut Session, region: &str) -> Value {
|
||||
let at = pt(s).last_range.unwrap().start.a1();
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": format!(r#"=GETPIVOTDATA("Sales",{at},"Region","{region}")"#)})).unwrap();
|
||||
val(s, "K1")
|
||||
}
|
||||
|
||||
fn reopen(s: &mut Session) {
|
||||
let wb = (*s.doc().unwrap().wb).clone();
|
||||
let (back, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(&wb).unwrap()).unwrap();
|
||||
s.add_document(crate::DocState::new(back, None, "Reopened".into(), crate::Trust::FromFile));
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_retries_after_a_lazy_build_fails() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
// A pivot read from a file carries no index, so the first lookup builds one.
|
||||
reopen(&mut s);
|
||||
assert!(pt(&s).data_index.cells().is_none());
|
||||
// The source changes, so the stored report is stale and the build finds nothing to trust.
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "Alpha"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Error(CellError::Ref));
|
||||
assert!(pt(&s).data_index.cells().is_none(), "the failed build caches nothing");
|
||||
// The lookup is shared, so caching a failure as an empty index would leave every state that
|
||||
// shares it #REF! for good. Restoring the source lets the lookup build the index.
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "East"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
assert!(pt(&s).data_index.cells().is_some(), "the lookup rebuilt after the failure");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_saves_no_index_when_the_lazy_build_fails() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
// A pivot read from a file carries no index; a source change makes the stored report stale,
|
||||
// so the lazy build fails and the lookup is #REF!.
|
||||
reopen(&mut s);
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "Alpha"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Error(CellError::Ref));
|
||||
// Saving the pivot must save no index at all, not an empty one: restoring an empty index
|
||||
// would count as an index and lock the reopened pivot against the lazy rebuild.
|
||||
let json = serde_json::to_string(&pt(&s)).unwrap();
|
||||
let back: gridcraft_model::PivotTable = serde_json::from_str(&json).unwrap();
|
||||
assert!(json.contains(r#""data_index":null"#), "a failed build saves no index, not an empty one: {json}");
|
||||
assert!(back.data_index.cells().is_none());
|
||||
// Reopened with the source restored, the lookup builds the index.
|
||||
reopen(&mut s);
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "East"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_keeps_the_refreshed_report_when_an_item_is_inserted() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
// A new first item in the source shifts every item of a fresh report down one row.
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "Alpha"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
// Before the refresh the report still shows the old items, and the lookup reads that report.
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
assert_eq!(region_sales(&mut s, "Alpha"), Value::Error(CellError::Ref));
|
||||
// Saved and reopened unrefreshed: the stored report no longer matches the source, so there
|
||||
// is no index to trust (never the shifted cell of a different item).
|
||||
reopen(&mut s);
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Error(CellError::Ref));
|
||||
assert_eq!(region_sales(&mut s, "Alpha"), Value::Error(CellError::Ref));
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("data.refreshAll", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "Alpha"), Value::Number(100.0));
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
assert_eq!(region_sales(&mut s, "East"), Value::Number(315.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_keeps_the_refreshed_report_when_an_item_is_removed() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "North"), Value::Number(135.0));
|
||||
// Every North row becomes East: North leaves the report and West moves up.
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
for a in ["A5", "A8", "A11"] {
|
||||
s.execute("cell.set", json!({"cell": a, "input": "East"})).unwrap();
|
||||
}
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "North"), Value::Number(135.0));
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
reopen(&mut s);
|
||||
assert_eq!(region_sales(&mut s, "North"), Value::Error(CellError::Ref));
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Error(CellError::Ref));
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("data.refreshAll", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
assert_eq!(region_sales(&mut s, "North"), Value::Error(CellError::Ref));
|
||||
assert_eq!(region_sales(&mut s, "West"), Value::Number(285.0));
|
||||
assert_eq!(region_sales(&mut s, "East"), Value::Number(550.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_index_is_saved_with_an_unrefreshed_pivot() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "Alpha"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap();
|
||||
// The pivot as saved by the native format: the index of the stored report comes back with
|
||||
// it even though the source has since changed.
|
||||
let back: gridcraft_model::PivotTable = serde_json::from_str(&serde_json::to_string(&pt(&s)).unwrap()).unwrap();
|
||||
let cells = back.data_index.cells().expect("index saved");
|
||||
assert_eq!(cells, pt(&s).data_index.cells().unwrap());
|
||||
let west = back.data_index.find("Sales", &[("Region", "West", false)]).unwrap();
|
||||
assert_eq!((west.row, west.col), (3, 1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_calculated_field() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap();
|
||||
s.execute("pivot.calculatedField", json!({"name": "Double", "formula": "=Sales*2"})).unwrap();
|
||||
let at = pt(&s).last_range.unwrap().start.a1();
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": format!(r#"=GETPIVOTDATA("Double",{at},"Region","East")"#)})).unwrap();
|
||||
assert_eq!(val(&s, "K1"), Value::Number(830.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn getpivotdata_report_filter() {
|
||||
let mut s = sales();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Product"], "values": ["Sales"], "filters": ["Region"]})).unwrap();
|
||||
let at = pt(&s).last_range.unwrap().start.a1();
|
||||
let f = |region: &str| format!(r#"=GETPIVOTDATA("Sales",{at},"Region","{region}","Product","Apples")"#);
|
||||
// With every item shown, naming the filter field is #REF!.
|
||||
s.execute("cell.set", json!({"cell": "H1", "input": f("East")})).unwrap();
|
||||
assert_eq!(val(&s, "H1"), Value::Error(CellError::Ref));
|
||||
// Filtered to East: East matches, West doesn't.
|
||||
s.execute("pivot.filter", json!({"field": "Region", "selected": ["East"]})).unwrap();
|
||||
assert_eq!(val(&s, "H1"), Value::Number(160.0));
|
||||
s.execute("cell.set", json!({"cell": "H2", "input": f("West")})).unwrap();
|
||||
assert_eq!(val(&s, "H2"), Value::Error(CellError::Ref));
|
||||
}
|
||||
|
||||
/// A workbook whose Sheet1 holds `rows` under a two-column header, with a PivotTable of the first
|
||||
/// column (rows) and the second (sum) on a new sheet; the report sheet is active.
|
||||
fn two_column_pivot(header: [&str; 2], rows: Json, key_format: Option<&str>) -> Session {
|
||||
let mut s = Session::new();
|
||||
s.new_workbook();
|
||||
let n = rows.as_array().map_or(0, Vec::len);
|
||||
let mut all = vec![json!(header)];
|
||||
all.extend(rows.as_array().cloned().unwrap_or_default());
|
||||
s.execute("range.setValues", json!({"range": "A1", "values": all})).unwrap();
|
||||
if let Some(code) = key_format {
|
||||
s.execute("home.numberFormat", json!({"range": format!("A2:A{}", n + 1), "code": code})).unwrap();
|
||||
}
|
||||
s.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
s.execute("insert.pivotTable", json!({"rows": [header[0]], "values": [header[1]]})).unwrap();
|
||||
s
|
||||
}
|
||||
|
||||
/// The value the report shows for the row item `label` (one value field: the report's last column).
|
||||
fn shown(s: &Session, label: &str) -> Value {
|
||||
let r = pt(s).last_range.unwrap();
|
||||
let row = find_row(s, label).unwrap_or_else(|| panic!("{label} is not in the report"));
|
||||
val(s, &CellRef::new(row, r.end.col).a1())
|
||||
}
|
||||
|
||||
/// Inserting or deleting rows or columns inside a report keeps GETPIVOTDATA on the requested
|
||||
/// item's cell (the stored offsets move with the cells), and undo restores the old positions.
|
||||
#[test]
|
||||
fn getpivotdata_follows_rows_and_columns_edited_inside_the_report() {
|
||||
let mut s = two_column_pivot(["Region", "Sales"], json!([["East", 10], ["North", 30], ["West", 20]]), None);
|
||||
assert_eq!(pt(&s).last_range.unwrap().start.a1(), "A3");
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": r#"=GETPIVOTDATA("Sales",A3,"Region","West")"#})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "K2", "input": r#"=GETPIVOTDATA("Sales",A3,"Region","North")"#})).unwrap();
|
||||
assert_eq!(val(&s, "K1"), Value::Number(20.0));
|
||||
// Row 4 splits the header from the items; column B splits the labels from the values (and
|
||||
// moves the formulas from K to L).
|
||||
for (cmd, lines, west, north) in
|
||||
[("home.insertRows", json!({"rows": "4:4"}), "K1", "K2"), ("home.insertColumns", json!({"cols": "B:B"}), "L1", "L2")]
|
||||
{
|
||||
s.execute(cmd, lines).unwrap();
|
||||
assert_eq!(shown(&s, "West"), Value::Number(20.0), "{cmd}");
|
||||
assert_eq!(val(&s, west), shown(&s, "West"), "{cmd}");
|
||||
assert_eq!(val(&s, north), shown(&s, "North"), "{cmd}");
|
||||
}
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(pt(&s).last_range.unwrap().start.a1(), "A3");
|
||||
s.execute("cell.set", json!({"cell": "K3", "input": r#"=GETPIVOTDATA("Sales",A3,"Region","West")"#})).unwrap();
|
||||
assert_eq!(val(&s, "K3"), Value::Number(20.0), "undo restores the report and its offsets");
|
||||
// Deleting North's row moves West up; North itself is gone from the report.
|
||||
let north = find_row(&s, "North").unwrap() + 1;
|
||||
s.execute("home.deleteRows", json!({"rows": format!("{north}:{north}")})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": r#"=GETPIVOTDATA("Sales",A3,"Region","West")"#})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "K2", "input": r#"=GETPIVOTDATA("Sales",A3,"Region","North")"#})).unwrap();
|
||||
assert_eq!(val(&s, "K1"), Value::Number(20.0));
|
||||
assert_eq!(val(&s, "K1"), shown(&s, "West"));
|
||||
assert_eq!(val(&s, "K2"), Value::Error(CellError::Ref), "a deleted value cell is not read from its old place");
|
||||
// Lines inserted at the report's first line move the whole report, and lines right after it
|
||||
// leave it alone; either way the offsets stay what they were.
|
||||
for rows in ["3:3", "9:9"] {
|
||||
s.execute("home.insertRows", json!({"rows": rows})).unwrap();
|
||||
let at = pt(&s).last_range.unwrap().start.a1();
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": format!(r#"=GETPIVOTDATA("Sales",{at},"Region","West")"#)})).unwrap();
|
||||
assert_eq!(val(&s, "K1"), Value::Number(20.0), "insert {rows}");
|
||||
assert_eq!(val(&s, "K1"), shown(&s, "West"), "insert {rows}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Numeric items whose formatted labels collide are told apart by value: an exact numeric item
|
||||
/// wins over another item that only displays the same label.
|
||||
#[test]
|
||||
fn getpivotdata_matches_numeric_items_by_value_before_label() {
|
||||
let mut s = two_column_pivot(["Key", "Sales"], json!([[0.6, 10], [1, 20]]), Some("0"));
|
||||
let p = pt(&s);
|
||||
let labels: Vec<&str> = p.data_index.cells().unwrap().iter().flat_map(|c| c.items.iter().map(|i| i.label.as_str())).collect();
|
||||
assert_eq!(labels, ["1", "1"], "both items display as 1");
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": r#"=GETPIVOTDATA("Sales",A3,"Key",1)"#})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "K2", "input": r#"=GETPIVOTDATA("Sales",A3,"Key",0.6)"#})).unwrap();
|
||||
assert_eq!(val(&s, "K1"), Value::Number(20.0));
|
||||
assert_eq!(val(&s, "K2"), Value::Number(10.0));
|
||||
// Dates: two times on one day show the same date label; each serial finds its own item.
|
||||
let mut s = two_column_pivot(["Day", "Sales"], json!([[45000, 10], [45000.25, 20]]), Some("m/d/yyyy"));
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": r#"=GETPIVOTDATA("Sales",A3,"Day",45000.25)"#})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "K2", "input": r#"=GETPIVOTDATA("Sales",A3,"Day",45000)"#})).unwrap();
|
||||
assert_eq!(val(&s, "K1"), Value::Number(20.0));
|
||||
assert_eq!(val(&s, "K2"), Value::Number(10.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replace_protection() {
|
||||
let mut s = sales();
|
||||
@@ -1062,6 +1368,34 @@ fn detail_cells_are_the_cells_show_details_takes() {
|
||||
assert!(crate::pivot::detail_cells(&d.wb, pt, d.wb.active_sheet()).unwrap_err().contains("out of date"));
|
||||
}
|
||||
|
||||
/// Measurement, not a check: `perf stat -e instructions:u` of this test with `PIVOT_BENCH_EDITS=0`
|
||||
/// and `=20` (and `PIVOT_BENCH_REOPEN=0/1`) gives the cost of edits on the report's sheet and of
|
||||
/// opening a saved workbook, with the setup subtracted. `PIVOT_BENCH_ROWS` is the source size.
|
||||
#[test]
|
||||
#[ignore = "measurement only"]
|
||||
fn bench_edit_on_sheet_with_large_report() {
|
||||
let env = |k: &str, d: usize| std::env::var(k).ok().and_then(|v| v.parse().ok()).unwrap_or(d);
|
||||
let (n, edits) = (env("PIVOT_BENCH_ROWS", 100_000), env("PIVOT_BENCH_EDITS", 0));
|
||||
let mut s = Session::new();
|
||||
s.new_workbook();
|
||||
let mut rows = vec![json!(["Id", "Product", "Sales"])];
|
||||
rows.extend((0..n).map(|i| {
|
||||
let (id, product) = (format!("id{i:07}"), ["Apples", "Pears", "Plums"][i % 3]);
|
||||
json!([id, product, i])
|
||||
}));
|
||||
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
|
||||
s.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
s.execute("insert.pivotTable", json!({"rows": ["Id"], "columns": ["Product"], "values": ["Sales"]})).unwrap();
|
||||
for i in 0..edits {
|
||||
s.execute("cell.set", json!({"cell": "K1", "input": i.to_string()})).unwrap();
|
||||
}
|
||||
if env("PIVOT_BENCH_REOPEN", 0) == 1 {
|
||||
let wb = (*s.doc().unwrap().wb).clone();
|
||||
let (back, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(&wb).unwrap()).unwrap();
|
||||
s.add_document(crate::DocState::new(back, None, "Reopened".into(), crate::Trust::FromFile));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rank_ties_are_dense_and_count_numbers_is_captioned_count_of() {
|
||||
// #1448 items 10 and 12: Excel ranks 5, 4, 3, 3, 4 as 1, 2, 3, 3, 2 (GridCraft gave 1, 2, 4,
|
||||
|
||||
@@ -4,7 +4,7 @@ use std::sync::Arc;
|
||||
|
||||
use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS, RangeRef};
|
||||
use gridcraft_formula::adjust::{Axis, Edit};
|
||||
use gridcraft_model::{Anchor, AnchorMode, Sheet, Visibility};
|
||||
use gridcraft_model::{Anchor, AnchorMode, PivotDataCell, PivotLookup, Sheet, Visibility};
|
||||
use serde_json::{Value as Json, json};
|
||||
|
||||
use super::{
|
||||
@@ -211,7 +211,31 @@ fn shift_sheet_features(sh: &mut Sheet, axis: Axis, at: u32, count: u32, insert:
|
||||
Axis::Rows => CellRef::new(at, pt.anchor.col),
|
||||
Axis::Cols => CellRef::new(pt.anchor.row, at),
|
||||
});
|
||||
pt.last_range = pt.last_range.and_then(map_range);
|
||||
let old = pt.last_range;
|
||||
pt.last_range = old.and_then(map_range);
|
||||
// GETPIVOTDATA's value cells are stored as offsets from the report's top-left cell, so
|
||||
// lines inserted or deleted inside the report move them as well. The moved cells go into
|
||||
// a new index (the shared one stays with the undo snapshots); a value cell deleted with
|
||||
// its line drops out, so it is #REF! rather than read from whatever took its place.
|
||||
let index = pt.data_index.clone();
|
||||
if let (Some(o), Some(n), Some(cells)) = (old, pt.last_range, index.cells()) {
|
||||
let (lo, hi) = if axis == Axis::Rows { (o.start.row, o.end.row) } else { (o.start.col, o.end.col) };
|
||||
let inside = at <= hi && if insert { at > lo } else { at.saturating_add(count) > lo };
|
||||
if inside {
|
||||
let moved: Vec<PivotDataCell> = cells
|
||||
.iter()
|
||||
.filter_map(|c| {
|
||||
let to = map_cell(CellRef::new(o.start.row.checked_add(c.row)?, o.start.col.checked_add(c.col)?))?;
|
||||
Some(PivotDataCell { row: to.row.checked_sub(n.start.row)?, col: to.col.checked_sub(n.start.col)?, ..c.clone() })
|
||||
})
|
||||
.collect();
|
||||
let lookup = PivotLookup::default();
|
||||
if !moved.is_empty() {
|
||||
lookup.set_cells(moved);
|
||||
}
|
||||
pt.data_index = lookup;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -323,6 +323,7 @@ impl DocState {
|
||||
/// read from bytes (a file, a download, a recovery copy), [`Trust::Local`] only for a workbook
|
||||
/// made in this session.
|
||||
pub fn new(mut wb: Workbook, path: Option<String>, title: String, trust: Trust) -> DocState {
|
||||
let _ = gridcraft_model::PIVOT_INDEXER.set(pivot::index_cells);
|
||||
let mut calc = Calc::new();
|
||||
calc.recalc_all(&mut wb);
|
||||
Self::with_calc(wb, calc, path, title, trust)
|
||||
|
||||
@@ -12,8 +12,8 @@ use gridcraft_core::date::datetime_from_serial;
|
||||
pub use gridcraft_core::quote_sheet;
|
||||
use gridcraft_core::{CellError, CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value, split_sheet};
|
||||
use gridcraft_model::{
|
||||
BorderLine, BorderStyle, Cell, Chart, Color, Fill, HAlign, NumFmt, PivotDateGroup, PivotField, PivotFilter, PivotFunc, PivotLayout, PivotShowAs,
|
||||
PivotSort, PivotTable, PivotValue, Series, Style, StyleId, Workbook,
|
||||
BorderLine, BorderStyle, Cell, Chart, Color, Fill, HAlign, NumFmt, PivotDataCell, PivotDateGroup, PivotField, PivotFilter, PivotFunc,
|
||||
PivotItemRef, PivotLayout, PivotLookup, PivotShowAs, PivotSort, PivotTable, PivotValue, Series, Style, StyleId, Workbook,
|
||||
};
|
||||
|
||||
use crate::pivot_calc::CalcSet;
|
||||
@@ -41,6 +41,9 @@ pub const MAX_OUTPUT_CELLS: u64 = 2_000_000;
|
||||
pub const MAX_AXIS_FIELDS: usize = 32;
|
||||
/// Most fields in the Values area.
|
||||
pub const MAX_VALUE_FIELDS: usize = 256;
|
||||
/// Most report value cells a report may have (every one is indexed for GETPIVOTDATA; bigger
|
||||
/// reports are rejected).
|
||||
pub const MAX_INDEXED_CELLS: usize = 1_000_000;
|
||||
|
||||
const MONTHS: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"];
|
||||
|
||||
@@ -314,9 +317,42 @@ pub fn refresh(wb: &mut Workbook, sheet: usize, idx: usize, replace: bool) -> Re
|
||||
}
|
||||
let sheet_name = shm.name.clone();
|
||||
rebind_charts(wb, &sheet_name, &pt.name, origin, &out);
|
||||
// The index of the new report; its lookup hash is built on the first GETPIVOTDATA.
|
||||
if let Some(p) = wb.sheet_mut(sheet).and_then(|s| s.pivots.get_mut(idx)) {
|
||||
let index = PivotLookup::default();
|
||||
index.set_cells(out.data);
|
||||
p.data_index = index;
|
||||
}
|
||||
Ok(area)
|
||||
}
|
||||
|
||||
/// The value cells of a PivotTable's report, for GETPIVOTDATA when a refresh did not set the
|
||||
/// index (a workbook just opened from XLSX). The positions must describe the report as it is
|
||||
/// stored on the sheet, not as the current source would render it: the source is recomputed only
|
||||
/// to learn the layout, and the result is accepted when every label and header cell of that
|
||||
/// layout equals the stored cell at the same place. A source changed since the last refresh gives
|
||||
/// a different layout, so there is no trustworthy index and the lookup fails until a refresh.
|
||||
pub fn index_cells(wb: &Workbook, sheet: usize, pt: &PivotTable) -> Result<Vec<PivotDataCell>, String> {
|
||||
const STALE: &str = "The stored report no longer matches the source. Refresh the PivotTable.";
|
||||
let report = pt.last_range.ok_or("The PivotTable has no report.")?;
|
||||
let out = compute(wb, pt, sheet)?;
|
||||
let sh = wb.sheet(sheet).ok_or("There's no such sheet.")?;
|
||||
let rows = u64::from(report.end.row) - u64::from(report.start.row) + 1;
|
||||
let cols = u64::from(report.end.col) - u64::from(report.start.col) + 1;
|
||||
if rows != u64::from(out.height().max(1)) || cols != u64::from(out.width.max(1)) {
|
||||
return Err(STALE.into());
|
||||
}
|
||||
let labels = [CellKind::Header, CellKind::RowLabel, CellKind::ColLabel, CellKind::Filter];
|
||||
for (r, row) in out.grid.iter().enumerate() {
|
||||
for (c, oc) in row.iter().enumerate() {
|
||||
if labels.contains(&oc.kind) && sh.value(CellRef::new(report.start.row + r as u32, report.start.col + c as u32)) != oc.value {
|
||||
return Err(STALE.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(out.data)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod fit_tests {
|
||||
use super::data_cols_exceed_sheet;
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
use super::output::Output;
|
||||
use super::{CellRef, Chart, Series, Workbook, quote_sheet};
|
||||
use super::{CellRef, Chart, PivotLookup, Series, Workbook, quote_sheet};
|
||||
|
||||
// ---------------------------------------------------------------- PivotCharts
|
||||
|
||||
@@ -109,6 +109,7 @@ pub fn clear_output(wb: &mut Workbook, sheet: usize, idx: usize) {
|
||||
shm.take_cells(r);
|
||||
if let Some(p) = shm.pivots.get_mut(idx) {
|
||||
p.last_range = None;
|
||||
p.data_index = PivotLookup::default();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -126,6 +127,7 @@ mod tests {
|
||||
chart_rows: vec![1, 2, 3],
|
||||
chart_cols: (1..1000).map(|c| (c, format!("S{c}"))).collect(),
|
||||
label_col: Some(0),
|
||||
data: vec![],
|
||||
};
|
||||
assert_eq!(chart_series("Sheet1", CellRef::new(0, 0), &out).len(), MAX_CHART_SERIES);
|
||||
}
|
||||
|
||||
@@ -3,8 +3,8 @@ use super::output::{
|
||||
AggMap, CellKind, ColKind, ColSpec, Ctx, Grid, Line, LineKind, Output, RowKind, ValSrc, caption, data_cols_exceed_sheet, flatten_cols,
|
||||
};
|
||||
use super::{
|
||||
CalcSet, CellError, CellRef, HashMap, HashSet, MAX_AXIS_FIELDS, MAX_COLS, MAX_VALUE_FIELDS, PivotDateGroup, PivotField, PivotFilter, PivotFunc,
|
||||
PivotLayout, PivotShowAs, PivotTable, PivotValue, Source, Value, Workbook, read_source,
|
||||
CalcSet, CellError, CellRef, HashMap, HashSet, MAX_AXIS_FIELDS, MAX_COLS, MAX_INDEXED_CELLS, MAX_VALUE_FIELDS, PivotDataCell, PivotDateGroup,
|
||||
PivotField, PivotFilter, PivotFunc, PivotItemRef, PivotLayout, PivotShowAs, PivotTable, PivotValue, Source, Value, Workbook, read_source,
|
||||
};
|
||||
|
||||
/// Why Show Details turns a cell down when it isn't a value cell of a PivotTable (the front end
|
||||
@@ -653,6 +653,7 @@ pub(super) fn compute_from(wb: &Workbook, pt: &PivotTable, src: &Source, mut pro
|
||||
chart_rows: vec![],
|
||||
chart_cols: vec![],
|
||||
label_col: None,
|
||||
data: vec![],
|
||||
});
|
||||
}
|
||||
|
||||
@@ -728,6 +729,16 @@ pub(super) fn compute_from(wb: &Workbook, pt: &PivotTable, src: &Source, mut pro
|
||||
} else {
|
||||
Some(row_levels.saturating_sub(1) as u32)
|
||||
};
|
||||
let item_refs = |fields: &[&PivotField], items: &[FieldItems], path: &[u32]| -> Vec<PivotItemRef> {
|
||||
path.iter()
|
||||
.enumerate()
|
||||
.filter_map(|(d, it)| {
|
||||
let (f, item) = (fields.get(d)?, items.get(d)?.get(*it)?);
|
||||
Some(PivotItemRef { field: f.source_col.clone(), label: item.label.clone(), value: item.value.clone() })
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
let mut data: Vec<PivotDataCell> = Vec::new();
|
||||
for line in &lines {
|
||||
let rk = match line.kind {
|
||||
LineKind::Item(b) => RowKind::Item(b),
|
||||
@@ -758,6 +769,21 @@ pub(super) fn compute_from(wb: &Workbook, pt: &PivotTable, src: &Source, mut pro
|
||||
let Some(raw) = lookup.get(rp, cp, *vi) else { continue };
|
||||
let context = ShowContext { row_path: rp, col_path: cp, value_index: *vi, level0: &level0, siblings: &siblings, cache: &show_cache };
|
||||
let shown = show_value(vdef, raw, &lookup, &context);
|
||||
// Every value cell of the report is indexed, so a report that needs more cells than
|
||||
// the index cap is rejected outright — never written with a truncated index that
|
||||
// would answer #REF! for the values the sheet shows.
|
||||
if data.len() >= MAX_INDEXED_CELLS {
|
||||
return Err(format!("A PivotTable report can have at most {MAX_INDEXED_CELLS} value cells."));
|
||||
}
|
||||
let mut items = item_refs(&ctx.rows, &ctx.row_items, rp);
|
||||
items.extend(item_refs(&col_fields, &col_items, cp));
|
||||
data.push(PivotDataCell {
|
||||
data_field: vdef.name.clone(),
|
||||
source_col: vdef.source_col.clone(),
|
||||
items,
|
||||
row: u32::try_from(r).unwrap_or(u32::MAX),
|
||||
col: u32::try_from(l + j).unwrap_or(u32::MAX),
|
||||
});
|
||||
if let Some(pr) = probe.as_deref_mut() {
|
||||
if pr.at == Some((r, l + j)) {
|
||||
pr.hit = Some((rp.to_vec(), cp.to_vec()));
|
||||
@@ -769,5 +795,5 @@ pub(super) fn compute_from(wb: &Workbook, pt: &PivotTable, src: &Source, mut pro
|
||||
g.set(r, l + j, shown, ck.1, 0, data_fmt.get(*vi).cloned().flatten());
|
||||
}
|
||||
}
|
||||
Ok(Output { width: g.width as u32, grid: g.rows, row_kinds: g.kinds, body_offset, chart_rows, chart_cols, label_col })
|
||||
Ok(Output { width: g.width as u32, grid: g.rows, row_kinds: g.kinds, body_offset, chart_rows, chart_cols, label_col, data })
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
use super::items::{Acc, FieldItems, Node, is_collapsed};
|
||||
use super::{
|
||||
BorderLine, BorderStyle, Color, Fill, HAlign, HashMap, MAX_COLS, MAX_OUTPUT_CELLS, MAX_ROWS, NumFmt, PivotField, PivotLayout, PivotTable, Style,
|
||||
Value,
|
||||
BorderLine, BorderStyle, Color, Fill, HAlign, HashMap, MAX_COLS, MAX_OUTPUT_CELLS, MAX_ROWS, NumFmt, PivotDataCell, PivotField, PivotLayout,
|
||||
PivotTable, Style, Value,
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------- output
|
||||
@@ -61,6 +61,8 @@ pub struct Output {
|
||||
pub chart_cols: Vec<(u32, String)>,
|
||||
/// Grid column holding the innermost row labels (`None` without row fields).
|
||||
pub label_col: Option<u32>,
|
||||
/// Every value cell with the items that locate it (for GETPIVOTDATA).
|
||||
pub data: Vec<PivotDataCell>,
|
||||
}
|
||||
|
||||
impl Output {
|
||||
|
||||
@@ -9,7 +9,10 @@ use crate::style::{Color, Style};
|
||||
mod pivot;
|
||||
mod print;
|
||||
|
||||
pub use pivot::{PivotCalcField, PivotDateGroup, PivotField, PivotFilter, PivotFunc, PivotLayout, PivotShowAs, PivotSort, PivotTable, PivotValue};
|
||||
pub use pivot::{
|
||||
PIVOT_INDEXER, PivotCalcField, PivotDataCell, PivotDateGroup, PivotField, PivotFilter, PivotFunc, PivotIndexer, PivotItemRef, PivotLayout,
|
||||
PivotLookup, PivotShowAs, PivotSort, PivotTable, PivotValue, pivot_key,
|
||||
};
|
||||
pub use print::{EditRange, HfPage, LineInfo, Orientation, PageOrder, PrintSettings, SheetProtection, password_hash};
|
||||
|
||||
// ---------------------------------------------------------------- tables
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
use gridcraft_core::Value;
|
||||
|
||||
use super::{CellRef, RangeRef};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
@@ -131,6 +133,11 @@ pub struct PivotTable {
|
||||
pub last_range: Option<RangeRef>,
|
||||
/// Calculated fields, usable in the Values area by name.
|
||||
pub calculated_fields: Vec<PivotCalcField>,
|
||||
/// Where GETPIVOTDATA finds the report's value cells, as of the last refresh. Shared between
|
||||
/// workbook clones (so editing the pivot's sheet doesn't copy it), set on refresh and saved
|
||||
/// with the report, so a reopened unrefreshed pivot still reads the report it stored. Only a
|
||||
/// pivot loaded without one (XLSX) builds it on first use, checked against the stored report.
|
||||
pub data_index: PivotLookup,
|
||||
/// The Σ Values pseudo-field (it exists with two or more value fields) sits in Rows rather
|
||||
/// than Columns (Excel's "values on rows").
|
||||
pub values_on_rows: bool,
|
||||
@@ -149,6 +156,143 @@ pub struct PivotCalcField {
|
||||
pub formula: String,
|
||||
}
|
||||
|
||||
/// One value cell of a computed PivotTable report: the value field, the row and column items
|
||||
/// that locate it (fewer for subtotals, none for the grand total) and where the report shows it.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct PivotDataCell {
|
||||
/// The value field's caption ("Sum of Sales").
|
||||
pub data_field: String,
|
||||
/// The value field's source column ("Sales").
|
||||
pub source_col: String,
|
||||
pub items: Vec<PivotItemRef>,
|
||||
/// Row and column of the cell, counted from the report's top-left cell.
|
||||
pub row: u32,
|
||||
pub col: u32,
|
||||
}
|
||||
|
||||
/// Builds a PivotTable's value cells from its source (`sheet` is the pivot's sheet). The engine
|
||||
/// sets it; the model can't compute reports.
|
||||
pub type PivotIndexer = fn(&crate::Workbook, usize, &PivotTable) -> Result<Vec<PivotDataCell>, String>;
|
||||
pub static PIVOT_INDEXER: std::sync::OnceLock<PivotIndexer> = std::sync::OnceLock::new();
|
||||
|
||||
fn norm(s: &str) -> String {
|
||||
s.trim().to_lowercase()
|
||||
}
|
||||
|
||||
/// One part of a lookup key: length-prefixed, so no label text can imitate a separator.
|
||||
fn key_part(out: &mut String, s: &str) {
|
||||
out.push_str(&s.len().to_string());
|
||||
out.push(':');
|
||||
out.push_str(s);
|
||||
}
|
||||
|
||||
/// The lookup key of a value field with field/item labels (names ignore case and outer blanks).
|
||||
/// Each item is `(field, label, is_bool)`: a Boolean item and a text item with the same label
|
||||
/// are different items.
|
||||
pub fn pivot_key(data_field: &str, items: &[(&str, &str, bool)]) -> String {
|
||||
let mut parts: Vec<String> = items
|
||||
.iter()
|
||||
.map(|(f, l, b)| {
|
||||
let mut p = String::new();
|
||||
key_part(&mut p, &norm(f));
|
||||
key_part(&mut p, &norm(l));
|
||||
p.push(if *b { 'b' } else { 't' });
|
||||
p
|
||||
})
|
||||
.collect();
|
||||
parts.sort_unstable();
|
||||
let mut key = String::new();
|
||||
key_part(&mut key, &norm(data_field));
|
||||
key.push_str(&parts.len().to_string());
|
||||
key.push('|');
|
||||
for p in parts {
|
||||
key.push_str(&p);
|
||||
}
|
||||
key
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct LookupInner {
|
||||
cells: std::sync::OnceLock<Vec<PivotDataCell>>,
|
||||
keys: std::sync::OnceLock<std::collections::HashMap<String, usize>>,
|
||||
}
|
||||
|
||||
/// A PivotTable's GETPIVOTDATA index. Clones share it, so `Workbook` clones per command stay
|
||||
/// cheap however large the report is. The cells are set on refresh or on first use; the hash of
|
||||
/// them is built on the first lookup.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct PivotLookup(std::sync::Arc<LookupInner>);
|
||||
|
||||
impl PivotLookup {
|
||||
/// The value cells, if they were set.
|
||||
pub fn cells(&self) -> Option<&[PivotDataCell]> {
|
||||
self.0.cells.get().map(Vec::as_slice)
|
||||
}
|
||||
|
||||
/// Sets the value cells (the first set wins) and returns them.
|
||||
pub fn set_cells(&self, cells: Vec<PivotDataCell>) -> &[PivotDataCell] {
|
||||
self.0.cells.get_or_init(|| cells)
|
||||
}
|
||||
|
||||
/// The cell with this value field and exactly these items, by hash.
|
||||
pub fn find(&self, data_field: &str, items: &[(&str, &str, bool)]) -> Option<&PivotDataCell> {
|
||||
let cells = self.0.cells.get()?;
|
||||
let keys = self.0.keys.get_or_init(|| {
|
||||
let mut m = std::collections::HashMap::with_capacity(cells.len());
|
||||
for (i, c) in cells.iter().enumerate() {
|
||||
let items: Vec<(&str, &str, bool)> =
|
||||
c.items.iter().map(|it| (it.field.as_str(), it.label.as_str(), matches!(it.value, Value::Bool(_)))).collect();
|
||||
m.entry(pivot_key(&c.data_field, &items)).or_insert(i);
|
||||
m.entry(pivot_key(&c.source_col, &items)).or_insert(i);
|
||||
}
|
||||
m
|
||||
});
|
||||
keys.get(&pivot_key(data_field, items)).and_then(|i| cells.get(*i))
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for PivotLookup {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "PivotLookup({} cells)", self.cells().map_or(0, <[PivotDataCell]>::len))
|
||||
}
|
||||
}
|
||||
|
||||
/// Saved as the list of value cells (or nothing before the first refresh); the hash is rebuilt.
|
||||
/// An empty list is never saved: it is no index, and restoring one would lock the shared lookup
|
||||
/// against the lazy rebuild that a reopened workbook needs.
|
||||
impl Serialize for PivotLookup {
|
||||
fn serialize<S: serde::Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
|
||||
self.cells().filter(|cells| !cells.is_empty()).serialize(s)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for PivotLookup {
|
||||
fn deserialize<D: serde::Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
|
||||
let l = PivotLookup::default();
|
||||
if let Some(cells) = Option::<Vec<PivotDataCell>>::deserialize(d)?
|
||||
&& !cells.is_empty()
|
||||
{
|
||||
l.set_cells(cells);
|
||||
}
|
||||
Ok(l)
|
||||
}
|
||||
}
|
||||
|
||||
/// The index is a cache, not part of a PivotTable's identity.
|
||||
impl PartialEq for PivotLookup {
|
||||
fn eq(&self, _: &Self) -> bool {
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
/// A row or column field item: the field's source column, the item's label and its value.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct PivotItemRef {
|
||||
pub field: String,
|
||||
pub label: String,
|
||||
pub value: Value,
|
||||
}
|
||||
|
||||
impl Default for PivotTable {
|
||||
fn default() -> Self {
|
||||
PivotTable {
|
||||
@@ -170,8 +314,65 @@ impl Default for PivotTable {
|
||||
show_headers: true,
|
||||
last_range: None,
|
||||
calculated_fields: vec![],
|
||||
data_index: PivotLookup::default(),
|
||||
values_on_rows: false,
|
||||
values_position: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod pivot_lookup_tests {
|
||||
use super::*;
|
||||
|
||||
fn cell(field: &str, label: &str, value: Value, row: u32) -> PivotDataCell {
|
||||
PivotDataCell {
|
||||
data_field: "Sum of Sales".into(),
|
||||
source_col: "Sales".into(),
|
||||
items: vec![PivotItemRef { field: field.into(), label: label.into(), value }],
|
||||
row,
|
||||
col: 1,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn labels_with_separator_characters_do_not_collide() {
|
||||
let l = PivotLookup::default();
|
||||
// Two different (field, label) pairs that a separator-joined key would make identical.
|
||||
l.set_cells(vec![cell("A\u{2}B", "C", Value::Text("C".into()), 1), cell("A", "B\u{2}C", Value::Text("B\u{2}C".into()), 2)]);
|
||||
assert_eq!(l.find("Sales", &[("A\u{2}B", "C", false)]).unwrap().row, 1);
|
||||
assert_eq!(l.find("Sales", &[("A", "B\u{2}C", false)]).unwrap().row, 2);
|
||||
// A label carrying the old pair separator cannot stand for two items.
|
||||
assert!(l.find("Sales", &[("A", "B", false), ("C", "D", false)]).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn boolean_and_text_items_with_one_label_are_different_items() {
|
||||
let l = PivotLookup::default();
|
||||
l.set_cells(vec![cell("Flag", "TRUE", Value::Text("TRUE".into()), 1), cell("Flag", "TRUE", Value::Bool(true), 2)]);
|
||||
assert_eq!(l.find("Sales", &[("Flag", "TRUE", true)]).unwrap().row, 2);
|
||||
assert_eq!(l.find("Sales", &[("Flag", "TRUE", false)]).unwrap().row, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn index_round_trips_through_serde() {
|
||||
let l = PivotLookup::default();
|
||||
l.set_cells(vec![cell("Region", "East", Value::Text("East".into()), 1)]);
|
||||
let back: PivotLookup = serde_json::from_str(&serde_json::to_string(&l).unwrap()).unwrap();
|
||||
assert_eq!(back.find("Sales", &[("Region", "East", false)]).unwrap().row, 1);
|
||||
let none: PivotLookup = serde_json::from_str(&serde_json::to_string(&PivotLookup::default()).unwrap()).unwrap();
|
||||
assert!(none.cells().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_lookup_is_saved_as_no_index() {
|
||||
// An empty cell list is no index, so it must not be saved as an empty one: restoring an
|
||||
// empty index counts as an initialized lookup, which blocks the lazy rebuild.
|
||||
let l = PivotLookup::default();
|
||||
l.set_cells(vec![]);
|
||||
let saved = serde_json::to_string(&l).unwrap();
|
||||
assert_eq!(saved, "null", "an empty lookup saves no index: {saved}");
|
||||
let back: PivotLookup = serde_json::from_str("[]").unwrap();
|
||||
assert!(back.cells().is_none(), "an empty saved list restores no index");
|
||||
}
|
||||
}
|
||||
@@ -116,6 +116,7 @@ fn workbook() -> Workbook {
|
||||
show_headers: false,
|
||||
last_range: Some(rr("A2:H12")),
|
||||
calculated_fields: vec![],
|
||||
data_index: Default::default(),
|
||||
values_on_rows: false,
|
||||
values_position: None,
|
||||
};
|
||||
@@ -176,6 +177,7 @@ fn workbook() -> Workbook {
|
||||
show_headers: true,
|
||||
last_range: Some(rr("K2:Z30")),
|
||||
calculated_fields: vec![],
|
||||
data_index: Default::default(),
|
||||
values_on_rows: false,
|
||||
values_position: None,
|
||||
};
|
||||
|
||||
+10
-13
@@ -12,13 +12,9 @@ Excel's worksheet functions, category by category, against GridCraft's function
|
||||
them), plus 14 text functions the page lists under combined entries (FIND/FINDB, LEFT/LEFTB,
|
||||
LEN/LENB, MID/MIDB, REPLACE/REPLACEB, RIGHT/RIGHTB, SEARCH/SEARCHB), BETA.INV, and COPILOT and
|
||||
PY (Microsoft 365 functions not on that page when fetched).
|
||||
- **Ours (505):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the
|
||||
- **Ours (503):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the
|
||||
- **Ours (502):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the
|
||||
- **Ours (502):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the
|
||||
- **Ours (505):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the
|
||||
- **Ours (514):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the
|
||||
evaluator's special forms in `crates/calc/src/eval.rs` (`SPECIAL_FUNCTIONS`: IF, IFS, CHOOSE,
|
||||
OFFSET, INDIRECT, INDEX, SUBTOTAL, AGGREGATE, TEXT, LET, LAMBDA, MAP, REDUCE, SCAN, BYROW, BYCOL,
|
||||
OFFSET, INDIRECT, INDEX, GETPIVOTDATA, SUBTOTAL, AGGREGATE, TEXT, LET, LAMBDA, MAP, REDUCE, SCAN, BYROW, BYCOL,
|
||||
MAKEARRAY…), excluding the test-only `ADD2`. Every one is an Excel name.
|
||||
- **Not on Microsoft's list:** USDOLLAR and ECMA.CEILING, legacy functions Excel still evaluates (like DOLLAR),
|
||||
are in our library as Compatibility functions for files that use them. They are not counted above.
|
||||
@@ -29,7 +25,7 @@ Excel's worksheet functions, category by category, against GridCraft's function
|
||||
|
||||
## Coverage
|
||||
|
||||
**Measured: 513 / 523 by name (98.1%); 504 / 523 working (96.4%).**
|
||||
**Measured: 514 / 523 by name (98.3%); 505 / 523 working (96.6%).**
|
||||
|
||||
| Category | Excel | Ours | Working | % working | Missing |
|
||||
|---|---|---|---|---|---|
|
||||
@@ -38,7 +34,7 @@ Excel's worksheet functions, category by category, against GridCraft's function
|
||||
| Financial | 55 | 55 | 55 | 100% | — |
|
||||
| Engineering | 54 | 54 | 54 | 100% | — |
|
||||
| Text | 50 | 48 | 48 | 96% | DETECTLANGUAGE, TRANSLATE |
|
||||
| Lookup and reference | 40 | 36 | 36 | 90% | GETPIVOTDATA, IMAGE, RTD, TRIMRANGE |
|
||||
| Lookup and reference | 40 | 38 | 38 | 95% | IMAGE, RTD |
|
||||
| Compatibility | 40 | 40 | 40 | 100% | — |
|
||||
| Date and time | 25 | 25 | 25 | 100% | — |
|
||||
| Information | 22 | 21 | 21 | 95% | STOCKHISTORY |
|
||||
@@ -48,10 +44,10 @@ Excel's worksheet functions, category by category, against GridCraft's function
|
||||
| Web | 3 | 3 | 1 | 33% | (WEBSERVICE, FILTERXML are stubs; ENCODEURL works) |
|
||||
| Add-in and Automation | 3 | 1 | 1 | 33% | CALL, REGISTER.ID |
|
||||
| AI and Python | 2 | 0 | 0 | 0% | COPILOT, PY |
|
||||
| **Total** | **523** | **505** | **496** | **94.8%** | 18 missing, 9 stubs |
|
||||
| **Total** | **523** | **514** | **505** | **96.6%** | 9 missing, 9 stubs |
|
||||
|
||||
By the Excel version that introduced them (working / total): before 2010 346 / 360, 2010
|
||||
56 / 56, 2013 49 / 51, 2016 1 / 5, 2019 6 / 6, 2021 11 / 11, 2024 22 / 23, Microsoft 365 5 / 11.
|
||||
By the Excel version that introduced them (working / total): before 2010 349 / 360, 2010
|
||||
56 / 56, 2013 49 / 51, 2016 5 / 5, 2019 6 / 6, 2021 11 / 11, 2024 22 / 23, Microsoft 365 7 / 11.
|
||||
Dynamic-array and LAMBDA-era functions (FILTER, SORT, UNIQUE, XLOOKUP, VSTACK, TAKE, GROUPBY,
|
||||
PIVOTBY, REGEXTEST…) are all present.
|
||||
|
||||
@@ -66,7 +62,7 @@ This improves series preparation; it does not change the model-fitting or season
|
||||
|---|---|---|---|
|
||||
| FORECAST.ETS family (4) | yes | 4–6 h | AAA exponential smoothing with seasonality detection; needed for Forecast Sheet too |
|
||||
| ODDFPRICE, ODDFYIELD, ODDLPRICE, ODDLYIELD | yes | 2–4 h | Odd first/last period bond maths |
|
||||
| GETPIVOTDATA | yes | 2–3 h | Needs the pivot engine's result cache; also the "Generate GetPivotData" option |
|
||||
| Generate GetPivotData | yes | 1–2 h | GETPIVOTDATA works; clicking a PivotTable value while typing a formula doesn't insert it yet |
|
||||
| IMAGE | yes | 3–5 h | Pictures in cells (see PR #105); fetching URLs needs a privacy decision |
|
||||
| CUBE* (7), WEBSERVICE, FILTERXML | partly | 4–8 h | FILTERXML can work offline; CUBE needs an OLAP/data-model source |
|
||||
| TRANSLATE, DETECTLANGUAGE, STOCKHISTORY, COPILOT, PY | no (cloud) | — | Microsoft cloud services; open equivalents only by owner decision |
|
||||
@@ -128,7 +124,7 @@ inputs, so large finite middle values do not become `#NUM!` merely because their
|
||||
Sorting, argument coercion and error propagation are unchanged; this is a numerical correction
|
||||
under the documented median rule, not an extreme-value comparison against desktop Excel.
|
||||
|
||||
Work to close (16–26 h in total): the in-scope missing functions (10–16 h with the table above),
|
||||
Work to close (15–25 h in total): the in-scope missing functions (10–16 h with the table above),
|
||||
an oracle corpus of Excel-computed results per function (owner runs Excel on generated
|
||||
workbooks; results checked in as data, not Excel files) and fixing what it finds (6–10 h).
|
||||
|
||||
@@ -183,6 +179,7 @@ Positive fractional inputs still truncate, and signed zero remains valid. Covere
|
||||
| 2026-10-14 | minor | Bounded references up to 1,048,576 cells (`D3:AZ400`) keep their full shape instead of stopping at the used range, so `SUMPRODUCT(D3:AZ400*B3:B400,(A3:A400=x)*(D1:AZ1=y))` broadcasts like Excel and `ROWS`, `COUNTBLANK` and `--(r="")` count the blanks; references that run to the edge of the sheet (`A:A`) still stop at the used range (#1010) |
|
||||
| 2026-10-13 | minor | Legacy formulas from files (no dynamic-array metadata) evaluate with implicit intersection: cells differing from Excel's cached values 68,964 → 49,866 across 4,224 test-corpus workbooks (none gets more) |
|
||||
| 2026-10-13 | minor | Clear reports changed stored cells sparsely, including those beyond the first million coordinates; dependent formulas and removed spills update, while clearing empty merges still unblocks spills (#1032) |
|
||||
| 2026-10-12 | minor | GETPIVOTDATA added: 514 / 523 by name, 505 working |
|
||||
| 2026-10-12 | minor | USDOLLAR added as a Compatibility function (not on Microsoft's list, so the counts don't change); MIRR with no positive flow is -1, COUNTIF's bare `"<="` / `">="` skip empty strings, INDIRECT with `a1` = FALSE gives `#REF!` for A1-style text (#738) |
|
||||
| 2026-10-11 | minor | TEXT / number formats fixed against Excel-cached results: fraction literals and zero fractions, `0` denominators zero padded, `?` pads the thousands separator, booleans use the text section (226 mismatches → 0 in 352 TEXT cells) |
|
||||
| 2026-10-11 | minor | Iterative calculation: circular references iterate up to Maximum Iterations / Maximum Change when enabled |
|
||||
|
||||
+4
-6
@@ -91,8 +91,8 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha-
|
||||
|
||||
### 8. PivotTables: no slicers, calculated items, PivotCharts; unverified in Excel (F)
|
||||
- **Evidence:** `parity-checklist.md` missing `insert.slicer`, `insert.timeline`,
|
||||
`insert.pivotChart`, `table.insertSlicer`; calculated items and GETPIVOTDATA missing (calculated fields
|
||||
and values on rows done). Show Details (double-click a value, subtotal or grand total cell) lists
|
||||
`insert.pivotChart`, `table.insertSlicer`; calculated items and Generate GetPivotData missing (calculated fields,
|
||||
values on rows and GETPIVOTDATA done). Show Details (double-click a value, subtotal or grand total cell) lists
|
||||
its source records on a new sheet. The "Enable show details" option (`pivot.options`
|
||||
`enableShowDetails`, saved as `enableDrill`) turns it off; it has no PivotTable Options dialog
|
||||
yet (command only).
|
||||
@@ -142,10 +142,7 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha-
|
||||
Margins are dragged on the rulers (#792). Remaining: pictures in headers/footers, Different First Page / Odd & Even pages. Desktop now submits directly through craft-print with printer choice, copies, duplex and grayscale; browser printing stays available. Print ranges and driver tray/options UI remain. **Estimate:** 5–10 h.
|
||||
|
||||
### 15. Remaining functions and an oracle suite (F)
|
||||
- 18 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 16–26 h.
|
||||
- 20 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 17–29 h.
|
||||
- 21 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 18–30 h.
|
||||
- 18 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 18–30 h.
|
||||
- 9 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 15–25 h.
|
||||
[function-parity.md](function-parity.md)
|
||||
|
||||
### 16. Collaboration and review (F)
|
||||
@@ -212,6 +209,7 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha-
|
||||
| 2026-10-15 | minor | §13: Solver prepares reports before committing the solution, model and report sheets together; report refusal preserves the workbook and Undo/Redo (#1050) |
|
||||
| 2026-10-13 | trivial | §4: multi-level category axes landed |
|
||||
| 2026-10-12 | minor | §9: next/previous sheet commands retain the current visible sheet and selection when there is no visible destination |
|
||||
| 2026-10-12 | minor | §8, §15: GETPIVOTDATA added (9 functions missing) |
|
||||
| 2026-10-12 | minor | rows auto-fit oversized fonts on open and whole-row/column formatting (#833) |
|
||||
| 2026-10-12 | minor | Page Layout view: dragging margins on the rulers (#792) removed from the §14 remainder |
|
||||
| 2026-10-12 | minor | §13: Forecast Sheet Forecast Start (hindcast) landed (#785) |
|
||||
|
||||
@@ -14,8 +14,8 @@ generated ribbon checklist is [`parity-checklist.md`](parity-checklist.md).
|
||||
|---|---|---|
|
||||
| Feature breadth (weighted by use) | **~75%** | estimated from the area table below; anchored on two measurements |
|
||||
| Ribbon and menu catalog | **281 / 296 (94.9%)** | measured: `cargo xtask parity` (`crates/engine/src/catalog.rs` ∩ registered command ids) |
|
||||
| Worksheet functions, by name | **513 / 523 (98.1%)** | measured: registered functions vs Microsoft's published function list (method below) |
|
||||
| Worksheet functions that do real work | **504 / 523 (96.4%)** | measured: excludes 9 stubs (7 CUBE functions, WEBSERVICE, FILTERXML) |
|
||||
| Worksheet functions, by name | **514 / 523 (98.3%)** | measured: registered functions vs Microsoft's published function list (method below) |
|
||||
| Worksheet functions that do real work | **505 / 523 (96.6%)** | measured: excludes 9 stubs (7 CUBE functions, WEBSERVICE, FILTERXML) |
|
||||
| **Ready for real work** | **~50%** | estimated; written weights below (51.6 computed) |
|
||||
| **Mainstream practitioner** | **~50%** | estimated; [method](#mainstream-practitioner) (69.5 depth × 0.90 × 0.92 × 0.88 = 50.6) |
|
||||
| **Essentials user** | **~63%** | estimated; [method](#essentials-user) (80.2 depth × 0.92 × 0.95 × 0.90 = 63.1) |
|
||||
@@ -81,11 +81,11 @@ performance 8, stability 8, localization 4, hardware 2, platforms 2, ecosystem 2
|
||||
| Area | Weight | Breadth | Ready | Hours | Evidence |
|
||||
|---|---|---|---|---|---|
|
||||
| Formula language and calculation | 18 | ~92% | ~72% | 20–30 | A1/R1C1 parser, reference adjustment, dependency-graph recalc, dynamic arrays and `#SPILL!`, `A1#`, LET/LAMBDA and helpers, structured references, names, 3-D references. Open correctness bugs: 15-digit equality (#57), stale spills after structural edits and cleared blockers (#133, #139), 3-D span endpoint deletion (#140), sheet rename and qualified names (#134). Single-threaded |
|
||||
| Worksheet functions | 12 | 98.1% (measured) | ~82% | 16–26 | 513 / 523 names, 504 working. Missing: GETPIVOTDATA, IMAGE plus cloud functions. No conformance suite against Excel-computed oracles; edge-case bugs keep surfacing (#137, #138). [function-parity.md](function-parity.md) |
|
||||
| Worksheet functions | 12 | 98.3% (measured) | ~82% | 15–25 | 514 / 523 names, 505 working. Missing: IMAGE plus cloud functions. No conformance suite against Excel-computed oracles; edge-case bugs keep surfacing (#137, #138). [function-parity.md](function-parity.md) |
|
||||
| Editing, navigation, clipboard, fill | 12 | ~90% | ~70% | 15–25 | Point mode, F4, AutoComplete, Paste Special, AutoFill, Flash Fill, Find/Replace, Go To Special. Bugs: rectangular drag-select (#28), monthly fill (#132), cut across sheets (#128), Insert Copied Cells missing (#21), key tips only on part of the Home tab (#43; #33, #49). [ui-parity.md](ui-parity.md) |
|
||||
| Cell formatting, styles, conditional formatting | 10 | ~95% | ~80% | 8–14 | Full number-format language, borders, 47 cell styles, 60 table styles, all CF rule kinds including data bars, colour scales and icon sets. Gaps: font weights beyond regular/bold (#168), system fonts list (#4), themes |
|
||||
| Sort, filter, tables, validation, outline | 8 | ~82% | ~70% | 10–16 | Multi-level sort, AutoFilter, Advanced Filter, tables with totals, validation, outline, subtotals, consolidate. Advanced Filter copy-to preserves cells outside its result and rejects worksheet overflow (#1031). Bugs: subtotal header (#131), validation edits (#129), table style panic caught (#130); no slicers on tables |
|
||||
| PivotTables | 7 | ~65% | ~45% | 25–40 | Insert/Recommended, field list, three layouts, values on rows or columns, date grouping, show-values-as, refresh, calculated fields, XLSX round-trip. Missing: slicers, timelines, calculated items, PivotCharts, value/label filters, number grouping, GETPIVOTDATA, data-model pivots. Round-trip never checked in Excel |
|
||||
| PivotTables | 7 | ~65% | ~45% | 25–40 | Insert/Recommended, field list, three layouts, values on rows or columns, date grouping, show-values-as, refresh, calculated fields, GETPIVOTDATA, XLSX round-trip. Missing: slicers, timelines, calculated items, PivotCharts, value/label filters, number grouping, Generate GetPivotData, data-model pivots. Round-trip never checked in Excel |
|
||||
| Charts and sparklines | 8 | ~50% | ~30% | 40–60 | 25 chart kinds, all 2-D. Doughnut charts draw every series as a concentric ring (first series innermost), with editable hole size and first slice angle. The chart model holds title, legend position, a data-labels flag, gridlines flag, axis titles and chart data tables (legend keys, borders, XLSX round-trip and print/PDF); no axis scaling, error bars, per-point formatting, chart sheets, 3-D, surface, map, Pareto. Kinds survive XLSX round trip (#158, #161), except stock charts: only 3–4 series are a stock chart, and any other count is written and reimported as a line chart. [chart-parity.md](chart-parity.md) |
|
||||
| Page layout, print, PDF | 5 | ~80% | ~55% | 12–20 | Page setup, print titles, breaks, page order, Page Break Preview, headers/footers, PDF export and printing. Paper sizes match Japanese Excel: B4/B5/B6 are the JIS sizes, ISO B4/B5/B6 stay reachable by name, and Japanese Postcard (はがき), Japanese Double Postcard (往復はがき), the chou/kaku envelopes and A6 are available. No true Page Layout view; the PDF writer uses only the 14 standard fonts without embedding (no CJK or other non-Latin text in PDF); web print only exports (#141, #143) |
|
||||
| What-if and analysis | 4 | ~50% | ~40% | 22–35 | Goal Seek, Scenario Manager, data tables, Analysis ToolPak (all 19 tools, no charts), Solver (Simplex LP, a local nonlinear method and Evolutionary; Answer, Sensitivity and Limits reports; Load/Save Model, Reset All and Solver Options; no Population report), Forecast Sheet. Missing: Analyze Data |
|
||||
@@ -351,6 +351,7 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land
|
||||
| 2026-10-18 | trivial | Stale chart notes: all chart kinds survive XLSX round-trip (PRs #158, #161) and trendlines landed |
|
||||
| 2026-10-16 | minor | Advanced Filter copy-to clears only its actual output and rejects overflow without changing cells (#1031) |
|
||||
| 2026-10-15 | minor | Row auto-fit: line breaks count only in wrapping cells (Excel: wrap off shows one line, AutoFit gives one line); unwrapped multi-line cells from files no longer grow rows on open (no score change) |
|
||||
| 2026-10-12 | minor | Worksheet functions 514 / 523 (98.3%; 505 working): GETPIVOTDATA |
|
||||
| 2026-10-21 | minor | Automation: typed macros in the engine and `macro.*` commands, no interface yet (no score change); catalog 274 / 290 → 275 / 291 (View › Macros › Record Macro) |
|
||||
| 2026-10-12 | minor | Essentials AutoFit evidence: rows auto-fit oversized fonts on open and row formatting (#833) |
|
||||
| 2026-10-12 | minor | Sheet tabs: reveal the selected tab on navigation and resize while preserving manual scrolling (#681); no score change |
|
||||
|
||||
Reference in new issue
Block a user