mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-18 16:24:38 +00:00
sniff_delimiter skipped every record without a non-whitespace character other than the delimiter, so a record such as ";" or ";;;" was blank to the semicolon candidate. A semicolon export of a sheet whose only value is far down (B40001) starts with 40,000 ";" rows, which fill the 64 KiB sample, and the file was read as comma-separated with one column. Records that hold only a candidate's delimiters and whitespace are now kept apart, at most 20 per candidate. When no candidate separates the records with other text, the same rules pick among the candidates whose sample holds only such records. Every result that did not fall back to comma stays the same: a row of tabs alone is still blank to the other candidates and does not outweigh them, as #1209 set out.
37 lines
2.9 KiB
Markdown
37 lines
2.9 KiB
Markdown
# craft-csv
|
|
|
|
Delimited text (CSV, TSV and their relatives) and SYLK (`.slk`). No `unsafe`; the only
|
|
dependency is `thiserror`.
|
|
|
|
Fields are text. Reading a field as a number or a date, deciding how a value is written, and
|
|
mapping a SYLK sheet onto a document are the caller's work.
|
|
|
|
## Delimited text
|
|
|
|
| Function | What it does |
|
|
|---|---|
|
|
| `decode_text(bytes)` | UTF-8 and UTF-16 byte order marks; without one, valid UTF-8 is kept; other bytes are Windows-1256 when they read as Arabic, else Windows-1252 |
|
|
| `sniff_delimiter(text, quote)` | comma, semicolon or tab: the one with the most consistent field count over the first 20 records with other text in a 64 KiB sample; when none separates those records, the candidates whose records hold only their delimiters and whitespace (rows of empty cells) are compared the same way, leaving out any candidate that found a record with other text; when no candidate finds coherent complete records with a delimiter in each, a candidate that finds no complete record with other text counts the delimiters of the first such record up to the end of the sample, or up to a quoted field still open there (after its rows of empty cells, a record without its delimiter is left out), and a quote inside a field that does not start with one makes that record incoherent; quote boundaries must be coherent, and ties prefer tab, comma, then semicolon |
|
|
| `parse_records(text, delim, quote, max_fields, on_record)` | records of fields, one callback per record (return `false` to stop); CR, LF and CRLF line ends; quoted fields with doubled quotes and line breaks; a quote in the middle of a field is text |
|
|
| `records(text, delim, quote, max_records, max_fields)` | the same, collected |
|
|
| `push_field(out, field, delim)`, `field_len(field, delim)` | one field as written, quoted only when it holds the delimiter, a quote or a line break; its exact length, to enforce a size cap before writing |
|
|
|
|
`max_fields` bounds what one record keeps, so a row of endless delimiters cannot allocate a
|
|
string per field. The delimiter between fields and the line end are the writer's own.
|
|
|
|
## SYLK
|
|
|
|
`sylk::read(bytes)` gives a `sylk::Sheet`: cells by (row, column) with a value, the formula in A1
|
|
notation (the file's R1C1 text with its references rewritten, not checked for syntax) and the
|
|
number format code; column widths in characters; and the number of cell records skipped because
|
|
their position is outside the grid. `sylk::r1c1_formula_to_a1` is the rewrite on its own.
|
|
|
|
Files over 512 MB are refused, positions outside 1,048,576 rows by 16,384 columns are skipped,
|
|
and at most 1,024 column-width records are kept.
|
|
|
|
## Origin
|
|
|
|
Moved from GridCraft: `crates/xlsx/src/csv.rs` (decoding, sniffing, the record parser and the
|
|
quoting of its writer) and `crates/engine/src/io/sylk.rs`, at `5cf52c5f`. GridCraft keeps the
|
|
parts that depend on its model: typed values and dates from fields, and the sheet it builds.
|