229f9b32d2 Affix engine: FLAG UTF-8, 128 flag bits, continuation classes, FORBIDDENWORD (storytold/wordcraft#100) (storytold/wordcraft#782)
* M15: Polish interface, and Polish spelling, grammar and hyphenation

Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
  pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
  or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
  default to pl-PL.

Proofing
- Each run is checked in its own language (w:lang, resolved through
  styles): squiggles, F7, review.issues, suggestions and hyphenation.
  Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
  read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
  ~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
  missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
  punctuation, sentence capitals, missing comma before że/który/ponieważ/
  aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
  pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
  no longer treats ł, ż, ś… as vowels.

review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.

Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Layout: rustfmt the test module order left unformatted by storytold/wordcraft#391

cargo fmt --check failed on main after storytold/wordcraft#391 added mod tests_grid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Review fixes for storytold/wordcraft#346: read Polish spelling from the installed Hunspell dictionary

- Polish spelling uses the Hunspell pl_PL dictionary installed on the
  system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
  WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
  without one, Polish text gets no spelling marks, like Word without the
  language's proofing tools. .aff/.dic are size-capped and decoded from
  UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
  re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
  depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
  example are removed; docs describe the installed dictionary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* storytold/wordcraft#100: affix — accept FLAG UTF-8 and DEFAULT flag modes

Per-character flag modes are what the reader already parses; only the
multi-character encodings (long, num) stay errors. VERO pt_BR declares
FLAG UTF-8.

* storytold/wordcraft#100: affix — widen flag masks to u128 (128 flag capacity)

VERO pt_BR needs 106 distinct flags; the u64 bitmask capped at 64.
Serialization is unaffected: WCAFF1 stores flag characters and
re-derives the masks on load.

* storytold/wordcraft#100: affix — support FORBIDDENWORD

VERO pt_BR marks spellings it must reject with a forbidden flag (ý).
Stems carrying it are listed but never correct and never inflect.

* storytold/wordcraft#100: affix — continuation classes in check

SFX A 0 able/B: the form derived by A carries flag B, licensing a
further suffix of class B (workable -> workables) while the bare stem
cannot take one. form_has_flag walks the chain depth-bounded
(MAX_CHAIN = 3); with no continuations declared it degenerates to the
stem lookup, so existing dictionaries are unaffected. VERO pt_BR uses
26 continuation rules.

* storytold/wordcraft#100: affix — continuation classes in forms

Enumerate chained suffix forms (workable -> workables) under the same
MAX_CHAIN bound the checker walks, so every form the checker accepts
is listed and vice versa.

* storytold/wordcraft#100: affix — round-trip continuations and FORBIDDENWORD through WCAFF1

The compact format keeps its WCAFF1 magic: continuation classes ride
on the rule's appended field (as in the .aff source) and the forbidden
flag becomes a FORBIDDENWORD line. Old files read as before.

* storytold/wordcraft#100: affix — strip a UTF-8 BOM from .aff/.dic input

VERO's files open with a BOM; glued to the count line it parsed as a
stem (one garbage entry per load) and to SET it hid the directive.

* storytold/wordcraft#782: affix — the module doc still rejected what the code now reads

Continuation classes and FORBIDDENWORD are supported; the header said
Rejected for both.

* Review fixes for storytold/wordcraft#782: two-suffix chain limit, module doc

- MAX_CHAIN = 2 counts suffixes in a row, matching Hunspell's two-suffix limit
  (the old bound of 3 allowed up to four); test pins that a third is rejected.
- Module doc: say probes allocate small scratch strings instead of claiming none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: witkowski01 <konradwitkowski01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-12 07:38:10 -04:00
S
Description
No description provided
109 MiB
0 Stars 1 Watchers 0 Forks
Languages
Rust 98.5%
Max 1.3%
Python 0.2%