Commit Graph
92 Commits
Author SHA1 Message Date
Martin BallandClaude Opus 5.5 22144c823a proof: licence warnings report the actual error; machine-independent English test; SuggestSteps drop note
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 00:47:39 +00:00
Martin BallandClaude Opus 5.5 3aca2642c5 craft-proof tr — the double-space rule uses space_runs; the all-languages test adds Tr, EnAu, EnCa and pt-PT (#496 review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:56:09 +00:00
Martin BallandClaude Opus 5.5 cb82e91428 craft-proof — a run of spaces is one double-space issue over the whole run in every language, through one shared space_runs helper (#496 review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:23:15 +00:00
Martin BallandClaude Opus 5.5 7b64b78cb0 craft-proof english — Australian English also takes a word in SCOWL's en-GB list, and without SCOWL every en_AU source is read before any en_GB one (#496 review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:23:09 +00:00
Martin BallandClaude Opus 5.5 65ce4f882a FU.7: craft-proof pack languages — another script is told by the letters' Unicode scripts, not the dictionary's letter inventory, so a misspelling in the dictionary's own script (zzzz in Dutch) is always checked
Found by a local CodeRabbit review of #496 (major).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 1f345ea809 FU.5: craft-proof id, ru — the cased-letter rule for words in capitals (craft-libs#485): a caseless part is no abbreviation; shared in_capitals / capitals_part
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 87defe169f FU.3: craft-proof pack languages — missing-dictionary list forgets its oldest, not all; a panic inside a check unpins its dictionaries; a word in another script than the dictionary's isn't marked
Follow-ups from craft-libs#485: `CHECK_FROM` is reset by a drop guard; `remember_missing` drops
the oldest of 256; a word none of whose letters the dictionary's words use (a Cyrillic word in
a Dutch run) is left alone, as Word does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 820695e951 FU.2: craft-proof — Australian English reads SCOWL's own en-AU dictionary (ProofLang::EnAu), with en-GB's spelling policy; en-NZ stays on en-GB
SCOWL en-AU has 238 words en-GB lacks (billabong, brumby, larrikinism, Australian place
names) and spells program, inquiry, livable, licorice, verandah the Australian way (no
programme, enquiry, liveable, dialog or veranda, as Hunspell en_AU says); -ise/-ize policy and
affix file are the same as en-GB's. craft-i18n already ships language-data/en-au.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 e36fb5d069 FU.1: craft-proof — British English never swaps -yze for -yse (reanalyze, breathalyzer, paralyzingly stay American, as Hunspell en_GB says)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 be52680c9c CP6.13: craft-libs#495 review — an oversized pack licence is skipped alone; a score (3 : 1) keeps its spaces in hr, it, pl, pt and ru; ' ,,' (a typed quote) is left to the quote rule in pl and ru; pt narration after a dialogue dash is no new sentence
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:27:37 +00:00
Martin BallandClaude Opus 5.5 8f16fe5a13 CP4.2: review — test dictionaries installed in memory hold the dictionary-tests lock until the test ends, so a pack or folder reload in another test can't drop them mid-test (the with-data ru suggestion flake)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 30f0f2a0e8 CP4.2: review — pack licences and licence file names are plain text; missing_dictionary documents that every language is named; past 256 pack names the hint names a generic dictionary
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 9d48d8145f CP4.2: review — the bare es and de packs are Spain's and Germany's; de-CH/de-LI and pt-PT read their own region's dictionary (sharing German's and Portuguese's hyphenation, grammar and messages); a region's text reads the system's exact region before the language's bare pack
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 f67dd7761e CP4.2: review — "Ignore words in UPPERCASE" skips only words with a cased letter, so a misspelt Hebrew or Arabic word is marked
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 ba11ccd855 CP4.2: review — English never takes another dialect's dictionary or pack: each English reads its own names in the dictionaries folder, an exactly named pack and the system, else the built-in Moby list; the British-spelling test runs unconditionally again
hunspell::candidates_exact (folder → exact-locale pack → system, own names only) for English without SCOWL; packs::exact_spelling_dirs. Other languages keep the region-ordered cross-region tiers. Drops has_british_source().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 3893192e88 CP4.2: review — en-GB British-spelling test skips a US-only Hunspell source (the CI goitre failure); German, Bulgarian and pack languages name their missing dictionary
The goitre failure was not a race: GitHub's Ubuntu runners have a system en_US Hunspell dictionary, which en-GB uses without the Language data (by design), and it marks goitre. Reproduced locally with a fake system en_US; the test now checks British words only when en-GB reads the Moby list or a British dictionary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 ee3be01b73 CP4.2: review — English tests hold the dictionaries lock; another region's pack never beats the system's exact region; bounded pack hyphenation lookups; NFC hyphenation; licences budgeted before reading; pack languages report like the others
P2:
- Every test whose English results depend on installed dictionaries takes DICTIONARY_TESTS (the en-GB goitre CI failure: tiny en_US fixtures raced unlocked en-GB tests).
- hunspell::candidates/find order: dictionaries folder, the first name's region pack (or the language alone), the system's exact-region dictionary, any other region's pack, the system's other names. pt_BR system beats a pt-pt pack; de_DE / de-ch, nl_NL / nl-be, es_ES / es-mx, en_US / en-gb likewise.

P3:
- Pack hyphenation lookups: a map, at most 256 languages (cleared when full); 3000 tags stay bounded and fast.
- A decomposed word (a + U+0301) is hyphenated composed (NFC) and the breaks mapped back; combining marks stay in their word.
- licence_texts checks each file's size against what is left before reading it; never reads past 16 MB.
- German, Bulgarian and pack languages: spelling message in the language where we have it; missing_dictionary/unreadable_dictionary report them like the others.
- build.rs: a missing file no longer says nothing is embedded; ru ignore message says CRAFT_I18N_DIR is enough. Built-in English list notice names the whole Moby list (en-words.dic).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 664af9bc6f CP4.2: review — on 48ed6f8 English without data looks in the dictionaries folder, then an English Language pack, then the system's Hunspell, then the built-in list
hunspell::candidates (which English now loads from) searches the pack tier between the dictionaries folder and the system, as find() does; pack-loaded English goes through the same reloads guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 66d4dd3a5d CP4.2: review — pack names drop bidi/format characters, licence texts are capped in all, code() documented for Pack
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 867f44a3d8 CP4.2: review — pack hyphenation patterns only take words in the language's script; others fall through as before
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 930365bb6d CP4.2: review — a tag in its language's default script reads the script-less pack (sr-Cyrl → sr, bs-Latn → bs); scripts must be letters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 db341c5e15 CP4.2: review — pack dictionaries share a memory budget and a paragraph never evicts its own, so many languages load once
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 2495fad496 CP4.2: review — German and Bulgarian are spelled from their pack; pack languages are checked in any script
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 3ad9ea765b CP4.2: ProofLang::Pack(PackLocale) proofs any language from its installed pack or Hunspell dictionary
Languages without tools of their own (nl, fr, sr-Latn…) are spell-checked and given suggestions
through the shared affix engine with default tuning, and hyphenated from their pack's patterns;
no grammar. At most 8 such dictionaries stay loaded. Never another script's pack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 576cf914e3 CP4.2/CP4.4: craft_proof::packs reads installed Language packs; only en-* embedded by default
- packs::set_packs_dir / reload / installed / licence_texts / load_errors / embedded_locales
- spelling lookup: dictionaries folder -> installed pack -> system Hunspell -> (English) core list
- hyphenation: installed pack patterns before the embedded ones, reloaded without a restart
- hostile packs: size caps, no symlinks, pack.toml path traversal rejected, errors reported
- build.rs embeds only en-* unless CRAFT_PROOF_EMBED names more (Language builds)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 de593dff4d CP6.11: craft-libs#489 review — dialect mapping after OCONV, bounded conversions
- An OCONV-written suggestion (colour’s) is mapped to the run's English
  spelling: the dialect lists are read with ', and the result keeps the
  dictionary's apostrophe (color’s). Straight and curly duplicates (color's,
  color’s) are one suggestion. Test: curly_suggestions_are_mapped_to_the_dialect.
- Conv::at looks up each prefix of the text (at most MAX_CONV_BYTES) in a
  hash map instead of scanning the pairs under its first character: 65,536
  pairs on one character cost 64 lookups per position, not a scan (Korean's
  OCONV has 588 pairs on one character, so a per-character cap would not
  fit). Test: a_huge_conversion_table_on_one_character_stays_fast.
- The reviewers' OCONV repros are a test (oconv_suggestions_match_hunspell):
  OCONV applies after a suggestion's case is set, case-sensitively, as the
  hunspell 1.7.5 CLI answers (KBT → KAT; Kbt → Kat under k → q).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:32 +00:00
Martin BallandClaude Opus 5.5 6e45fda67c CP2.2: review — British English takes an -ize spelling only when en-US or en-CA spells the word so
craft-libs#483 re-review (P2): ise_spelling turned any z between i/y and a
vowel into s, so en-GB (and the other British Englishes) accepted
misspellings whose -s- form is a word: wize, rize, noize, dizease, advize,
surprize, exercize… (19 reproduced; Hunspell en-GB rejects them all). The
-ize form now counts only when the word itself is in the en-US or en-CA
SCOWL dictionary and its -ise form in en-GB's (organize, realization,
colourize). English with no dialect no longer needs the swap (it reads all
three). Test: british_ize_needs_a_real_ize_word (the 19, plus colorize, wrong
in en-GB and en-AU; organize, realization, colourize, organizing, realizes
right).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:21:02 +00:00
Martin BallandClaude Opus 5.5 0675d58341 CP2.2: review — each English checks only its own SCOWL dictionary; English suggestions drop junk splits and names
craft-libs#483 review (P2): English spelling checked the union of the three
SCOWL dictionaries, so dialects crossed (en-US took colourblind, greyscale;
en-GB took colorblind, esophagi). Each English now checks its own dictionary
only. British English also takes the -ize spellings of its -ise words
(SCOWL's en-GB is -ise only; Word accepts both), and en-variants.txt widens
each dialect by its pairs; judgment/judgement come from craft-i18n#114, which
CI now pins. English with no dialect (is_correct) still takes a word any
English knows. Test: scowl_dialects_dont_cross samples the stems one
dialect has and another lacks, for every pair.

P3s:
- English suggestions (a Tuning of their own): a missing space only between
  two lower-case words of three letters or more, ranked after every one-word
  candidate (publically: publicly first; no "Thea tr", "Che Que"), and no
  word the dictionary only capitalizes (Col, Wis) for a lower-case word.
- A FORBIDDENWORD whole word is never rebuilt from compound parts.
- to_bytes writes ONLYINCOMPOUND without compound rules too (round-trip test).
- Failed lookups: the compound part table is filled lazily, and a word that
  doesn't start like a compound part (SCOWL: a digit) skips compounding;
  20,000 failed long words take 83 ms, as without compounding (1.09 s before).
- README: en-CA is not a British-spelling English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 21:44:18 +00:00
Martin BallandClaude Opus 5.5 e6e948f7d5 CP2.2: English spelling reads SCOWL 2026 through the affix engine
craft-i18n's SCOWL rel-2026.02.25 dictionaries (en-us, en-gb -ise, en-ca)
are embedded by build.rs and read per English: en-US, the British-spelling
Englishes (en-AU, en-NZ, en-IE...) through en-GB, en-CA. Suggestions come from
the run's own dictionary; a word is right when any of the three (or
en-extra.txt) knows it, unless en-variants.txt marks it as the other dialect's.
en-variants.txt stays: SCOWL's size-60 lists keep one spelling per dialect and
Word accepts more (organize and organise in en-GB, judgement in en-US).

English lookup order: dictionaries folder (the run's own English only),
[installed English pack, craft-libs#485], embedded SCOWL, system Hunspell
(own English only; not searched while SCOWL is embedded), the built-in Moby
list. The embedded Moby list is no
longer a spelling source (the built-in list is the same list); it stays for
hyphenation. The dialect policy applies to every source.

The affix engine gains COMPOUNDRULE (one-character flags, * and ?),
ONLYINCOMPOUND and COMPOUNDMIN, which SCOWL uses for ordinals (21st, 112th).
Hostile patterns are capped (64 rules, 32 tokens, 48-byte words) and checked
by a memoised walk. load_english now reads them instead of ignoring them (and
keeps ONLYINCOMPOUND stems as compound parts); only flag-based compounding and
OCONV are still ignored for English.

scowl::notice() exposes the SCOWL/Ispell/Titze notice for About. Embedded
Language data grows by 1,542,396 bytes. CI pins craft-i18n#109's merge commit efb1127.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 21:06:00 +00:00
Martin BallandClaude Opus 5.5 9e5862a1c1 CP1.2: each English uses only its own Hunspell dictionary, else the built-in Moby list
Re-review of craft-libs#482:
- en-GB and en-CA never fall back to an en_US Hunspell dictionary, which marked colour,
  organise, centre… (CI's runner has one); without their own they use the built-in Moby list,
  which has both spellings. en-US looks for en_US/en-US only.
- hunspell::candidates goes name by name across every folder, so a system en_GB beats an
  en_US in the dictionaries folder.
- Built-in suggestions rank swaps, then common words, neighbouring keys, the same first letter;
  no one-letter suggestions but a and i (hte -> the, recieve -> receive, wprd -> word).
- The generator leaves out Moby entries that are far more often typos: teh, abd, nad, ur, u, r
  (wont and cant stay: words). en-words.dic regenerated (343,988 bytes).
- A load that started before a reload no longer notes its load error; reload clears the
  English load errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:45:01 +00:00
Martin BallandClaude Opus 5.5 48ed6f8cbd CP1.2: build.rs keys the data tests on a whole data folder, handles non-UTF-8 paths; CI pins craft-i18n
Review of craft-libs#482:
- craft_proof_data is set only when every Language data file is embedded, and the new
  craft_proof_no_data only when none is, so a partial folder (en-us but no pl) runs neither
  the pattern tests nor the no-data tests.
- A CRAFT_I18N_DIR without language-data/ is watched (rerun-if-changed), so the build
  notices when the checkout gains it.
- A CRAFT_I18N_DIR that isn't valid UTF-8 is reported and embeds nothing, instead of a lossy
  path in include_str!.
- proof.yml checks out craft-i18n at 062a6375c322c88b96691a17f7b5fed71c51f7a8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 18:37:02 +00:00
Martin BallandClaude Opus 5.5 c80b6225f1 CP1.2: English without Language data uses the whole Moby list and every installed English dictionary
Review of craft-libs#482:
- The built-in fallback is now the full public-domain Moby word list (data/en-words.dic,
  words only, 343,992 bytes, from craft-i18n's en-us.dic by examples/en_words.rs) plus
  WordCraft's own en-extra.txt (MIT OR Apache-2.0). An unknown word is misspelled; the
  near-miss scan (which flagged fiber, theater, organize… and took 14 s on a 2,000-word
  token) is gone. Suggestions are the words one edit away, looked up, for words of up to 32
  letters. spelling_scales_over_one_long_token runs without the data, with a time bound.
- English Hunspell dictionaries ignore compounding (COMPOUNDMIN, COMPOUNDRULE…; their
  ONLYINCOMPOUND stems are left out) and OCONV instead of refusing the file, and a file that
  can't be read is skipped for the next name or folder.
- A load that started before english::reload() isn't kept (reloads counter, as slot.rs);
  Englishes that use the same .dic share one copy. en-GB/en-CA falling back to en_US is
  documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 18:37:02 +00:00
Martin BallandClaude Opus 5.5 547f303095 CP1.2: craft-proof carries wordcraft crates/proof up to storytold/wordcraft@9c72ddf56
Merges the crates/proof history from storytold/wordcraft a88e00525..9c72ddf56
(storytold/wordcraft#5000, storytold/wordcraft#4235: 21 commits and 5 merges),
split with the same git filter-repo run as the first import, so the shared
history has the same commit ids and every commit keeps its author.

Conflicts in src/lib.rs, resolved with the CP1.2 changes kept: the embedded
`data` module beside the new `affix_suggest`; a dictionaries-folder change
reloads every language slot and English; English load errors still come from
the English source; the dictionaries-folder test helpers stay as CP1.2 split
them, with DICTIONARY_TESTS documented for the user-dictionary tests too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 16:12:06 +00:00
aiongg 921569d0a7 Merge pull request storytold/wordcraft#5000 from benandez/integration/batch-proof
Integration batch proof: 2 approved proofing PRs (merge commit, please)
2026-11-01 10:46:45 -05:00
.CooCoo 24aac64a83 Tests: border lines, tab indents and wide spaces before a word (storytold/wordcraft#4235)
Tests and notes for rules that storytold/wordcraft#4129, storytold/wordcraft#4113, storytold/wordcraft#4109 and storytold/wordcraft#4101 added and
left uncovered:

- A border line at the top of a story with formatting left out of the
  Track Changes markup is not a formatting change, and Reject All keeps
  it.
- The author's own earlier formatting change on that paragraph stays the
  one change.
- Reject All keeps the table of a "+---+" line.
- A border line that a content control's lock stops Backspace from
  deleting stays as it is, and Enter is refused, untracked too.
- With tab indents on, Tab in a table cell moves to the next cell; with
  them off, Backspace before an indented paragraph joins it.
- AutoCorrect and the spelling check's web and e-mail addresses after a
  narrow no-break space and a figure space.

docs/typing-parity.md describes each, and lists a no-break space typed
after a word, which does not finish the word for AutoCorrect, as a
known gap.
2026-11-01 10:46:25 -05:00
Martin BallandClaude Opus 5.5 23151270ab CP1.2: English never goes unchecked — embedded data, then system Hunspell, then a built-in core word list
- No embedded Moby list: English (en-US, en-GB, en-CA) uses an English
  Hunspell .aff/.dic from the dictionaries folder or the system (the run's
  English first, then another English), else data/en-core.txt.
- en-core.txt: 11,367 public-domain Moby Hyphenator II words (Grady Ward),
  ranked by lower-case frequency in 50 public-domain Gutenberg books
  (examples/en_core.rs, book ids recorded). 94 KB. Marks only near misses
  of common words (swapped letters; one edit for lower-case words of 4+).
- Unreadable English dictionary (encoding, not UTF-8, no words, size):
  error via dictionary_load_error, falls through to the core list; a bad
  embedded list is reported the same way. Never a panic.
- Non-English text is never checked against English.
- New: english_source(lang) -> Option<EnglishSource>.
- Embedded-data path unchanged; tests with data unchanged.
- Tests: core list flags/accepts, folder Hunspell preferred over core list,
  unreadable dictionaries reported, no_data English now marks misspellings;
  two data-ignored tests now run without data. README, ATTRIBUTION, NOTICE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 10:55:57 +00:00
Martin BallandClaude Opus 5.5 b758ba6943 CP1.2: craft-proof embeds Language data from CRAFT_I18N_DIR; works without it
- Package renamed wordcraft-proof -> craft-proof; [workspace.dependencies]
  entry; public API unchanged.
- build.rs embeds craft-i18n language-data/ (en-us, de, pl, pt, it, ru)
  instead of include_str!/include_bytes! of WordCraft's assets/: warns when
  CRAFT_I18N_DIR is unset, errors under CRAFT_I18N_REQUIRED=1.
- No data: English spelling marks and suggests nothing, hyphenation that
  needs data is off, grammar and rule-based hyphenation work. no_data.rs runs
  every public entry point without data; data-pinning tests are ignored then
  (cfg craft_proof_data).
- es timing bound 75 -> 150 ms: craft-libs tests run at opt-level 0.
- proof.yml (fmt, clippy -D warnings, tests with and without data, release
  guard, wasm32), README, CODEOWNERS for crates/proof.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 10:02:10 +00:00
VoiceLessQ 61dbb4bf69 Merge remote-tracking branch 'upstream/main' into fix/proof-shared-language-engine
# Conflicts:
#	crates/proof/src/es/mod.rs
#	crates/proof/src/tr/mod.rs
2026-10-27 15:36:17 -02:00
VoiceLessQ 140c7041e1 Proofing: the spelling options are read in one place
Czech, Croatian, Indonesian, Italian, Polish, Portuguese and Spanish began
their spelling checks with the same thirteen lines, byte for byte: no
dictionary means no marks, single letters and words without letters pass,
words with digits and short words in capitals pass when the File › Options
› Proofing choices say so, and words in another script pass. Each also had
its own copy of the Latin-script test. These were the lines storytold/wordcraft#3966 had to fix
in six copies one by one, and where the Russian and Turkish copies stayed
wrong until the first commit here.

spelling.rs has them once (`word_to_check`), and each language goes on
from there with its own rules (Polish apostrophe endings, Italian elisions,
Indonesian clitics, Uruguayan `vos` forms).

Tests: words_needing_no_lookup. Ignoring the uppercase option there fails
it and all seven languages' proofing-options tests. Proof: 286 tests.
2026-10-24 19:58:30 -01:00
VoiceLessQ a388c4787c Proofing: one language table instead of nine lists
lib.rs named every dictionary language in nine places: reloading for a new
folder, the installed list, load errors, the missing-dictionary hint, the
loaded check, the spelling message, spelling, suggestions and grammar. The
lists had drifted. dictionary_load_error had no Russian or Czech entry, so a
Russian or Czech dictionary that failed to load showed no reason in File ›
Options › Proofing, though the status bar, which reads another list, said
why.

LANGUAGES holds one row per language: its dictionary slot, its spelling
message, and its spelling, suggestion and grammar functions. The nine
functions read the row, so a language now joins with its module, its
ProofLang and one row. The per-language reload_dictionary and
installed_path wrappers had no callers left, inside the crate or out, and
are gone.

Tests: every_dictionary_language_reports_its_load_error, which fails for
Russian on the old code, and the_language_table_covers_the_proofed_languages
(each ProofLang::PROOFED language but English has exactly one row). Proof
285, engine 927 and layout 501 (lib).
2026-10-24 19:55:43 -01:00
VoiceLessQ c19b1cae73 Proofing: one suggestion engine with per-language tuning
Czech's dictionary-driven ranking (case, MAP, REP, missing spaces, KEY,
TRY edits, a stem scan) had five older copies. Polish, Portuguese,
Croatian, Italian and Russian each kept one: 300 to 400 lines of the same
rounds with a few lines of language knowledge mixed in. Fixes reached some
copies only. NOSUGGEST (storytold/wordcraft#1048) and the first-difference tie-break (storytold/wordcraft#797)
never got to all of them, and none had the KEY round or the bound on TRY
letters.

affix_suggest.rs is that ranking once. A language's `Tuning` holds what is
its own: the short words that run into their neighbours (Polish,
Portuguese, Croatian, Italian), Russian's split rule, the letter groups and
confusions its dictionaries lack (Croatian, Russian), letters to try for a
dictionary without a TRY line, and cost hooks for Polish final devoicing
and Russian е/ё. Czech, Indonesian and Spanish use the dictionary's tables
alone. Turkish keeps its own suggester, which reads another dictionary type
with rounds of its own.

The language tables add to the dictionary's instead of replacing them, and
TRY letters are tried in the dictionary's order, commonest first, as the
copies did (the Czech engine sorted them).

Old and new compared on the test dictionaries, with every one-letter typo
of each word plus the tests' misspellings, top eight suggestions: Czech,
Croatian, Italian and Portuguese give the same for all 7,903 inputs.
Polish differs for 44 of 3,244 and Russian for 33 of 2,346, all orderings
among candidates of equal cost, from the tie-break they now share (`ie` →
`nie` before `i`).

New tests pin the moved knowledge: the tuning mechanism, Croatian's groups
without MAP lines, and Russian's costs and split rule. Turning each knob
off fails one. Proof: 283 tests.
2026-10-24 19:49:22 -01:00
VoiceLessQ 172c935fca Proof tests: the user-dictionary file test waits for the dictionary tests
user_dictionary_persists_through_a_file saves the whole user dictionary to
a file and loads it back. Run beside
user_dictionary_and_ignore_all_match_both_normalization_forms, it could
catch a word that test had added and bring it back after the test removed
it, so the other test's last check failed now and then (once in about two
dozen full runs here). It now holds DICTIONARY_TESTS, as the other test
does.
2026-10-24 19:48:47 -01:00
aiongg 1dd057f534 Merge PR storytold/wordcraft#4109: Proofing: web addresses after a no-break space are not marked as misspelled 2026-10-24 20:38:49 +00:00
VoiceLessQ 8f4616eece Proofing: one dictionary slot for every language
Nine languages each kept their own copy of the same code: the installed
dictionary behind a lock, read on first use and logged, replaced by
install_dictionary, forgotten by reload_dictionary, and its file reported
by installed_path. The copies had drifted already. Every one documented that
a dictionary given with install_dictionary has no file and isn't reported,
but only Russian and Czech did that (storytold/wordcraft#3252); the other seven reported the
disk file anyway.

`DictSlot` (proof/src/slot.rs) is that code once. It is generic over the
dictionary type, so Turkish's `TrDict` fits, and keeps the dictionary and
its in-memory origin under one lock as storytold/wordcraft#3252 made it. Each language keeps
its functions as one-line wrappers over its slot, so no caller changes, and
the log lines are the same.

installed_path now follows the documented rule in every language. Nothing
outside tests installs a dictionary in memory, but tests do, concurrently,
so the dictionaries-folder tests in proof and engine could race on the
reported path of a language another test installs. They now accept the
folder's file or none there, and check that the folder is searched first
with hunspell::find, which no install affects.

Tests: proof 279, engine 927, layout 501 (lib); slot.rs has a test of its
own.
2026-10-24 19:33:17 -01:00
VoiceLessQ f417d88631 Proofing options reach the Russian and Turkish checks too
storytold/wordcraft#3966 made Ignore words in UPPERCASE and Ignore words that contain numbers
reach six languages' own checks, one copy at a time, and Russian and Turkish
were left out. lib.rs still called `ru::is_correct(word)` and
`tr::is_correct(word)`, which accept words with digits (both) and short words
in capitals (Russian) whatever the options say, so turning either option off
changed nothing in those languages.

Both now have `is_correct_with_options`, like the others. Russian's rule for
a part in capitals (`ИИ-консультант`) follows the option too, as Indonesian's
does, and its digits rule keeps counting superscripts (`м²`). Turkish had no
test dictionary, so its two dictionary tests returned early wherever none was
installed, CI included; it has a synthetic one now, also in the
dictionaries-folder test.

Tests: proofing_options_reach_the_russian_check and
proofing_options_reach_the_turkish_check. Calling the old checks from lib.rs
fails each, and so does ungating the Russian part rule or the Turkish hyphen
recursion.
2026-10-24 19:19:25 -01:00
Federico Olivares 03bbefd484 Merge origin/main 2026-10-24 14:34:57 -03:00
dotCooCoo 51090dd6e5 Proofing: web addresses after a no-break space are not marked as misspelled
The spelling check found the start of the white-space-delimited token around
a word one byte after the last white-space character. After a no-break space
(two bytes in UTF-8) or an ideographic space (three bytes) that offset is
inside the character, so the token could not be read and was never classified
as a web or e-mail address. With Ignore web and file addresses on, the words
of such an address were still marked as misspelled.

The token now starts after the whole white-space character. A proofing test
covers a web address, a www. address and an e-mail address after the two
spaces, with the option on and off.
2026-10-24 01:54:29 -07:00
Dylan M. TaylorandClaude Sonnet 5.5 a75a9383b3 Sonar S1612: pass PoisonError::into_inner instead of a closure (storytold/wordcraft#4102)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 04:33:12 -04:00
ab30d37997 chore: resolve SonarCloud maintainability issues (storytold/wordcraft#4083)
* chore: resolve SonarCloud maintainability issues

* fix: restore compiling refactors, revert behaviour changes, run cargo fmt

- fontdb::norm: keep the closure for filter (char::is_alphanumeric takes
  self, not &self, so it does not fit Iterator::filter).
- story.rs: write_mark_rpr needs &mut self (rev_attrs, rpr_change).
- Revert the viewport meta change and the return 0 added to
  workspace_version/sha256, which changed behaviour or masked failures.
- cargo fmt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Import Document in odt page_parts after dropping glob re-export

* fix: apply_kind_properties binds ContentControl; cell_ref keeps char closures

* fix(engine): pass case helpers by value (clippy needless borrows)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan Taylor <dylan@craftmaker.dev>
2026-10-24 03:54:19 -04:00
Federico Olivares 205278483e Merge origin/main 2026-10-23 21:10:47 -03:00