Follow-ups from craft-libs#485: up to 256 languages' pack patterns (64 MB files each) were kept
by count only; `hunspell::ordered` checked pack files with `is_file`, which follows links,
while the pack scan refuses them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
Follow-ups from craft-libs#485: `CHECK_FROM` is reset by a drop guard; `remember_missing` drops
the oldest of 256; a word none of whose letters the dictionary's words use (a Cyrillic word in
a Dutch run) is left alone, as Word does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
SCOWL en-AU has 238 words en-GB lacks (billabong, brumby, larrikinism, Australian place
names) and spells program, inquiry, livable, licorice, verandah the Australian way (no
programme, enquiry, liveable, dialog or veranda, as Hunspell en_AU says); -ise/-ize policy and
affix file are the same as en-GB's. craft-i18n already ships language-data/en-au.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
The total flag-set bound (MAX_SET_DATA, 8.4M) stays the hostile-input guard: LibreOffice's
tr_TR loads with 5.2M of it in 0.6 s, and abart, abartmıştım, savaşıyor, inandım, aranıyor,
açılamadı and bilinmiyor are right, as hunspell 1.7 answers. The merged-homonym cap is now
4 × the entry cap (32,768), under the 65,536 distinct flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
hunspell::candidates_exact (folder → exact-locale pack → system, own names only) for English without SCOWL; packs::exact_spelling_dirs. Other languages keep the region-ordered cross-region tiers. Drops has_british_source().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
- build.rs: unset CRAFT_PROOF_EMBED embeds de, pl, pt, it patterns and ru-extra as before (DEFAULT_EMBED); `en` embeds English and our supplements only; `de,pl` and `all` as before. Making `en` the default is CP4.4, in the release that publishes and pins language-packs-1. The default-build data tests cover pl/it/de/pt again; CI's extra step is now the English-only build.
- licence_texts: one Moby entry (the data's, else the built-in en-words.dic) and en-extra.txt (WordCraft contributors, MIT OR Apache-2.0); the deleted core-list notice is gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
The goitre failure was not a race: GitHub's Ubuntu runners have a system en_US Hunspell dictionary, which en-GB uses without the Language data (by design), and it marks goitre. Reproduced locally with a fake system en_US; the test now checks British words only when en-GB reads the Moby list or a British dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
P2:
- Every test whose English results depend on installed dictionaries takes DICTIONARY_TESTS (the en-GB goitre CI failure: tiny en_US fixtures raced unlocked en-GB tests).
- hunspell::candidates/find order: dictionaries folder, the first name's region pack (or the language alone), the system's exact-region dictionary, any other region's pack, the system's other names. pt_BR system beats a pt-pt pack; de_DE / de-ch, nl_NL / nl-be, es_ES / es-mx, en_US / en-gb likewise.
P3:
- Pack hyphenation lookups: a map, at most 256 languages (cleared when full); 3000 tags stay bounded and fast.
- A decomposed word (a + U+0301) is hyphenated composed (NFC) and the breaks mapped back; combining marks stay in their word.
- licence_texts checks each file's size against what is left before reading it; never reads past 16 MB.
- German, Bulgarian and pack languages: spelling message in the language where we have it; missing_dictionary/unreadable_dictionary report them like the others.
- build.rs: a missing file no longer says nothing is embedded; ru ignore message says CRAFT_I18N_DIR is enough. Built-in English list notice names the whole Moby list (en-words.dic).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
hunspell::candidates (which English now loads from) searches the pack tier between the dictionaries folder and the system, as find() does; pack-loaded English goes through the same reloads guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
Languages without tools of their own (nl, fr, sr-Latn…) are spell-checked and given suggestions
through the shared affix engine with default tuning, and hyphenated from their pack's patterns;
no grammar. At most 8 such dictionaries stay loaded. Never another script's pack.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
Tables from the Unicode Consortium's 8859-7.TXT (2003, with € ₯ ͺ) and
8859-15.TXT mapping files; unassigned bytes become U+FFFD. These are the
only manifest SETs craft-proof couldn't decode. el now loads and passes its
check words; et loads and next stops at COMPOUNDFLAG (compounding,
storytold/wordcraft#2070).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
Recasing a suggestion to capitals turned ß into SS for every dictionary,
so without CHECKSHARPS "STRAßEE" was offered "STRASSE", which check()
rejects. Now ß stays ß (STRAßE); under CHECKSHARPS the SS form is offered
first, and a KEEPCASE word with ß may be capitalized or written in capitals
with SS (FUSBALL → FUSSBALL, fußball; Fusball → Fußball; GRUS → GRUSS,
Gruß). Every expected list is the hunspell 1.7.5 CLI's on the same files
(sharp_s_suggestions_match_hunspell), and every suggestion passes check().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
With CHECKSHARPS, a word in capitals may write `SS` for `ß` (`STRASSE`,
`MASSE`), and a KEEPCASE word with `ß` may be capitalized and written in
capitals with `SS` but not with `ß` (`Fußball`, `FUSSBALL`, not
`FUßBALL`). The first five `ss` pairs are tried in every combination.
`ß` counts as neither case when a word's casing is read, as in Hunspell,
so `STRAßE` is in capitals (correct with or without CHECKSHARPS). Every
answer in the test matches Hunspell 1.7's on the same files.
de, de-at and de-ch now stop at compounding (storytold/wordcraft#2070).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
- An OCONV-written suggestion (colour’s) is mapped to the run's English
spelling: the dialect lists are read with ', and the result keeps the
dictionary's apostrophe (color’s). Straight and curly duplicates (color's,
color’s) are one suggestion. Test: curly_suggestions_are_mapped_to_the_dialect.
- Conv::at looks up each prefix of the text (at most MAX_CONV_BYTES) in a
hash map instead of scanning the pairs under its first character: 65,536
pairs on one character cost 64 lookups per position, not a scan (Korean's
OCONV has 588 pairs on one character, so a per-character cap would not
fit). Test: a_huge_conversion_table_on_one_character_stays_fast.
- The reviewers' OCONV repros are a test (oconv_suggestions_match_hunspell):
OCONV applies after a suggestion's case is set, case-sensitively, as the
hunspell 1.7.5 CLI answers (KBT → KAT; Kbt → Kat under k → q).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
48ed6f8's lenient English path ignored OCONV because craft-proof couldn't
read it. Now that it can (CP6.4), English reads it like every language:
suggestions are shown as the dictionary writes them (`don’t` with the
usual en_US/en_GB `OCONV ' ’`), as Hunspell shows them, and ICONV reads
them back. Compounding is still ignored for English.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
- OCONV (output conversion, as Hunspell 1.7 applies it): read from the
.aff and the compact format, applied to suggestions as they are shown
(`AffixDict::output`); what is correct doesn't change. The French
(Grammalecte) dictionary writes `’` for the `'` its stems use.
- ICONV and OCONV tables hold at most 65,536 pairs of at most 64 bytes a
side (the Korean tables hold 11,172); more is an error. Each table is
indexed by the first character of its `from` texts, so converting a word
looks at the few pairs that can start at each character instead of
scanning the whole table per character (review of storytold/wordcraft#5000).
- Input longer than MAX_INPUT_BYTES (4 × MAX_WORD_BYTES) is no word and
is no longer converted before the length check rejects it (review of
storytold/wordcraft#5000); input made long by IGNORE characters within
that bound still checks.
- Longest match wins, and of equal ones the later pair, as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
