Commit Graph
8 Commits
Author SHA1 Message Date
Martin BallandClaude Opus 5.5 83a4d73c69 proof: affix_check generators give up on a dictionary with no usable stems
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:46:00 +00:00
Martin BallandClaude Opus 5.5 b8efc5add9 CP7.4: craft-libs#497 re-review — hu prefixed suffix chains, homonyms kept apart, all-caps mixed-case stems; compound suggestions cheapest-first in one budget
- A prefix with two chained suffixes (hu `meg`+`épít`+`és`+`e`, eo `enjugo`): every rule
  cross, the prefix flag from the stem or a continuation class (CLI answers).
- Homonyms with different flags are kept apart (and written apart in the compact form):
  a prefix and a suffix, or a compound flag and a suffix, must come from one homonym
  (nl `afstopje`, `schuurpoederden` wrong in the CLI). nl now 100% in both comparisons.
- Hungarian syllable rule leaves out a last suffix with no continuation classes;
  COMPOUNDROOT counts on the stem only.
- Near-word check: a word listed as an ONLYINCOMPOUND/NEEDAFFIX stem counts only with a
  suffix, not a prefix (de `Fahrradverkehr` right).
- A word in capitals may be a mixed-case stem (`KB`/`KBS` from `kB/S`, hu `DNY`, nl `BSC`).
- Mixed-case retry keeps KEEPCASE for the first part; KEEPCASE kept apart from the other
  exclusions (NOSUGGEST sharing its flag is no longer cleared); overflow of the homonym
  groups counts as dropped.
- Suggestions: candidates are checked as simple words first; the rest get the compound
  check cheapest first, those changed where the word stops being parts before others, in
  one step budget per call, with piece analyses shared between candidates. The suggestions'
  stem scan makes forms in one shared budget (mn).
- affix_check example: `suggest` command; AffixDict::suggest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 03:18:31 +00:00
Martin BallandClaude Opus 5.5 3eaa276786 CP7.3: craft-libs#497 review — compounds as the hunspell CLI answers: ONLYINCOMPOUND homonyms keep their flags, triples after SIMPLIFIEDTRIPLE, KEEPCASE and capitals in parts, CHECKCOMPOUNDPATTERN 0, a forbidden prefix, FORCEUCASE; suggestions and many-way pieces stay cheap
- ONLYINCOMPOUND homonyms beside others are kept apart (compound parts only), so their suffixes
  make no standalone words (de verfilmungs, sv norges); round-trips as a second stem line.
- CHECKCOMPOUNDTRIPLE also checks a SIMPLIFIEDTRIPLE join (busssen, booob wrong).
- KEEPCASE holds for the first part only; a capitalized stem is a part (FooQux).
- CHECKCOMPOUNDPATTERN `0`: no suffix (a prefix doesn't count): unfoobar wrong.
- COMPOUNDFORBIDFLAG on a prefix: no part anywhere (bardefoo wrong).
- FORCEUCASE: from the stem only; wrong only without any capital (fooQuxbar right).
- Suggestion candidates: 2,048 compound steps each, compounds for the first 128 only.
- A piece's ways are hashed, capped at 256 and counted (with the patterns) against the budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 02:08:35 +00:00
Martin BallandClaude Opus 5.5 9a442e7048 CP7.2: affix_check example — compare the affix engine with another checker's answers (black-box)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:43:54 +00:00
Martin BallandClaude Opus 5.5 d2d5c737a6 CP6.13: craft-libs#482/#485/#488 review — NOTICE carries the embedded Portuguese patterns' BSD notice; README: the usage example defines text, the eight-candidate limit, what the default build embeds; en_words writes under crates/proof
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:27:37 +00:00
Martin BallandClaude Opus 5.5 9e5862a1c1 CP1.2: each English uses only its own Hunspell dictionary, else the built-in Moby list
Re-review of craft-libs#482:
- en-GB and en-CA never fall back to an en_US Hunspell dictionary, which marked colour,
  organise, centre… (CI's runner has one); without their own they use the built-in Moby list,
  which has both spellings. en-US looks for en_US/en-US only.
- hunspell::candidates goes name by name across every folder, so a system en_GB beats an
  en_US in the dictionaries folder.
- Built-in suggestions rank swaps, then common words, neighbouring keys, the same first letter;
  no one-letter suggestions but a and i (hte -> the, recieve -> receive, wprd -> word).
- The generator leaves out Moby entries that are far more often typos: teh, abd, nad, ur, u, r
  (wont and cant stay: words). en-words.dic regenerated (343,988 bytes).
- A load that started before a reload no longer notes its load error; reload clears the
  English load errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:45:01 +00:00
Martin BallandClaude Opus 5.5 c80b6225f1 CP1.2: English without Language data uses the whole Moby list and every installed English dictionary
Review of craft-libs#482:
- The built-in fallback is now the full public-domain Moby word list (data/en-words.dic,
  words only, 343,992 bytes, from craft-i18n's en-us.dic by examples/en_words.rs) plus
  WordCraft's own en-extra.txt (MIT OR Apache-2.0). An unknown word is misspelled; the
  near-miss scan (which flagged fiber, theater, organize… and took 14 s on a 2,000-word
  token) is gone. Suggestions are the words one edit away, looked up, for words of up to 32
  letters. spelling_scales_over_one_long_token runs without the data, with a time bound.
- English Hunspell dictionaries ignore compounding (COMPOUNDMIN, COMPOUNDRULE…; their
  ONLYINCOMPOUND stems are left out) and OCONV instead of refusing the file, and a file that
  can't be read is skipped for the next name or folder.
- A load that started before english::reload() isn't kept (reloads counter, as slot.rs);
  Englishes that use the same .dic share one copy. en-GB/en-CA falling back to en_US is
  documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 18:37:02 +00:00
Martin BallandClaude Opus 5.5 23151270ab CP1.2: English never goes unchecked — embedded data, then system Hunspell, then a built-in core word list
- No embedded Moby list: English (en-US, en-GB, en-CA) uses an English
  Hunspell .aff/.dic from the dictionaries folder or the system (the run's
  English first, then another English), else data/en-core.txt.
- en-core.txt: 11,367 public-domain Moby Hyphenator II words (Grady Ward),
  ranked by lower-case frequency in 50 public-domain Gutenberg books
  (examples/en_core.rs, book ids recorded). 94 KB. Marks only near misses
  of common words (swapped letters; one edit for lower-case words of 4+).
- Unreadable English dictionary (encoding, not UTF-8, no words, size):
  error via dictionary_load_error, falls through to the core list; a bad
  embedded list is reported the same way. Never a panic.
- Non-English text is never checked against English.
- New: english_source(lang) -> Option<EnglishSource>.
- Embedded-data path unchanged; tests with data unchanged.
- Tests: core list flags/accepts, folder Hunspell preferred over core list,
  unreadable dictionaries reported, no_data English now marks misspellings;
  two data-ignored tests now run without data. README, ATTRIBUTION, NOTICE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 10:55:57 +00:00