Commit Graph
13 Commits
Author SHA1 Message Date
Martin BallandClaude Opus 5.5 cb82e91428 craft-proof — a run of spaces is one double-space issue over the whole run in every language, through one shared space_runs helper (#496 review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:23:15 +00:00
Martin BallandClaude Opus 5.5 1f345ea809 FU.5: craft-proof id, ru — the cased-letter rule for words in capitals (craft-libs#485): a caseless part is no abbreviation; shared in_capitals / capitals_part
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 8f16fe5a13 CP4.2: review — test dictionaries installed in memory hold the dictionary-tests lock until the test ends, so a pack or folder reload in another test can't drop them mid-test (the with-data ru suggestion flake)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 ee3be01b73 CP4.2: review — English tests hold the dictionaries lock; another region's pack never beats the system's exact region; bounded pack hyphenation lookups; NFC hyphenation; licences budgeted before reading; pack languages report like the others
P2:
- Every test whose English results depend on installed dictionaries takes DICTIONARY_TESTS (the en-GB goitre CI failure: tiny en_US fixtures raced unlocked en-GB tests).
- hunspell::candidates/find order: dictionaries folder, the first name's region pack (or the language alone), the system's exact-region dictionary, any other region's pack, the system's other names. pt_BR system beats a pt-pt pack; de_DE / de-ch, nl_NL / nl-be, es_ES / es-mx, en_US / en-gb likewise.

P3:
- Pack hyphenation lookups: a map, at most 256 languages (cleared when full); 3000 tags stay bounded and fast.
- A decomposed word (a + U+0301) is hyphenated composed (NFC) and the breaks mapped back; combining marks stay in their word.
- licence_texts checks each file's size against what is left before reading it; never reads past 16 MB.
- German, Bulgarian and pack languages: spelling message in the language where we have it; missing_dictionary/unreadable_dictionary report them like the others.
- build.rs: a missing file no longer says nothing is embedded; ru ignore message says CRAFT_I18N_DIR is enough. Built-in English list notice names the whole Moby list (en-words.dic).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Harrison SmithandClaude Opus 5.5 e4182149a5 proof: read FLAG num (and long) affix dictionaries (storytold/wordcraft#4967)
The affix engine read one-character flags only, at most 128 of them, as
bits of a u128. It now reads `FLAG num` (comma-separated numbers 1-65535,
leading digits as in Hunspell) and `FLAG long` (two characters), with up
to 65,536 distinct flags (storytold/wordcraft#4966):

- One reader (`FlagMode`) parses and writes every flag field: .dic flags,
  PFX/SFX flags and continuation classes, FORBIDDENWORD and NOSUGGEST.
- Each flag gets a dense u16 id; a stem's flags and a rule's continuation
  classes are sorted id sets interned in one flat table (set 0 is empty).
- Hostile input: at most 1,024 flags per entry (Belarusian uses 264), a
  bad .aff flag is an error, a .dic entry whose flags don't read is
  skipped, and a FLAG line after flags were read is refused.
- Lookups skip the continuation walk for groups and flags no continuation
  class can reach, so checks are as fast or faster with the Polish,
  Portuguese, Czech, Italian and Russian dictionaries (same answers).
- The compact format writes a `FLAG num`/`FLAG long` line only for those
  modes, so one-character data is byte-for-byte unchanged and old data
  still loads.

LibreOffice's Serbian (Cyrillic and Latin), Belarusian, Sinhala, Kannada
and Nepali dictionaries now load; Danish, Galician, Icelandic and Korean
still stop at COMPOUNDMIN, NEEDAFFIX, FULLSTRIP and ICONV.

Ported from storytold/wordcraft#5142 (f5778bb76) onto craft-proof.
craft-proof already had storytold/wordcraft#735 (AF aliases, SUBSTANDARD,
KEEPCASE, ICONV, IGNORE, 256 one-character flags as a bitmask), which
this commit predates; conflicts resolved by carrying those through the
flag ids: AF aliases resolve to flag text read in the FLAG mode,
SUBSTANDARD and KEEPCASE are flag ids like FORBIDDENWORD and NOSUGGEST,
and lookups exclude up to four flags (Excluded). The FLAG long test
lines in the language modules and slot.rs now use an unknown mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:28 +00:00
VoiceLessQ 140c7041e1 Proofing: the spelling options are read in one place
Czech, Croatian, Indonesian, Italian, Polish, Portuguese and Spanish began
their spelling checks with the same thirteen lines, byte for byte: no
dictionary means no marks, single letters and words without letters pass,
words with digits and short words in capitals pass when the File › Options
› Proofing choices say so, and words in another script pass. Each also had
its own copy of the Latin-script test. These were the lines storytold/wordcraft#3966 had to fix
in six copies one by one, and where the Russian and Turkish copies stayed
wrong until the first commit here.

spelling.rs has them once (`word_to_check`), and each language goes on
from there with its own rules (Polish apostrophe endings, Italian elisions,
Indonesian clitics, Uruguayan `vos` forms).

Tests: words_needing_no_lookup. Ignoring the uppercase option there fails
it and all seven languages' proofing-options tests. Proof: 286 tests.
2026-10-24 19:58:30 -01:00
VoiceLessQ a388c4787c Proofing: one language table instead of nine lists
lib.rs named every dictionary language in nine places: reloading for a new
folder, the installed list, load errors, the missing-dictionary hint, the
loaded check, the spelling message, spelling, suggestions and grammar. The
lists had drifted. dictionary_load_error had no Russian or Czech entry, so a
Russian or Czech dictionary that failed to load showed no reason in File ›
Options › Proofing, though the status bar, which reads another list, said
why.

LANGUAGES holds one row per language: its dictionary slot, its spelling
message, and its spelling, suggestion and grammar functions. The nine
functions read the row, so a language now joins with its module, its
ProofLang and one row. The per-language reload_dictionary and
installed_path wrappers had no callers left, inside the crate or out, and
are gone.

Tests: every_dictionary_language_reports_its_load_error, which fails for
Russian on the old code, and the_language_table_covers_the_proofed_languages
(each ProofLang::PROOFED language but English has exactly one row). Proof
285, engine 927 and layout 501 (lib).
2026-10-24 19:55:43 -01:00
VoiceLessQ c19b1cae73 Proofing: one suggestion engine with per-language tuning
Czech's dictionary-driven ranking (case, MAP, REP, missing spaces, KEY,
TRY edits, a stem scan) had five older copies. Polish, Portuguese,
Croatian, Italian and Russian each kept one: 300 to 400 lines of the same
rounds with a few lines of language knowledge mixed in. Fixes reached some
copies only. NOSUGGEST (storytold/wordcraft#1048) and the first-difference tie-break (storytold/wordcraft#797)
never got to all of them, and none had the KEY round or the bound on TRY
letters.

affix_suggest.rs is that ranking once. A language's `Tuning` holds what is
its own: the short words that run into their neighbours (Polish,
Portuguese, Croatian, Italian), Russian's split rule, the letter groups and
confusions its dictionaries lack (Croatian, Russian), letters to try for a
dictionary without a TRY line, and cost hooks for Polish final devoicing
and Russian е/ё. Czech, Indonesian and Spanish use the dictionary's tables
alone. Turkish keeps its own suggester, which reads another dictionary type
with rounds of its own.

The language tables add to the dictionary's instead of replacing them, and
TRY letters are tried in the dictionary's order, commonest first, as the
copies did (the Czech engine sorted them).

Old and new compared on the test dictionaries, with every one-letter typo
of each word plus the tests' misspellings, top eight suggestions: Czech,
Croatian, Italian and Portuguese give the same for all 7,903 inputs.
Polish differs for 44 of 3,244 and Russian for 33 of 2,346, all orderings
among candidates of equal cost, from the tie-break they now share (`ie` →
`nie` before `i`).

New tests pin the moved knowledge: the tuning mechanism, Croatian's groups
without MAP lines, and Russian's costs and split rule. Turning each knob
off fails one. Proof: 283 tests.
2026-10-24 19:49:22 -01:00
VoiceLessQ 8f4616eece Proofing: one dictionary slot for every language
Nine languages each kept their own copy of the same code: the installed
dictionary behind a lock, read on first use and logged, replaced by
install_dictionary, forgotten by reload_dictionary, and its file reported
by installed_path. The copies had drifted already. Every one documented that
a dictionary given with install_dictionary has no file and isn't reported,
but only Russian and Czech did that (storytold/wordcraft#3252); the other seven reported the
disk file anyway.

`DictSlot` (proof/src/slot.rs) is that code once. It is generic over the
dictionary type, so Turkish's `TrDict` fits, and keeps the dictionary and
its in-memory origin under one lock as storytold/wordcraft#3252 made it. Each language keeps
its functions as one-line wrappers over its slot, so no caller changes, and
the log lines are the same.

installed_path now follows the documented rule in every language. Nothing
outside tests installs a dictionary in memory, but tests do, concurrently,
so the dictionaries-folder tests in proof and engine could race on the
reported path of a language another test installs. They now accept the
folder's file or none there, and check that the folder is searched first
with hunspell::find, which no install affects.

Tests: proof 279, engine 927, layout 501 (lib); slot.rs has a test of its
own.
2026-10-24 19:33:17 -01:00
Dylan M. TaylorandClaude Sonnet 5.5 a75a9383b3 Sonar S1612: pass PoisonError::into_inner instead of a closure (storytold/wordcraft#4102)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 04:33:12 -04:00
57943c142e Proofing options reach the Czech, Croatian, Indonesian, Italian, Polish and Portuguese checks (storytold/wordcraft#3966)
Each of these languages accepted every word in capitals of up to six letters and every word
with a digit on its own, so turning off Ignore words in UPPERCASE or Ignore words that contain
numbers had no effect on them for short words. The check now takes the options, as the Spanish
one does; the direct is_correct/is_correct_with keep the defaults.

Co-authored-by: Federico Olivares <FedeOlivares@users.noreply.github.com>
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
2026-10-23 20:07:34 -04:00
MarekSpodBiedryandBrandon Thomas 401ea058d1 Proofing: say when a dictionary is installed but unreadable, with file and reason (storytold/wordcraft#2830) (storytold/wordcraft#2900)
* Proofing: say when a dictionary is installed but unreadable, with file and reason (storytold/wordcraft#2830)

* proof test: use SHIFT_JIS as the unsupported encoding (ISO 8859-1 is decoded since storytold/wordcraft#2904)

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 11:51:13 -04:00
4da49a6fe8 storytold/wordcraft#498: Indonesian (id-ID) spelling proofing (storytold/wordcraft#2576)
Indonesian spelling reads the installed Hunspell id_ID dictionary at runtime
(Debian/Ubuntu hunspell-id, LibreOffice dict-id, the Dictionaries folder or
WORDCRAFT_DICTIONARIES), like Czech, Polish and Portuguese: nothing is bundled,
and without a dictionary Indonesian text gets no spelling marks.

- ProofLang::Id for id, id-ID, ind and the legacy in tag
- suggestions reuse the dictionary-driven Czech ranking (MAP/REP/TRY/KEY of
  the loaded dictionary), nothing Indonesian hard-coded
- hyphenated reduplication accepted whole or by parts (kupu-kupu, anak-anak);
  hyphenated clitics after capitals/abbreviations (KTP-nya, se-Indonesia)
- generic grammar only: repeated word (offers anak-anak first), space before
  punctuation, double space, with Indonesian messages
- Review > Language, status bar, Set Proofing Language, File > Options >
  Proofing list Indonesian; name translated in every catalog
- tests use a hand-written synthetic dictionary; engine end-to-end test

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 03:42:52 -04:00