Commit Graph
23 Commits
Author SHA1 Message Date
Martin BallandClaude Opus 5.5 cb82e91428 craft-proof — a run of spaces is one double-space issue over the whole run in every language, through one shared space_runs helper (#496 review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:23:15 +00:00
Martin BallandClaude Opus 5.5 61030150dc craft-proof cs, hr, it — a hyphenated word is looked up whole before its parts, and an elision (l'ex-amico) is read in each part (#496 review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:56:44 +00:00
Martin BallandClaude Opus 5.5 9ef9185131 FU.8: craft-proof cs — regression test from wordcraft#4683: a letter that lowercases to two chars (İ) still gets its one-letter edits
The shared suggestion code already edits across the whole lowercased word, so no code change is needed; the test keeps it that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:43:25 +00:00
Martin BallandClaude Opus 5.5 c5cc9c4b7a CP6.13: craft-libs#495 review — cs: ne ne, ano ano, už už and to to are no repeated words; ru: no comma asked for inside тем более что, при условии что, в то время как; -ой adjectives take и after г, к, х, ж, ш, ч, щ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:06:58 +00:00
Martin BallandClaude Opus 5.5 8f16fe5a13 CP4.2: review — test dictionaries installed in memory hold the dictionary-tests lock until the test ends, so a pack or folder reload in another test can't drop them mid-test (the with-data ru suggestion flake)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 ee3be01b73 CP4.2: review — English tests hold the dictionaries lock; another region's pack never beats the system's exact region; bounded pack hyphenation lookups; NFC hyphenation; licences budgeted before reading; pack languages report like the others
P2:
- Every test whose English results depend on installed dictionaries takes DICTIONARY_TESTS (the en-GB goitre CI failure: tiny en_US fixtures raced unlocked en-GB tests).
- hunspell::candidates/find order: dictionaries folder, the first name's region pack (or the language alone), the system's exact-region dictionary, any other region's pack, the system's other names. pt_BR system beats a pt-pt pack; de_DE / de-ch, nl_NL / nl-be, es_ES / es-mx, en_US / en-gb likewise.

P3:
- Pack hyphenation lookups: a map, at most 256 languages (cleared when full); 3000 tags stay bounded and fast.
- A decomposed word (a + U+0301) is hyphenated composed (NFC) and the breaks mapped back; combining marks stay in their word.
- licence_texts checks each file's size against what is left before reading it; never reads past 16 MB.
- German, Bulgarian and pack languages: spelling message in the language where we have it; missing_dictionary/unreadable_dictionary report them like the others.
- build.rs: a missing file no longer says nothing is embedded; ru ignore message says CRAFT_I18N_DIR is enough. Built-in English list notice names the whole Moby list (en-words.dic).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Harrison SmithandClaude Opus 5.5 e4182149a5 proof: read FLAG num (and long) affix dictionaries (storytold/wordcraft#4967)
The affix engine read one-character flags only, at most 128 of them, as
bits of a u128. It now reads `FLAG num` (comma-separated numbers 1-65535,
leading digits as in Hunspell) and `FLAG long` (two characters), with up
to 65,536 distinct flags (storytold/wordcraft#4966):

- One reader (`FlagMode`) parses and writes every flag field: .dic flags,
  PFX/SFX flags and continuation classes, FORBIDDENWORD and NOSUGGEST.
- Each flag gets a dense u16 id; a stem's flags and a rule's continuation
  classes are sorted id sets interned in one flat table (set 0 is empty).
- Hostile input: at most 1,024 flags per entry (Belarusian uses 264), a
  bad .aff flag is an error, a .dic entry whose flags don't read is
  skipped, and a FLAG line after flags were read is refused.
- Lookups skip the continuation walk for groups and flags no continuation
  class can reach, so checks are as fast or faster with the Polish,
  Portuguese, Czech, Italian and Russian dictionaries (same answers).
- The compact format writes a `FLAG num`/`FLAG long` line only for those
  modes, so one-character data is byte-for-byte unchanged and old data
  still loads.

LibreOffice's Serbian (Cyrillic and Latin), Belarusian, Sinhala, Kannada
and Nepali dictionaries now load; Danish, Galician, Icelandic and Korean
still stop at COMPOUNDMIN, NEEDAFFIX, FULLSTRIP and ICONV.

Ported from storytold/wordcraft#5142 (f5778bb76) onto craft-proof.
craft-proof already had storytold/wordcraft#735 (AF aliases, SUBSTANDARD,
KEEPCASE, ICONV, IGNORE, 256 one-character flags as a bitmask), which
this commit predates; conflicts resolved by carrying those through the
flag ids: AF aliases resolve to flag text read in the FLAG mode,
SUBSTANDARD and KEEPCASE are flag ids like FORBIDDENWORD and NOSUGGEST,
and lookups exclude up to four flags (Excluded). The FLAG long test
lines in the language modules and slot.rs now use an unknown mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:28 +00:00
VoiceLessQ 140c7041e1 Proofing: the spelling options are read in one place
Czech, Croatian, Indonesian, Italian, Polish, Portuguese and Spanish began
their spelling checks with the same thirteen lines, byte for byte: no
dictionary means no marks, single letters and words without letters pass,
words with digits and short words in capitals pass when the File › Options
› Proofing choices say so, and words in another script pass. Each also had
its own copy of the Latin-script test. These were the lines storytold/wordcraft#3966 had to fix
in six copies one by one, and where the Russian and Turkish copies stayed
wrong until the first commit here.

spelling.rs has them once (`word_to_check`), and each language goes on
from there with its own rules (Polish apostrophe endings, Italian elisions,
Indonesian clitics, Uruguayan `vos` forms).

Tests: words_needing_no_lookup. Ignoring the uppercase option there fails
it and all seven languages' proofing-options tests. Proof: 286 tests.
2026-10-24 19:58:30 -01:00
VoiceLessQ a388c4787c Proofing: one language table instead of nine lists
lib.rs named every dictionary language in nine places: reloading for a new
folder, the installed list, load errors, the missing-dictionary hint, the
loaded check, the spelling message, spelling, suggestions and grammar. The
lists had drifted. dictionary_load_error had no Russian or Czech entry, so a
Russian or Czech dictionary that failed to load showed no reason in File ›
Options › Proofing, though the status bar, which reads another list, said
why.

LANGUAGES holds one row per language: its dictionary slot, its spelling
message, and its spelling, suggestion and grammar functions. The nine
functions read the row, so a language now joins with its module, its
ProofLang and one row. The per-language reload_dictionary and
installed_path wrappers had no callers left, inside the crate or out, and
are gone.

Tests: every_dictionary_language_reports_its_load_error, which fails for
Russian on the old code, and the_language_table_covers_the_proofed_languages
(each ProofLang::PROOFED language but English has exactly one row). Proof
285, engine 927 and layout 501 (lib).
2026-10-24 19:55:43 -01:00
VoiceLessQ c19b1cae73 Proofing: one suggestion engine with per-language tuning
Czech's dictionary-driven ranking (case, MAP, REP, missing spaces, KEY,
TRY edits, a stem scan) had five older copies. Polish, Portuguese,
Croatian, Italian and Russian each kept one: 300 to 400 lines of the same
rounds with a few lines of language knowledge mixed in. Fixes reached some
copies only. NOSUGGEST (storytold/wordcraft#1048) and the first-difference tie-break (storytold/wordcraft#797)
never got to all of them, and none had the KEY round or the bound on TRY
letters.

affix_suggest.rs is that ranking once. A language's `Tuning` holds what is
its own: the short words that run into their neighbours (Polish,
Portuguese, Croatian, Italian), Russian's split rule, the letter groups and
confusions its dictionaries lack (Croatian, Russian), letters to try for a
dictionary without a TRY line, and cost hooks for Polish final devoicing
and Russian е/ё. Czech, Indonesian and Spanish use the dictionary's tables
alone. Turkish keeps its own suggester, which reads another dictionary type
with rounds of its own.

The language tables add to the dictionary's instead of replacing them, and
TRY letters are tried in the dictionary's order, commonest first, as the
copies did (the Czech engine sorted them).

Old and new compared on the test dictionaries, with every one-letter typo
of each word plus the tests' misspellings, top eight suggestions: Czech,
Croatian, Italian and Portuguese give the same for all 7,903 inputs.
Polish differs for 44 of 3,244 and Russian for 33 of 2,346, all orderings
among candidates of equal cost, from the tie-break they now share (`ie` →
`nie` before `i`).

New tests pin the moved knowledge: the tuning mechanism, Croatian's groups
without MAP lines, and Russian's costs and split rule. Turning each knob
off fails one. Proof: 283 tests.
2026-10-24 19:49:22 -01:00
VoiceLessQ 8f4616eece Proofing: one dictionary slot for every language
Nine languages each kept their own copy of the same code: the installed
dictionary behind a lock, read on first use and logged, replaced by
install_dictionary, forgotten by reload_dictionary, and its file reported
by installed_path. The copies had drifted already. Every one documented that
a dictionary given with install_dictionary has no file and isn't reported,
but only Russian and Czech did that (storytold/wordcraft#3252); the other seven reported the
disk file anyway.

`DictSlot` (proof/src/slot.rs) is that code once. It is generic over the
dictionary type, so Turkish's `TrDict` fits, and keeps the dictionary and
its in-memory origin under one lock as storytold/wordcraft#3252 made it. Each language keeps
its functions as one-line wrappers over its slot, so no caller changes, and
the log lines are the same.

installed_path now follows the documented rule in every language. Nothing
outside tests installs a dictionary in memory, but tests do, concurrently,
so the dictionaries-folder tests in proof and engine could race on the
reported path of a language another test installs. They now accept the
folder's file or none there, and check that the folder is searched first
with hunspell::find, which no install affects.

Tests: proof 279, engine 927, layout 501 (lib); slot.rs has a test of its
own.
2026-10-24 19:33:17 -01:00
Federico Olivares 03bbefd484 Merge origin/main 2026-10-24 14:34:57 -03:00
Dylan M. TaylorandClaude Sonnet 5.5 a75a9383b3 Sonar S1612: pass PoisonError::into_inner instead of a closure (storytold/wordcraft#4102)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 04:33:12 -04:00
Federico Olivares 205278483e Merge origin/main 2026-10-23 21:10:47 -03:00
57943c142e Proofing options reach the Czech, Croatian, Indonesian, Italian, Polish and Portuguese checks (storytold/wordcraft#3966)
Each of these languages accepted every word in capitals of up to six letters and every word
with a digit on its own, so turning off Ignore words in UPPERCASE or Ignore words that contain
numbers had no effect on them for short words. The check now takes the options, as the Spanish
one does; the direct is_correct/is_correct_with keep the defaults.

Co-authored-by: Federico Olivares <FedeOlivares@users.noreply.github.com>
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
2026-10-23 20:07:34 -04:00
Federico Olivares 6d27ab0de9 Merge origin/main into spanish-proofing
Upstream merged storytold/wordcraft#3177 (dictionary hints for every Hunspell language) and storytold/wordcraft#3480.
Resolve the conflicts in crates/proof/src/lib.rs by taking storytold/wordcraft#3177's
dictionary_loaded/dictionary_needed structure and adding the four Spanish
varieties to it, and in docs/spelling-dictionaries.md keeping both sides.
2026-10-23 19:06:06 -03:00
36651e4df8 Options › Language names the proofing language at the caret (storytold/wordcraft#3150) (storytold/wordcraft#3177)
The note under "Proofing language" always said English (United States). It now names
the language at the caret, as the status bar does, and the 22 catalogs carry the new sentence.

Co-authored-by: Naxvog <4963183+naxvog@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
2026-10-23 17:24:46 -04:00
Federico Olivares 970bfcf7a8 storytold/wordcraft#3360: Spanish proofing code clean-up (no behaviour change)
- es/mod.rs: one variety() match gives the dictionary slot, log name and file
  names (variety_name() folded in); load_installed() destructures it
- es/grammar.rs: the sentence-start rule for the missing ¿/¡ is one named
  function, ends_clause(), replacing is_boundary() and three closures
  rebuilt per mark; in_address() is shared; repeated_words() builds its
  helpers once instead of per word pair
- es/suggest.rs reuses the Czech ranking's MAX_LETTERS instead of a copy

Same tests pass unchanged; a temporary old-vs-new differential proptest
(20,000+ generated inputs with URLs and punctuation) found identical output.
2026-10-23 09:19:18 -03:00
MichaelandClaude Sonnet 5 eab36a9ec8 Update the dictionary and its in-memory origin under one lock
CodeRabbit (storytold/wordcraft#3252): reload_dictionary() wrote DICT and DICT_IN_MEMORY
under two separate locks. If install_dictionary() ran between them,
the final state could hold an in-memory dictionary with the flag left
false, so installed_path() would report a same-named disk file
instead of None.

Combined both into one DictState struct behind a single RwLock in
both ru and cs, so install_dictionary() and reload_dictionary() each
update the dictionary and its origin in one critical section.

Didn't add a dedicated concurrency test: genuinely proving a race's
absence needs real multi-threaded stress testing, and the previous
commit on this PR already found that tests directly exercising
install_dictionary()/reload_dictionary() concurrency in this crate's
shared global state cause real, reproducible flakiness in unrelated
tests. The fix is correct by construction (one lock acquisition around
both field writes) and all 222 existing proof tests, run 15 times in
a row, stay green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-20 17:41:20 +08:00
MichaelandClaude Sonnet 5 36d64a2859 installed_path() ignores a matching file while an in-memory dictionary is in use
CodeRabbit (storytold/wordcraft#3252): installed_path() searched the filesystem
unconditionally, so a dictionary given directly with install_dictionary
(a user-picked one, or a test fixture) was reported as the file of a
same-named dictionary that happened to also be installed, even though
spelling was actually using the in-memory one, not that file.

Added a DICT_IN_MEMORY flag to both modules, set in install_dictionary
and cleared in reload_dictionary; installed_path() now returns None
immediately when it's set, before even searching the filesystem.

Tried adding a dedicated test that calls install_dictionary() and
asserts installed_path() is None. It made dictionaries_folder_is_
searched_first flaky (~15-35% of runs across repeated local testing):
ru's and cs's own test modules already call install_dictionary()
directly in several tests of their own, sharing the same process-wide
static, and dictionaries_folder_is_searched_first's exact-path
assertion for Russian/Czech could observe whichever one last touched
it. Reverted the dedicated test; the production fix alone never
touches that shared state (it's a pure read-only check) and caused no
flakiness by itself across 25 repeated runs. Instead, narrowed
dictionaries_folder_is_searched_first's Russian/Czech assertion to the
language only, not the exact path, since unlike the other four
languages, installed_path() for these two is now sensitive to
concurrently-running tests' legitimate in-memory installs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-20 17:31:06 +08:00
MichaelandClaude Sonnet 5 ac06c87831 Russian and Czech dictionaries are reported, not just checked (storytold/wordcraft#3226)
installed_dictionaries() never included Russian or Czech at all,
because neither crates/proof/src/ru nor crates/proof/src/cs had an
installed_path() function for it to call (every other proofed
Hunspell language — Polish, Portuguese, Italian, Croatian, Turkish,
Indonesian — already did). File > Options > Proofing's dictionaries
list iterates installed_dictionaries() directly, so Russian and Czech
never showed up there at all, installed or not, matching the report.

Added installed_path() to both modules, mirroring the existing
pattern (crate::hunspell::find(NAMES).map(|(_, dic)| dic)), and added
both to installed_dictionaries().

The actual spell-checking dispatch (is_correct_in_with) already
routed Russian and Czech correctly — confirmed by installing the real
system ru_RU Hunspell dictionary and checking it directly
(ru::is_correct("молоко") true, ru::is_correct("малоко") false), so
that part of the pipeline isn't what's broken. The reporter's other
symptom (Russian text in the document not being checked at all) did
not reproduce with the real dictionary installed; I could not pin
that part down further without more specific repro details from
their setup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-20 16:50:41 +08:00
4da49a6fe8 storytold/wordcraft#498: Indonesian (id-ID) spelling proofing (storytold/wordcraft#2576)
Indonesian spelling reads the installed Hunspell id_ID dictionary at runtime
(Debian/Ubuntu hunspell-id, LibreOffice dict-id, the Dictionaries folder or
WORDCRAFT_DICTIONARIES), like Czech, Polish and Portuguese: nothing is bundled,
and without a dictionary Indonesian text gets no spelling marks.

- ProofLang::Id for id, id-ID, ind and the legacy in tag
- suggestions reuse the dictionary-driven Czech ranking (MAP/REP/TRY/KEY of
  the loaded dictionary), nothing Indonesian hard-coded
- hyphenated reduplication accepted whole or by parts (kupu-kupu, anak-anak);
  hyphenated clitics after capitals/abbreviations (KTP-nya, se-Indonesia)
- generic grammar only: repeated word (offers anak-anak first), space before
  punctuation, double space, with Indonesian messages
- Review > Language, status bar, Set Proofing Language, File > Options >
  Proofing list Indonesian; name translated in every catalog
- tests use a hand-written synthetic dictionary; engine end-to-end test

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 03:42:52 -04:00
c14e974285 proof: Czech (cs-CZ) spelling from the installed Hunspell dictionary (storytold/wordcraft#1048)
Same pattern as Portuguese (storytold/wordcraft#797) and Polish (storytold/wordcraft#346): the cs_CZ
dictionary is read from the system at runtime (e.g. Debian/Ubuntu
hunspell-cs), nothing is bundled. Suggestions use the dictionary's own
MAP/REP/TRY/KEY; grammar stays generic (repeated word, spacing), with
digit ratios like "3 : 1" exempt. The Czech interface now writes new
documents (and opened .txt/.md) in cs-CZ.

Engine fixes the Czech dictionary needs, also used by pl/pt:
- PFX/SFX header lines may carry a trailing comment (cs_CZ has
  "SFX y Y 80 #…"); rule lines are left literal;
- NOSUGGEST stems are accepted when typed but never offered;
- KEY is parsed for neighbour-key suggestions.

Tests use a hand-written synthetic Czech dictionary only; an
end-to-end engine test mirrors the Portuguese one.

Co-authored-by: Losminek <301412321+Losminek@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-15 00:52:43 -04:00