- proof: pass PoisonError::into_inner instead of an identity closure (S1612).
- layout tests: replace glob super::* imports with explicit names (S2208).
- packaging/macos/import-cert.sh: use [[ for the status test (S7688).
Wall-clock bounds of 1-2 s failed on loaded runners. Each bound is now 10 s, at least 10x the
fixed code's time and well under the old code's on the same input (measured by running the
pre-fix code):
- self-continuing rules: 40,000 rules (was 16,000); ~1 ms now, old checker 14 s at 16,000.
- many strips per suffix: 200,000 rules (was 30,000); 0.4 s now, old index 3 s at 30,000 and
11 min at 300,000.
- forms of self-continuing rules: the MAX_FORMS count catches the old code (9 million forms);
the time bound is a backstop.
- overlong Turkish words: ~20 ms now; without the caps it ran past 5 minutes.
- capped Hunspell reads: no clock; the "not a regular file" error is the proof (the old reader
never returned from /dev/zero).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CX1hMKX28764Lrx5moxs1o
Asked to confirm a non-UTF-8 Turkish dictionary still loads: before this PR
tr_TR was read with read_to_string, so a non-UTF-8 file never loaded; now it
goes through read_capped and decode, which didn't know ISO 8859-9 (Latin-5,
the encoding of older tr_TR dictionaries) and refused it as unsupported.
decode now reads `SET ISO8859-9`, and a test writes an ISO 8859-9 .aff/.dic
to disk and finds words with ğ ı ş İ ç ö ü, as stems and with suffixes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CX1hMKX28764Lrx5moxs1o
Audit finding 20.
- tr::suggest_with refuses words over 64 letters (and words with spaces), like the other
languages' suggesters (40 there; Turkish agglutinated words run past 40). Each candidate edit
copies the whole word, so a long word cost O(n^2) per suggestion request on the UI thread.
- TrDict::contains returns false past MAX_WORD_BYTES (120, the affix engine's bound) instead of
collecting the char indices of the whole word.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Audit findings 10 (storytold/wordcraft#3657) and 15 (storytold/wordcraft#3692).
- ru::hyphen::points: the cuts come in increasing order, so a repeat can only be the last one;
compare with `out.last()` instead of `out.contains` per syllable (O(V^2) on a long Cyrillic
word, during layout).
- hyphenate_word_in: walk the sorted break points alongside the chars instead of searching the
point list for every char. Same output.
Test: russian_hyphenation_and_hyphenated_output_are_linear (8x the letters costs ~8x; the old
code measured 31x and 30x).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Audit finding 4, and the rest of storytold/wordcraft#3661.
- dict.rs: `Dictionary::from_bytes` inflates with `decompress_to_vec_with_limit` (64 MiB; the
bundled en-US one is ~1 MB) and caps the words it keeps (bytes and count): front coding lets a
2-byte line spell out a 255-byte word, so the inflated size alone doesn't bound storage.
- hunspell.rs: one `read_capped` reader — regular files only, size checked on the open handle and
the read itself bounded by `take(MAX_FILE + 1)`, so no TOCTOU between metadata and read and no
endless read from a FIFO or device.
- tr: the Turkish loader reads through `read_capped` and decodes with `hunspell::decode` (it
read both files whole, with no cap at all).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Part of storytold/wordcraft#3613 (audit findings 2 and 6).
- Checker: `suffixed` and `form_has_flag` OR the flags of a group's matching rules into one
mask and recurse once, not once per rule. Same answer (the walk is a disjunction over flags);
16k self-continuing empty-append rules: 6 s -> <1 ms per check (debug).
- Finding 2: `forms` expands each (form, continuation) pair once, walks continuation rules through
a per-flag rule list, and stops at MAX_FORMS (4096) forms / MAX_FORMS_WORK rule visits, so a
dictionary of self-continuing rules can't allocate R^2 strings per stem.
- Finding 6: the suffix index finds a rule's (append, strip) group by hash, not by scanning the
append's groups: building with many strips per append is linear.
Not done here (rest of storytold/wordcraft#3613): a work budget for distinct-strip chains in the checker, rule caps
in parse_aff, proofing off the UI thread.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It came in with the first proofing commit (bcbe09df8) and has never had a caller; `#[allow(dead_code)]` kept the warning quiet. Because it is `pub(crate)`, nothing outside wordcraft-proof can reach it.
SonarCloud's analysis of main at b4da59f0c lists 23 closures such as `|v| v.as_slice()` that hand their argument straight to one method. This passes the method itself: `Vec::as_slice`, `str::to_ascii_lowercase`, `f64::total_cmp` and the like, all forms the repo already uses elsewhere. Behaviour is unchanged.
Five stay. The four in crates/proof/src/es/mod.rs sit in code that storytold/wordcraft#4191 replaces, and its replacement already uses `PoisonError::into_inner`. In crates/engine/src/io/safe_save.rs the path would be `<dyn std::error::Error + Send + Sync>::is::<Guided>`, which reads worse than `|inner| inner.is::<Guided>()`.
A dictionary is read outside the lock. If the dictionaries folder changed
while it was being read (File › Options › Proofing), the reload emptied
the slot first and the late load then filled it with the old folder's
dictionary. It stayed until the next reload. Every language's copy of
the loader had this window; the shared slot now closes it in one place.
The slot counts reloads. A load that started before a reload still
answers the check that asked for it, but it isn't kept: the next check
reads the new folder.
Found by CodeRabbit on storytold/wordcraft#4191. Tests: a_load_from_before_a_reload_is_not_kept;
not counting reloads fails it. Proof: 288 tests.
The suggestion rounds don't run in rising cost: a MAP diacritic drop (11)
comes before a REP confusion (10, 9 for Russian). The engine checked each
candidate once, at the first cost it was offered, so the same word found
again more cheaply kept the dearer cost and could rank one place too low.
Czech's engine always did this, and the merged engine took it over.
The candidate set is now a map to the cheapest cost offered so far, and
an offer that is no cheaper is skipped as before.
Found by CodeRabbit on storytold/wordcraft#4191. Tests: a_cheaper_later_offer_wins; skipping
every repeat offer again fails it. Proof: 288 tests.
Czech, Croatian, Indonesian, Italian, Polish, Portuguese and Spanish began
their spelling checks with the same thirteen lines, byte for byte: no
dictionary means no marks, single letters and words without letters pass,
words with digits and short words in capitals pass when the File › Options
› Proofing choices say so, and words in another script pass. Each also had
its own copy of the Latin-script test. These were the lines storytold/wordcraft#3966 had to fix
in six copies one by one, and where the Russian and Turkish copies stayed
wrong until the first commit here.
spelling.rs has them once (`word_to_check`), and each language goes on
from there with its own rules (Polish apostrophe endings, Italian elisions,
Indonesian clitics, Uruguayan `vos` forms).
Tests: words_needing_no_lookup. Ignoring the uppercase option there fails
it and all seven languages' proofing-options tests. Proof: 286 tests.
lib.rs named every dictionary language in nine places: reloading for a new
folder, the installed list, load errors, the missing-dictionary hint, the
loaded check, the spelling message, spelling, suggestions and grammar. The
lists had drifted. dictionary_load_error had no Russian or Czech entry, so a
Russian or Czech dictionary that failed to load showed no reason in File ›
Options › Proofing, though the status bar, which reads another list, said
why.
LANGUAGES holds one row per language: its dictionary slot, its spelling
message, and its spelling, suggestion and grammar functions. The nine
functions read the row, so a language now joins with its module, its
ProofLang and one row. The per-language reload_dictionary and
installed_path wrappers had no callers left, inside the crate or out, and
are gone.
Tests: every_dictionary_language_reports_its_load_error, which fails for
Russian on the old code, and the_language_table_covers_the_proofed_languages
(each ProofLang::PROOFED language but English has exactly one row). Proof
285, engine 927 and layout 501 (lib).
Czech's dictionary-driven ranking (case, MAP, REP, missing spaces, KEY,
TRY edits, a stem scan) had five older copies. Polish, Portuguese,
Croatian, Italian and Russian each kept one: 300 to 400 lines of the same
rounds with a few lines of language knowledge mixed in. Fixes reached some
copies only. NOSUGGEST (storytold/wordcraft#1048) and the first-difference tie-break (storytold/wordcraft#797)
never got to all of them, and none had the KEY round or the bound on TRY
letters.
affix_suggest.rs is that ranking once. A language's `Tuning` holds what is
its own: the short words that run into their neighbours (Polish,
Portuguese, Croatian, Italian), Russian's split rule, the letter groups and
confusions its dictionaries lack (Croatian, Russian), letters to try for a
dictionary without a TRY line, and cost hooks for Polish final devoicing
and Russian е/ё. Czech, Indonesian and Spanish use the dictionary's tables
alone. Turkish keeps its own suggester, which reads another dictionary type
with rounds of its own.
The language tables add to the dictionary's instead of replacing them, and
TRY letters are tried in the dictionary's order, commonest first, as the
copies did (the Czech engine sorted them).
Old and new compared on the test dictionaries, with every one-letter typo
of each word plus the tests' misspellings, top eight suggestions: Czech,
Croatian, Italian and Portuguese give the same for all 7,903 inputs.
Polish differs for 44 of 3,244 and Russian for 33 of 2,346, all orderings
among candidates of equal cost, from the tie-break they now share (`ie` →
`nie` before `i`).
New tests pin the moved knowledge: the tuning mechanism, Croatian's groups
without MAP lines, and Russian's costs and split rule. Turning each knob
off fails one. Proof: 283 tests.
user_dictionary_persists_through_a_file saves the whole user dictionary to
a file and loads it back. Run beside
user_dictionary_and_ignore_all_match_both_normalization_forms, it could
catch a word that test had added and bring it back after the test removed
it, so the other test's last check failed now and then (once in about two
dozen full runs here). It now holds DICTIONARY_TESTS, as the other test
does.
Nine languages each kept their own copy of the same code: the installed
dictionary behind a lock, read on first use and logged, replaced by
install_dictionary, forgotten by reload_dictionary, and its file reported
by installed_path. The copies had drifted already. Every one documented that
a dictionary given with install_dictionary has no file and isn't reported,
but only Russian and Czech did that (storytold/wordcraft#3252); the other seven reported the
disk file anyway.
`DictSlot` (proof/src/slot.rs) is that code once. It is generic over the
dictionary type, so Turkish's `TrDict` fits, and keeps the dictionary and
its in-memory origin under one lock as storytold/wordcraft#3252 made it. Each language keeps
its functions as one-line wrappers over its slot, so no caller changes, and
the log lines are the same.
installed_path now follows the documented rule in every language. Nothing
outside tests installs a dictionary in memory, but tests do, concurrently,
so the dictionaries-folder tests in proof and engine could race on the
reported path of a language another test installs. They now accept the
folder's file or none there, and check that the folder is searched first
with hunspell::find, which no install affects.
Tests: proof 279, engine 927, layout 501 (lib); slot.rs has a test of its
own.
storytold/wordcraft#1048 (c14e97428) taught affix.rs about NOSUGGEST, the dictionary flag for
words that are correct but never offered (vulgarisms, rare forms), and
patched the Polish and Portuguese copies of the suggestion engine; Czech's
started with it. The Croatian, Italian and Russian copies never got it: they
test candidates with `contains`, so a suggestion could be a word the
dictionary says not to offer. They now use `contains_suggestible` and drop a
suggestion with any part `can_suggest` refuses, as the others do.
Tests: Croatian and Italian run the shared assert_nosuggest_is_honoured.
Russian gets its own, since its MAP and REP tables are built in and the
shared Latin-script check doesn't reach them. Each fails on the old code.
storytold/wordcraft#3966 made Ignore words in UPPERCASE and Ignore words that contain numbers
reach six languages' own checks, one copy at a time, and Russian and Turkish
were left out. lib.rs still called `ru::is_correct(word)` and
`tr::is_correct(word)`, which accept words with digits (both) and short words
in capitals (Russian) whatever the options say, so turning either option off
changed nothing in those languages.
Both now have `is_correct_with_options`, like the others. Russian's rule for
a part in capitals (`ИИ-консультант`) follows the option too, as Indonesian's
does, and its digits rule keeps counting superscripts (`м²`). Turkish had no
test dictionary, so its two dictionary tests returned early wherever none was
installed, CI included; it has a synthetic one now, also in the
dictionaries-folder test.
Tests: proofing_options_reach_the_russian_check and
proofing_options_reach_the_turkish_check. Calling the old checks from lib.rs
fails each, and so does ungating the Russian part rule or the Turkish hyphen
recursion.
The spelling check found the start of the white-space-delimited token around
a word one byte after the last white-space character. After a no-break space
(two bytes in UTF-8) or an ideographic space (three bytes) that offset is
inside the character, so the token could not be read and was never classified
as a web or e-mail address. With Ignore web and file addresses on, the words
of such an address were still marked as misspelled.
The token now starts after the whole white-space character. A proofing test
covers a web address, a www. address and an e-mail address after the two
spaces, with the option on and off.
* chore: resolve SonarCloud maintainability issues
* fix: restore compiling refactors, revert behaviour changes, run cargo fmt
- fontdb::norm: keep the closure for filter (char::is_alphanumeric takes
self, not &self, so it does not fit Iterator::filter).
- story.rs: write_mark_rpr needs &mut self (rev_attrs, rpr_change).
- Revert the viewport meta change and the return 0 added to
workspace_version/sha256, which changed behaviour or masked failures.
- cargo fmt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Import Document in odt page_parts after dropping glob re-export
* fix: apply_kind_properties binds ContentControl; cell_ref keeps char closures
* fix(engine): pass case helpers by value (clippy needless borrows)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan Taylor <dylan@craftmaker.dev>
Each of these languages accepted every word in capitals of up to six letters and every word
with a digit on its own, so turning off Ignore words in UPPERCASE or Ignore words that contain
numbers had no effect on them for short words. The check now takes the options, as the Spanish
one does; the direct is_correct/is_correct_with keep the defaults.
Co-authored-by: Federico Olivares <FedeOlivares@users.noreply.github.com>
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
Maintainer review on storytold/wordcraft#3562: opening_marks reverse-scans the characters before a
question or exclamation mark, and ends_clause asked in_address for each one,
which scanned every address range, so a paragraph of thousands of addresses
followed by a question took quadratic time (about 99 KB: hundreds of ms).
address_at() finds the range a byte is in with a binary search over the
sorted, disjoint ranges, and also gives ends_clause the range's end without
another scan. Behaviour is unchanged (the URL and punctuation regressions
still pass).
- address_lookup_finds_the_range_a_byte_is_in: boundaries of the lookup
- many_addresses_before_a_question_stay_linear: 99 KB of addresses before a
question or exclamation (the review's reproducer and two variants), checked
for the right issue range and for taking under 75 ms (about 1 to 5 ms now,
about 300 ms with a scan per character)
Upstream merged storytold/wordcraft#3177 (dictionary hints for every Hunspell language) and storytold/wordcraft#3480.
Resolve the conflicts in crates/proof/src/lib.rs by taking storytold/wordcraft#3177's
dictionary_loaded/dictionary_needed structure and adding the four Spanish
varieties to it, and in docs/spelling-dictionaries.md keeping both sides.
Maintainer review on storytold/wordcraft#3562: spelling looks words up composed (NFC), but
add_word, remove_word, ignore_word, custom_word (so loading the dictionary
file) and save_user_dictionary's removed list kept the raw lower-cased
spelling. A word added or ignored decomposed (`zqprue\u{301}ba`) therefore
stayed flagged.
- dictionary_key(): lower case, then composed; every write uses it, and
in_user_dictionary() looks up composed, so either form adds, ignores,
removes and finds the same entry; NFD entries in an existing file load
and save composed
- English's known_core() reuses in_user_dictionary() instead of a copy of it
- issue byte ranges still cover the text as typed
- regressions: proof (add, Ignore All, remove, save/load with an NFD entry,
both forms rechecked) and engine (review.addToDictionary and
review.ignoreAll with the issue's NFD text)
- es/mod.rs: one variety() match gives the dictionary slot, log name and file
names (variety_name() folded in); load_installed() destructures it
- es/grammar.rs: the sentence-start rule for the missing ¿/¡ is one named
function, ends_clause(), replacing is_boundary() and three closures
rebuilt per mark; in_address() is shared; repeated_words() builds its
helpers once instead of per word pair
- es/suggest.rs reuses the Czech ranking's MAX_LETTERS instead of a copy
Same tests pass unchanged; a temporary old-vs-new differential proptest
(20,000+ generated inputs with URLs and punctuation) found identical output.
Maintainer review on storytold/wordcraft#3562:
- words() keeps combining marks with the word they follow, so a word typed
decomposed (NFD, `tambie\u{301}n`) is one word with its own byte range;
is_correct_in_with and suggest_in look it up composed (NFC), so it is
checked like `también` in every language (unicode-normalization, MIT/Apache)
- the Spanish missing-¿/¡ rule skips marks in web and e-mail addresses
(`Ver https://example.com/?`), and punctuation inside an address or a word
(`example.com`, `https:`, `10:30`) no longer starts the sentence the
opening mark is offered for; a mark ending an address still ends a sentence
- Spanish repeated-word check matches composed and decomposed spellings
- regressions: check_text with the Spanish fixture and NFC/NFD equivalents
(acceptance and ranges), words() and composed(), URLs ending in /? beside
ordinary questions
CodeRabbit review: Session::region_spanish takes the ISO 639-2 `spa` as
Spanish, so a `spa-MX` region writes es-MX documents.
CodeRabbit review on storytold/wordcraft#3562:
- with Ignore words in UPPERCASE off, short words in capitals are checked
against the Spanish dictionary instead of accepted as acronyms; likewise
words with digits with Ignore words that contain numbers off. Direct
es::is_correct/is_correct_with keep the defaults.
- Latin American Spanish (which es-AR resolves to) also looks for es_AR and
es-AR, after the neutral es_US and es_MX, so Argentine text is checked when
only that dictionary is installed
- docs: the exact candidate order per variety in localization-parity.md, and
es_AR in the spelling-dictionaries table
- the Spain variant's doc lists Uruguay among the regions it doesn't cover
Resolve conflicts with German hyphenation (ProofLang::De), Russian and Czech
in installed_dictionaries(), and storytold/wordcraft#2164's U+FE0F flag fix: affix.rs takes
main's fix and drops this branch's duplicate. The Danish, Dutch, Romanian and
Slovenian catalogs gain the Spanish (Latin America) and (Uruguay) names; the
language menu tests get room for the four Spanish entries.
Spanish text was never checked: ProofLang had no Spanish variant, so every
es-* run fell through to Other (no marks). And the RLA-ES dictionaries
LibreOffice ships were refused by the affix engine for their `☎️` flag
(☎ + variation selector U+FE0F), which Hunspell reads as ☎.
- ProofLang::Es (Spain, plain es, spa), EsMx (es-MX), Es419 (es-419 and
the other countries of the Americas) and EsUy (es-UY), chosen by region
like the English dialects
- each variety reads its own installed Hunspell dictionary with fallbacks
(es_ES; es_MX; es_419/es_US/es_MX; es_UY/es_AR); nothing bundled
- variation selectors in affix flag fields belong to the flag before them
- Uruguayan vos forms accepted when the dictionary has the infinitive:
imperatives (vení, andá), with pronouns (decime, decímelo), present
(tenés); a vos form typed without its accent gets it back first
- suggestions use the dictionary-driven Czech ranking (prueva → prueba,
corector → corrector, tambien → también)
- grammar: repeated word, spacing, missing opening ¿/¡, space after an
opening mark, with Spanish messages
- Review › Language, status bar, Language dialog and File › Options ›
Proofing list the four varieties; the two new names translated in every
catalog; a Spanish interface writes new documents in the region's Spanish
- the dictionaries-folder engine test holds the Italian and Spanish
fixtures, so parallel proofing tests no longer lose their dictionaries
- tests use hand-written synthetic dictionaries; engine end-to-end tests
crates/proof/src/lang.rs: main's English-dialect helpers kept; code() doc lists `de`.
crates/proof/src/lib.rs: hostile-input test loops over main's languages plus De.
crates/ui-egui/src/options.rs: language_name maps EnGb/EnCa and De to no dictionary name.
ROADMAP.md: main's status lines; both progress-log and revision rows kept.
docs/gaps.md: main's status line; gap 8 row is main's with German hyphenation added; revision rows unioned.
docs/layout-parity.md: main's status line, Tabs and Drop caps rows; the PR's Hyphenation row; revision rows unioned.
docs/localization-parity.md: main's status line; revision rows unioned.
VoiceLessQ
Dylan M. Taylor
Martin BallandClaude Opus 5.5
skylar
aiongg
aiongg
Federico Olivares
dotCooCoo
NetterSilverandClaude Haiku 5.5