mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-18 16:24:38 +00:00
Merge pull request #557 from totoz64/feat/french-language-pack-proofing
Proofing: add French pack rules and regional typography
This commit is contained in:
15 files changed
+2297
-8
No files matched your search
+49
-5
@@ -16,6 +16,7 @@ craft-proof = { git = "https://github.com/storytold/craft-libs", rev = "<commit>
|
||||
| English (en-US, en-CA, en-AU, en-GB and the British-spelling Englishes such as en-NZ) | SCOWL 2026 Hunspell dictionary per English + `en-extra.txt`, dialect policy (`en-variants.txt`); a dictionaries-folder English first; without SCOWL the installed Hunspell English, else the built-in Moby list ([below](#english-is-always-checked)) | yes | Moby list, our patterns, heuristic |
|
||||
| Polish, Portuguese, Italian | the installed Hunspell dictionary | yes | TeX patterns (embedded, or a Language pack) |
|
||||
| Croatian, Turkish, Russian, Czech, Indonesian, Spanish (es, es-MX, es-419, es-UY) | the installed Hunspell dictionary (Russian also `ru-extra.txt`) | yes | Turkish and Russian rules; English fallback otherwise |
|
||||
| French (fr-FR, fr-CA and other French pack locales) | the installed Language pack or Hunspell dictionary; dictionary-driven elisions and accent-oriented suggestions | conservative typography and basic checks, with Quebec punctuation spacing in fr-CA | the Language pack (generic fr fallback) |
|
||||
| German | — | — | TeX patterns (embedded, or a Language pack) |
|
||||
| Bulgarian | — | — | syllable rules |
|
||||
|
||||
@@ -41,6 +42,12 @@ application data (`%LOCALAPPDATA%\WordCraft\dictionaries` and `%APPDATA%\WordCra
|
||||
`~/Library/Application Support/WordCraft/dictionaries`, `$XDG_DATA_HOME/wordcraft/dictionaries`),
|
||||
then the usual Hunspell and LibreOffice locations. None on the web.
|
||||
|
||||
French remains a `ProofLang::Pack` locale: use its full `tag()` for settings and issue identity.
|
||||
The shared pack loader keeps regional dictionary overrides, fallback to `fr`, reloads and its
|
||||
memory budget; no separate French dictionary slot or embedded French patterns are added.
|
||||
Only French packs receive the French spelling tuning and typography; other pack languages
|
||||
keep their existing behavior. French agreement and conjugation are outside these checks.
|
||||
|
||||
State is process-wide: `add_word`, `ignore_word`, `set_dictionaries_folder`,
|
||||
`load_user_dictionary`/`save_user_dictionary`. The crate reads no document model; callers pass
|
||||
plain text and language spans.
|
||||
@@ -82,19 +89,22 @@ pub fn load_errors() -> Vec<(String, String)>; // + (locale or folder,
|
||||
pub fn embedded_locales() -> &'static [&'static str]; // + locales with embedded upstream data, not our supplements ("en-us", "en-gb", "en-ca", "en-au", "pl", "pt", "it", "de" by default; the four English ones with CRAFT_PROOF_EMBED=en)
|
||||
|
||||
// craft_proof (crate root)
|
||||
pub enum ProofLang { …, Pack(PackLocale), Other } // + Pack: any language without tools of its own (nl, fr, sr-Latn…)
|
||||
pub enum ProofLang { …, Pack(PackLocale), Other } // + Pack: a language without a dedicated built-in variant (nl, fr, sr-Latn…)
|
||||
pub struct PackLocale; // + Copy; PackLocale::new(tag) -> Option<Self>, .as_str() "nl-nl", .primary() "nl", Display
|
||||
impl ProofLang { pub fn pack_locale(self) -> Option<PackLocale>; } // +
|
||||
pub fn pack_dictionary_file(lang: ProofLang) -> Option<PathBuf>; // + the .dic a Pack language reads now; None = unchecked
|
||||
```
|
||||
|
||||
`ProofLang::from_tag` now gives `Pack(locale)` for every valid language tag it has no tools for
|
||||
`ProofLang::from_tag` now gives `Pack(locale)` for every valid language tag without a dedicated built-in variant
|
||||
(`nl-NL` → `Pack("nl-nl")`, `fr-FR`, `ja-JP`, `sr-Latn`…); `Other` is left for `x-none`, `und`,
|
||||
`zxx`, `mul`, `mis`, `i-…` and malformed tags. A `Pack` language is spell-checked and given
|
||||
suggestions from its Hunspell dictionary (dictionaries folder → installed pack → system Hunspell)
|
||||
through the shared affix engine with default tuning, and hyphenated from its pack's
|
||||
`hyphenation.pat`; it has no grammar rules and the generic English spelling message. Without a
|
||||
dictionary or pack it behaves exactly as `Other` did (no marks, English hyphenation fallback).
|
||||
through the shared affix engine with default tuning (French tuning for `fr` packs), and
|
||||
hyphenated from its pack's `hyphenation.pat`. Most pack languages use no dedicated grammar
|
||||
rules; regional variants can reuse a built-in language's rules. French packs use the French
|
||||
spelling message and the conservative French grammar and typography described above. Without a dictionary or pack,
|
||||
spelling has no marks and hyphenation falls back to English; French grammar and typography
|
||||
still apply independently of a spelling dictionary.
|
||||
`has_dictionary(tag)` / `ProofLang::is_proofed()` say whether a `Pack` language has a dictionary
|
||||
now (read on first use, cached). Loaded `Pack` dictionaries share a 256 MB budget (estimated from their files); the least recently used goes first, but never one used in the paragraph being checked, and languages with no dictionary take no room. Downstream `match`es on `ProofLang` need a `Pack(_)` arm; `code()` stays `&'static str`
|
||||
and is `""` for it, as for `Other`: anything that keys, saves or reports a language (settings, caches, JSON/MCP output) uses `tag()`, which is `code()` for the built-ins and the locale (`nl`, `fr-fr`) for a pack language.
|
||||
@@ -221,6 +231,40 @@ CI: `.github/workflows/proof.yml`.
|
||||
|
||||
storytold/craft-libs#30/#31 by brzuchal (Michał Marcin Brzuchalski) started the shared crate.
|
||||
|
||||
### French adaptation validation
|
||||
|
||||
The French rules and original synthetic fixtures are adapted from
|
||||
[totoz64’s WordCraft contribution #3185](https://github.com/storytold/wordcraft/pull/3185).
|
||||
They use the shared affix reader and Language pack cache; WCAFF1 remains deliberately rejected.
|
||||
The historical text fixtures retain their original expected data. Three observations for
|
||||
Indonesian `bermainkanlah` are explicitly updated in the compatibility test because the
|
||||
shared reader already supports that prefix plus two suffixes. This is a finite regression
|
||||
check, not a universal compatibility claim.
|
||||
|
||||
The ordinary tests cover French packs, regional overrides and reloads, options, user words,
|
||||
NFC/NFD lookup with original ranges, suggestions and pack hyphenation. The opt-in real French
|
||||
dictionary gate fails when its input is missing; it never silently skips its assertions:
|
||||
|
||||
```sh
|
||||
WORDCRAFT_DICTIONARIES=/absolute/isolated/french-dictionary cargo test -p craft-proof --test french_dictionary -- --ignored
|
||||
```
|
||||
|
||||
For a reproducible real-dictionary run, use Dicollecte 7.7 from LibreOffice/dictionaries
|
||||
revision `67032ee957631908098a72edf48dc81cac27e369`:
|
||||
[fr.aff](https://raw.githubusercontent.com/LibreOffice/dictionaries/67032ee957631908098a72edf48dc81cac27e369/fr_FR/dictionaries/fr.aff) and
|
||||
[fr.dic](https://raw.githubusercontent.com/LibreOffice/dictionaries/67032ee957631908098a72edf48dc81cac27e369/fr_FR/dictionaries/fr.dic).
|
||||
Verify SHA-256 before invoking the gate (the test checks location and behavior, not hashes):
|
||||
|
||||
| File | SHA-256 |
|
||||
|---|---|
|
||||
| fr.aff | `c176610cd5dc4846806a65ddd029f422d87978bf58f224aa44222662a16a2de5` |
|
||||
| fr.dic | `b78a868e31dd6e373b6c3217969afb898a9acde828a5e7ef97308da42218c88c` |
|
||||
|
||||
Use a dedicated directory containing only the intended `fr.aff`/`fr.dic` pair (or explicit
|
||||
regional pairs for both regions). The gate requires more than 50,000 dictionary entries and
|
||||
checks the actual source folder, spelling, accent/elision suggestions, regional dispatch,
|
||||
combining-mark ranges and hyphenated compounds. No real dictionary is committed here.
|
||||
|
||||
## Owners
|
||||
|
||||
WordCraft maintainers: echelon, dylanmtaylor, aiongg.
|
||||
|
||||
@@ -2227,6 +2227,38 @@ impl AffixDict {
|
||||
self.check_except(w, self.excluded(true))
|
||||
}
|
||||
|
||||
/// Validate a collected suggestion in its displayed case, before OCONV. Only identity,
|
||||
/// first-letter and uppercase variants are allowed (including uppercase that keeps sharp s).
|
||||
/// An exact target excluded from suggestions cannot be rescued by a lowercase fallback.
|
||||
/// This preserves the checker's homonym, compound and CHECKSHARPS rules without scanning
|
||||
/// stems or changing which words the spelling checker accepts.
|
||||
pub(crate) fn can_suggest_cased(&self, original: &str, changed: &str) -> bool {
|
||||
if original.len() > MAX_INPUT_BYTES || changed.len() > MAX_INPUT_BYTES {
|
||||
return false;
|
||||
}
|
||||
if changed != original
|
||||
&& changed != capitalize(original)
|
||||
&& changed != original.to_uppercase()
|
||||
&& changed != original.split('ß').map(str::to_uppercase).collect::<Vec<_>>().join("ß")
|
||||
{
|
||||
return false;
|
||||
}
|
||||
let target = self.normalize(changed);
|
||||
if self.is_forbidden(&target) {
|
||||
return false;
|
||||
}
|
||||
// The exact simple-word path includes affixes; asking with no exclusions detects a
|
||||
// target that may carry NOSUGGEST, SUBSTANDARD or FORBIDDENWORD. An eligible exact
|
||||
// compound derivation still counts, but a lowercase fallback must not rescue it. Spend
|
||||
// the same shared suggestion budget as the normal checker, and abstain if it runs out.
|
||||
match self.simple_word_budgeted(&target, Excluded::default(), true) {
|
||||
None => return false,
|
||||
Some(true) if !self.contains_except(&target, self.excluded(true)) => return false,
|
||||
_ => {}
|
||||
}
|
||||
self.can_suggest(changed)
|
||||
}
|
||||
|
||||
/// Whether `w` is a word only through a `KEEPCASE` stem, so it is correct in this letter case
|
||||
/// alone (`km` from `km/K`, which `Km` and `KM` are not).
|
||||
pub(crate) fn case_locked(&self, w: &str) -> bool {
|
||||
@@ -3313,6 +3345,41 @@ pub(crate) mod tests {
|
||||
assert!(d.check("ASSASSASSASSASS") && !d.check(&"SS".repeat(60)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn displayed_casing_rejects_exact_exclusions_before_case_fallback() {
|
||||
for flag in ["N", "F", "S"] {
|
||||
let aff = "SET UTF-8\nNOSUGGEST N\nFORBIDDENWORD F\nSUBSTANDARD S\nSFX A Y 1\nSFX A 0 s .\nSFX B Y 1\nSFX B 0 S .\n";
|
||||
let dic = format!("5\nnormal/A\nNORMAL/{flag}B\nmixCase\npour\nkm/K\n");
|
||||
let d = AffixDict::from_hunspell(&format!("{aff}KEEPCASE K\n"), &dic, &[]).unwrap();
|
||||
for (original, changed) in [("normal", "NORMAL"), ("normals", "NORMALS")] {
|
||||
assert!(!d.can_suggest_cased(changed, changed), "{flag}: {changed}");
|
||||
assert!(!d.can_suggest_cased(original, changed), "{flag}: {original} → {changed}");
|
||||
}
|
||||
assert!(d.can_suggest_cased("normal", "Normal"));
|
||||
assert!(d.can_suggest_cased("mixCase", "MIXCASE"));
|
||||
assert!(d.can_suggest_cased("km", "km") && !d.can_suggest_cased("km", "KM"));
|
||||
assert!(!d.can_suggest_cased("normal", "pour"), "no arbitrary replacement");
|
||||
let back = AffixDict::from_bytes(&d.to_bytes().unwrap()).unwrap();
|
||||
assert!(!back.can_suggest_cased("normal", "NORMAL"));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn displayed_casing_uses_normalization_and_native_homonym_policy() {
|
||||
let aff = "SET UTF-8\nNOSUGGEST N\nSUBSTANDARD S\nIGNORE ()\nICONV 1\nICONV Q NORMAL\n";
|
||||
for flag in ["N", "S"] {
|
||||
let d = AffixDict::from_hunspell(aff, &format!("3\nnormal\nNORMAL/{flag}\nNORMAL\n"), &[]).unwrap();
|
||||
// Native homonyms carry exclusions: a second entry does not erase NOSUGGEST.
|
||||
assert!(!d.contains_suggestible("NORMAL"));
|
||||
assert!(!d.can_suggest_cased("(normal)", "(NORMAL)"));
|
||||
assert!(!d.can_suggest_cased("Q", "Q"));
|
||||
}
|
||||
let plain = AffixDict::from_hunspell(aff, "3\nnormal\nNORMAL\nNORMAL\n", &[]).unwrap();
|
||||
assert!(plain.can_suggest_cased("normal", "NORMAL"));
|
||||
assert!(plain.can_suggest_cased("Q", "Q"));
|
||||
assert!(!plain.can_suggest_cased(&"x".repeat(MAX_INPUT_BYTES + 1), "NORMAL"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn oconv_converts_suggestions_only() {
|
||||
// Answers as Hunspell 1.7 gives them on the same files: `kbt` → `k’t`.
|
||||
|
||||
@@ -248,7 +248,8 @@ impl Collector<'_> {
|
||||
|
||||
/// Keep a suggestion known to be correct, unless the dictionary says never to offer it.
|
||||
fn keep(&mut self, text: String, cost: u32) {
|
||||
if !text.split(' ').all(|part| self.d.can_suggest(part)) {
|
||||
// Reject exact excluded spellings at admission, before later candidates spend the budget.
|
||||
if !text.split(' ').all(|part| self.d.can_suggest_cased(part, part)) {
|
||||
return;
|
||||
}
|
||||
self.seq += 1;
|
||||
@@ -447,6 +448,14 @@ pub(crate) fn suggest_in(d: &AffixDict, word: &str, max: usize, t: &Tuning) -> V
|
||||
_ => vec![text.clone()],
|
||||
};
|
||||
for t in recased {
|
||||
let eligible = |candidate: &str| {
|
||||
let original: Vec<_> = text.split(' ').collect();
|
||||
let changed: Vec<_> = candidate.split(' ').collect();
|
||||
original.len() == changed.len() && original.iter().zip(changed).all(|(a, b)| d.can_suggest_cased(a, b))
|
||||
};
|
||||
// Collector::keep already accepted text. Collection may have exhausted the shared
|
||||
// affix budget, so recheck only a changed case and otherwise retain that accepted form.
|
||||
let t = if t == text || eligible(&t) { t } else { text.clone() };
|
||||
// Shown as the dictionary's OCONV table writes it (`'` → `’` in French).
|
||||
let t = d.output(&t).into_owned();
|
||||
if t != word && !out.contains(&t) {
|
||||
@@ -795,6 +804,38 @@ mod tests {
|
||||
assert_eq!(suggest_in(&d, "Piesx", 4, &Tuning::DEFAULT).first().map(String::as_str), Some("Pies"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recasing_keeps_valid_original_when_exact_target_is_excluded() {
|
||||
for flag in ["N", "F", "S"] {
|
||||
let aff = "SET UTF-8\nNOSUGGEST N\nFORBIDDENWORD F\nSUBSTANDARD S\nTRY normal\n";
|
||||
let d = AffixDict::from_hunspell(aff, &format!("2\nnormal\nNORMAL/{flag}\n"), &[]).unwrap();
|
||||
let got = suggest_in(&d, "NORMALL", 4, &Tuning::DEFAULT);
|
||||
assert!(got.iter().any(|w| w == "normal"), "{flag}: {got:?}");
|
||||
assert!(!got.iter().any(|w| w == "NORMAL"), "{flag}: {got:?}");
|
||||
let split = AffixDict::from_hunspell(aff, &format!("3\nnormal\nNORMAL/{flag}\npour\n"), &[]).unwrap();
|
||||
let got = suggest_in(&split, "NORMALPOUR", 4, &Tuning::DEFAULT);
|
||||
assert!(got.iter().any(|w| w == "normal pour"), "{flag}: {got:?}");
|
||||
assert!(!got.iter().any(|w| w == "NORMAL POUR"), "{flag}: {got:?}");
|
||||
}
|
||||
let d = AffixDict::from_hunspell("NOSUGGEST N\n", "3\nnormal\nNORMAL/N\npour\n", &[]).unwrap();
|
||||
assert!(!d.can_suggest_cased("normal", "NORMAL"));
|
||||
assert!(!d.can_suggest_cased("normal", "pour"));
|
||||
}
|
||||
|
||||
/// An excluded simple entry does not exclude an eligible exact COMPOUNDRULE derivation.
|
||||
#[test]
|
||||
fn rep_keeps_eligible_exact_compound_beside_an_excluded_simple_entry() {
|
||||
for flag in ["N", "S", "F"] {
|
||||
let aff = "SET UTF-8\nNOSUGGEST N\nSUBSTANDARD S\nFORBIDDENWORD F\nCOMPOUNDRULE ab\nREP 1\nREP zz foobar\n";
|
||||
let d = AffixDict::from_hunspell(aff, &format!("3\nfoobar/{flag}\nfoo/a\nbar/b\n"), &[]).unwrap();
|
||||
for d in [&d, &AffixDict::from_bytes(&d.to_bytes().unwrap()).unwrap()] {
|
||||
assert_eq!(d.contains_suggestible("foobar"), flag != "F", "{flag}: exact compound eligibility");
|
||||
let got = suggest_in(d, "zz", 4, &Tuning::DEFAULT);
|
||||
assert_eq!(got.iter().any(|w| w == "foobar"), flag != "F", "{flag}: {got:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A language's tables come after its dictionary's, a TRY alphabet stands in for a missing TRY
|
||||
/// line, and a split rule decides which missing spaces are offered.
|
||||
#[test]
|
||||
@@ -874,4 +915,34 @@ mod tests {
|
||||
assert!(suggest_in(&d, "neer", 4, &english).contains(&"ne'er".to_string()));
|
||||
assert_eq!(suggest_in(&d, "neer", 4, &Tuning::DEFAULT).first().map(String::as_str), Some("ne'er"), "an elision");
|
||||
}
|
||||
/// A REP candidate is admitted before costly TRY checks consume the normal shared budget.
|
||||
/// Finalization must retain that accepted affixed word, including a conservative case fallback.
|
||||
#[test]
|
||||
fn accepted_affixed_suggestions_survive_exhausted_collection_budget() {
|
||||
let input = format!("{}x{}", "a".repeat(20), "a".repeat(19));
|
||||
let mut aff = String::from("SET UTF-8\nFLAG num\nTRY ");
|
||||
aff.extend(('a'..='z').chain((0x100..0x166).filter_map(char::from_u32)));
|
||||
aff.push_str(&format!("\nREP 1\nREP {input} kots\n"));
|
||||
// The bounded adversarial rule shape of the existing shared-budget regression.
|
||||
for f in 1..=1000u32 {
|
||||
let c = char::from_u32(0x4e00 + f).unwrap();
|
||||
aff.push_str(&format!("PFX {f} Y 1\nPFX {f} {c} 0 .\n"));
|
||||
}
|
||||
let conts: Vec<String> = (0..286).map(|k| (5000 + k * 7).to_string()).collect();
|
||||
for len in 1..=40 {
|
||||
for k in 0..50 {
|
||||
let f = 5000 + (len - 1) * 50 + k;
|
||||
aff.push_str(&format!("SFX {f} Y 1\nSFX {f} 0 {}/{} .\n", "a".repeat(len), conts.join(",")));
|
||||
}
|
||||
}
|
||||
aff.push_str("SFX 9000 Y 1\nSFX 9000 0 s .\n");
|
||||
let d = AffixDict::from_hunspell(&aff, "3\nb/1,5000\nab/2\nkot/9000\n", &[]).unwrap();
|
||||
assert!(!d.stems().any(|s| s == "kots") && d.can_suggest("kots"));
|
||||
for word in [input.clone(), input.to_uppercase()] {
|
||||
let got = suggest_in(&d, &word, 1, &Tuning::DEFAULT);
|
||||
assert_eq!(got.len(), 1, "accepted affixed candidate lost: {word}: {got:?}");
|
||||
assert!(got[0].eq_ignore_ascii_case("kots"), "{word}: {got:?}");
|
||||
assert!(d.can_suggest(&got[0]));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,665 @@
|
||||
//! French grammar and typography checks: cheap, rule-based and tuned to stay quiet when unsure
|
||||
//! (a missed mistake costs less than a wrong mark on correct text). Messages are in French.
|
||||
//!
|
||||
//! - **Space before `;` `!` `?` `:`**: French typography puts a non-breaking space there — a
|
||||
//! narrow one (U+202F) before `;` `!` `?`, a full one (U+00A0) before `:`. Only a missing
|
||||
//! space is flagged (`Bonjour!`): an ordinary one stays quiet until typing turns it into a
|
||||
//! non-breaking one (no French AutoFormat yet). Quebec French (`fr-CA`) has no space before
|
||||
//! `;` `!` `?`, only before `:`. Addresses (`https://…`) and times (`10:30`) are left alone.
|
||||
//! - **Space inside `« »`**: a missing one after `«` or before `»` (`«oui»`) → U+00A0.
|
||||
//! - **Straight double quotes** around a phrase (`"bonjour"`) → `« bonjour »`; inch marks
|
||||
//! (`27"`) and unbalanced quotes are left alone.
|
||||
//! - **Repeated word** (`le le`, `que que`), except repetitions French makes on purpose
|
||||
//! (`nous nous lavons`, `vous vous trompez`, `si si`, interjections) and a proper noun after
|
||||
//! the same lower-case word (`de De…`).
|
||||
//! - **Capital letter at the start of a sentence** after `.` `!` `?`, knowing French
|
||||
//! abbreviations (`etc.`, `M.`, `p.`, `cf.`), initials, numbers, ellipses, addresses,
|
||||
//! dialogue (`« Tu viens ? » demanda-t-elle`), inversions (`Quoi ? dit-il`) and exclamations
|
||||
//! after an interjection (`Ah ! que c'est beau`).
|
||||
//! - **Elision** (`de avoir` → `d’avoir`, `je ai` → `j’ai`, `le homme` → `l’homme`) for `le`,
|
||||
//! `la`, `de`, `je`, `me`, `te`, `se`, `ne`, `que` before a vowel. Before an `h` only for words
|
||||
//! known to have a mute `h`, so aspirated ones (`le hibou`, `la hache`, `le héros`) stay quiet;
|
||||
//! never before a proper noun, `onze`, `oui`, `y…` words (`le yaourt`) or `la une`.
|
||||
//!
|
||||
//! No agreement or conjugation rule, and no English rule (such as `a`/`an`), applies to French
|
||||
//! text.
|
||||
|
||||
use crate::{Issue, IssueKind, ProofLang, words};
|
||||
|
||||
/// Non-breaking space, before `:` and inside `« »`.
|
||||
const NBSP: char = '\u{A0}';
|
||||
/// Narrow non-breaking space, before `;` `!` `?`.
|
||||
const NNBSP: char = '\u{202F}';
|
||||
|
||||
/// Words French repeats on purpose: pronominal verbs (`nous nous lavons`), emphasis,
|
||||
/// interjections, onomatopoeia.
|
||||
const ALLOWED_REPEATS: &[&str] = &[
|
||||
"nous", "vous", "si", "oui", "non", "très", "bon", "vite", "là", "ah", "oh", "eh", "ha", "hé", "ho", "hi", "bla", "chut", "toc", "pan", "bravo",
|
||||
"merci", "allez", "encore", "beaucoup", "fort", "tchin", "bye", "couci", "tam", "miam", "glou",
|
||||
];
|
||||
|
||||
/// Abbreviations written with a final period after which a sentence goes on in lower case
|
||||
/// (lower case, without the period).
|
||||
const ABBREVIATIONS: &[&str] = &[
|
||||
"etc", "cf", "ex", "env", "av", "apr", "mm", "mme", "mmes", "mlle", "mlles", "mgr", "dr", "pr", "me", "st", "ste", "sts", "pp", "vol", "chap",
|
||||
"fig", "éd", "coll", "trad", "réf", "tél", "art", "al", "op", "cit", "ibid", "id", "resp", "min", "max", "sq", "sqq", "suiv", "hab", "janv",
|
||||
"févr", "avr", "juil", "sept", "oct", "nov", "déc", "bd", "boul", "cie", "éts", "vve", "nb", "ps", "no", "nos", "vs", "qqch", "qqn", "ca",
|
||||
"approx", "prof", "dir", "sté", "dépt", "adj", "adv", "fém", "masc", "sing", "plur", "à-d", "ms", "mss", "éds", "trim", "hôp",
|
||||
];
|
||||
|
||||
/// Words after which `!` or `?` doesn't end the sentence (`Ah ! que c'est beau`, `Eh bien ?
|
||||
/// et alors`).
|
||||
const INTERJECTIONS: &[&str] = &[
|
||||
"ah", "oh", "eh", "hé", "hein", "hélas", "bah", "ô", "aïe", "ouf", "oups", "quoi", "comment", "ha", "ho", "bon", "tiens", "allons", "zut",
|
||||
"chut", "bravo", "hourra", "dame", "bien", "là", "oui", "non", "pardon", "voyons",
|
||||
];
|
||||
|
||||
/// Pronouns ending an inversion (`dit-il`, `demanda-t-elle`): after one, a `!` or `?` doesn't
|
||||
/// end the sentence (`Quoi ? dit-il`).
|
||||
const INVERTED: &[&str] = &["-il", "-elle", "-on", "-ils", "-elles", "-je", "-tu", "-nous", "-vous"];
|
||||
|
||||
/// Speech verbs of a dialogue tag: after one, a `!` or `?` doesn't end the sentence (`— Tu
|
||||
/// viens ? demanda le voisin`).
|
||||
const SPEECH_VERBS: &[&str] = &[
|
||||
"dit",
|
||||
"demanda",
|
||||
"répondit",
|
||||
"cria",
|
||||
"murmura",
|
||||
"ajouta",
|
||||
"reprit",
|
||||
"fit",
|
||||
"lança",
|
||||
"souffla",
|
||||
"répliqua",
|
||||
"interrogea",
|
||||
"hurla",
|
||||
"poursuivit",
|
||||
"continua",
|
||||
"déclara",
|
||||
"expliqua",
|
||||
"protesta",
|
||||
"soupira",
|
||||
"chuchota",
|
||||
"insista",
|
||||
"s'écria",
|
||||
"s’écria",
|
||||
"s'exclama",
|
||||
"s’exclama",
|
||||
"objecta",
|
||||
"conclut",
|
||||
"dis",
|
||||
"dit-on",
|
||||
];
|
||||
|
||||
/// The words that elide before a vowel, and their elided form.
|
||||
const ELIDING: &[(&str, &str)] =
|
||||
&[("le", "l"), ("la", "l"), ("de", "d"), ("je", "j"), ("me", "m"), ("te", "t"), ("se", "s"), ("ne", "n"), ("que", "qu")];
|
||||
|
||||
/// Words starting with a vowel that don't take an elision (`le onze`, `le oui`, `de ouate`).
|
||||
const NO_ELISION: &[&str] = &["onze", "onzième", "onzièmes", "oui", "ouistiti", "ouistitis", "ouate", "ouates", "uhlan", "uhlans"];
|
||||
|
||||
/// Words with a mute `h`, which take an elision (`l’homme`, `d’habitude`, `j’habite`). Only these
|
||||
/// are flagged: an `h` word outside the list may be aspirated (`le hibou`, `la hache`, `le
|
||||
/// héros`), and staying quiet is the safe side.
|
||||
const MUTE_H: &[&str] = &[
|
||||
"homme",
|
||||
"hommes",
|
||||
"heure",
|
||||
"heures",
|
||||
"histoire",
|
||||
"histoires",
|
||||
"hiver",
|
||||
"hivers",
|
||||
"hôtel",
|
||||
"hôtels",
|
||||
"herbe",
|
||||
"herbes",
|
||||
"habitude",
|
||||
"habitudes",
|
||||
"habitant",
|
||||
"habitants",
|
||||
"habit",
|
||||
"habits",
|
||||
"hôpital",
|
||||
"hôpitaux",
|
||||
"humain",
|
||||
"humains",
|
||||
"humanité",
|
||||
"honneur",
|
||||
"honneurs",
|
||||
"horizon",
|
||||
"horizons",
|
||||
"huile",
|
||||
"huiles",
|
||||
"hypothèse",
|
||||
"hypothèses",
|
||||
"harmonie",
|
||||
"héritage",
|
||||
"héroïne",
|
||||
"héroïnes",
|
||||
"hésitation",
|
||||
"hommage",
|
||||
"hommages",
|
||||
"horaire",
|
||||
"horaires",
|
||||
"horreur",
|
||||
"horreurs",
|
||||
"humeur",
|
||||
"humour",
|
||||
"hygiène",
|
||||
"hymne",
|
||||
"habiter",
|
||||
"hériter",
|
||||
"hésiter",
|
||||
"habiller",
|
||||
"honorer",
|
||||
"habite",
|
||||
"habites",
|
||||
"habitent",
|
||||
"habitais",
|
||||
"habitait",
|
||||
"hésite",
|
||||
"hésites",
|
||||
"hésitent",
|
||||
"hésitais",
|
||||
"hésitait",
|
||||
"habille",
|
||||
"habillent",
|
||||
"honore",
|
||||
"hier",
|
||||
"hôte",
|
||||
"hôtes",
|
||||
"hôtesse",
|
||||
"humidité",
|
||||
"haleine",
|
||||
"hebdomadaire",
|
||||
"hectare",
|
||||
"hectares",
|
||||
"hélicoptère",
|
||||
"hémisphère",
|
||||
"heureux",
|
||||
"heureuse",
|
||||
"hippopotame",
|
||||
"hirondelle",
|
||||
"honnête",
|
||||
"honnêteté",
|
||||
"huître",
|
||||
"huîtres",
|
||||
"humble",
|
||||
"humilité",
|
||||
"hydrogène",
|
||||
"hypocrite",
|
||||
"horloge",
|
||||
"hôtellerie",
|
||||
"hérédité",
|
||||
"historien",
|
||||
"historique",
|
||||
"humanitaire",
|
||||
"hormone",
|
||||
"hormones",
|
||||
"hospitalité",
|
||||
];
|
||||
|
||||
fn issue(start: usize, end: usize, message: String, suggestions: Vec<String>) -> Issue {
|
||||
Issue { start, end, kind: IssueKind::Grammar, message, suggestions, lang: ProofLang::from_tag(Some("fr-FR")) }
|
||||
}
|
||||
|
||||
/// Is `c` a space that doesn't break (or a thin space some writers use instead)?
|
||||
fn is_fixed_space(c: char) -> bool {
|
||||
matches!(c, '\u{A0}' | '\u{202F}' | '\u{2007}' | '\u{2009}' | '\u{2060}')
|
||||
}
|
||||
|
||||
/// The byte range of the whitespace-delimited token around byte `i` of `text`.
|
||||
fn token_bounds(text: &str, i: usize) -> (usize, usize) {
|
||||
let start = text.get(..i).and_then(|t| t.char_indices().rev().find(|(_, c)| c.is_whitespace())).map_or(0, |(k, c)| k + c.len_utf8());
|
||||
let end = text.get(i..).and_then(|t| t.find(char::is_whitespace)).map_or(text.len(), |k| i + k);
|
||||
(start, end)
|
||||
}
|
||||
|
||||
/// The whitespace-delimited token around byte `i` of `text`.
|
||||
fn token_at(text: &str, i: usize) -> &str {
|
||||
let (start, end) = token_bounds(text, i);
|
||||
text.get(start..end).unwrap_or("")
|
||||
}
|
||||
|
||||
/// Is `token` a web or e-mail address, a path or an HTML entity, where punctuation isn't prose?
|
||||
fn is_address(token: &str) -> bool {
|
||||
token.contains("://") || token.contains('@') || token.starts_with("www.") || token.contains('&') || token.contains('\\')
|
||||
}
|
||||
|
||||
/// `«` + text + `»` in a message, with French spacing.
|
||||
fn quoted(s: &str) -> String {
|
||||
format!("«{NBSP}{s}{NBSP}»")
|
||||
}
|
||||
|
||||
/// `;` `!` `?` `:` without the non-breaking space French puts before them.
|
||||
/// Quebec typography (`quebec`) puts no space before `;` `!` `?`, only before `:`.
|
||||
fn high_punctuation(text: &str, quebec: bool, v: &mut Vec<Issue>) {
|
||||
let mut token: Option<(usize, usize, bool)> = None;
|
||||
for (i, p) in text.char_indices() {
|
||||
if !matches!(p, ';' | ':' | '!' | '?') || (quebec && p != ':') {
|
||||
continue;
|
||||
}
|
||||
let before = text.get(..i).and_then(|t| t.chars().next_back());
|
||||
let after = text.get(i + p.len_utf8()..).and_then(|t| t.chars().next());
|
||||
// One look per token, so a long run of marks stays linear.
|
||||
let address = match token {
|
||||
Some((start, end, address)) if start <= i && i < end => address,
|
||||
_ => {
|
||||
let (start, end) = token_bounds(text, i);
|
||||
let address = is_address(text.get(start..end).unwrap_or(""));
|
||||
token = Some((start, end, address));
|
||||
address
|
||||
}
|
||||
};
|
||||
if address {
|
||||
continue;
|
||||
}
|
||||
let (space, kind) = if p == ':' { (NBSP, "insécable") } else { (NNBSP, "fine insécable") };
|
||||
let message = format!("Espace {kind} attendue avant {}", quoted(&p.to_string()));
|
||||
let fix = format!("{space}{p}");
|
||||
// Only a missing space is flagged: an ordinary one stays quiet until typing turns it into
|
||||
// a non-breaking one by itself (French AutoFormat).
|
||||
if before.is_some_and(char::is_alphabetic) {
|
||||
// `10:30`, `a:b`, `C:` and `x;y` aren't prose: the mark must end the word.
|
||||
let ends = match p {
|
||||
':' => after.is_none_or(char::is_whitespace),
|
||||
_ => after.is_none_or(|a| a.is_whitespace() || matches!(a, '!' | '?' | '»' | '"' | '”' | ')')),
|
||||
};
|
||||
if ends {
|
||||
v.push(issue(i, i + p.len_utf8(), message, vec![fix]));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `«` and `»` without a non-breaking space inside.
|
||||
fn guillemets(text: &str, v: &mut Vec<Issue>) {
|
||||
for (i, c) in text.char_indices() {
|
||||
let len = c.len_utf8();
|
||||
match c {
|
||||
'«' => {
|
||||
let mut next = text.get(i + len..).unwrap_or("").chars();
|
||||
let message = format!("Espace insécable attendue après {}", quoted("«"));
|
||||
if next.next().is_some_and(char::is_alphanumeric) {
|
||||
v.push(issue(i, i + len, message, vec![format!("«{NBSP}")]));
|
||||
}
|
||||
}
|
||||
'»' => {
|
||||
let prev = text.get(..i).unwrap_or("").chars().next_back();
|
||||
let message = format!("Espace insécable attendue avant {}", quoted("»"));
|
||||
if prev.is_some_and(|n| n.is_alphanumeric() || matches!(n, '.' | '!' | '?' | '…')) {
|
||||
v.push(issue(i, i + len, message, vec![format!("{NBSP}»")]));
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `"…"` around a phrase, where French uses `« … »`.
|
||||
fn straight_quotes(text: &str, v: &mut Vec<Issue>) {
|
||||
let quotes: Vec<usize> = text.match_indices('"').map(|(i, _)| i).collect();
|
||||
// An odd count leaves no telling which quote opens and which closes.
|
||||
if !quotes.len().is_multiple_of(2) {
|
||||
return;
|
||||
}
|
||||
for pair in quotes.chunks(2) {
|
||||
let [a, b] = pair else { continue };
|
||||
let (Some(inner), Some(before), Some(after)) = (text.get(a + 1..*b), text.get(..*a), text.get(b + 1..)) else { continue };
|
||||
let opens = before.chars().next_back().is_none_or(|c| c.is_whitespace() || matches!(c, '(' | '[' | '—' | '–'));
|
||||
let closes = after.chars().next().is_none_or(|c| c.is_whitespace() || matches!(c, '.' | ',' | ';' | ':' | '!' | '?' | ')' | ']' | '…'));
|
||||
let phrase = inner.trim();
|
||||
if !opens || !closes || !phrase.chars().any(char::is_alphabetic) || phrase.chars().count() > 120 || phrase.contains('\n') {
|
||||
continue;
|
||||
}
|
||||
v.push(issue(*a, b + 1, format!("Guillemets français attendus{NBSP}: {}", quoted(phrase)), vec![quoted(phrase)]));
|
||||
}
|
||||
}
|
||||
|
||||
fn repeated_words(text: &str, ws: &[(usize, usize)], v: &mut Vec<Issue>) {
|
||||
for pair in ws.windows(2) {
|
||||
let [(a0, b0), (a1, b1)] = pair else { continue };
|
||||
let (Some(w0), Some(w1), Some(gap)) = (text.get(*a0..*b0), text.get(*a1..*b1), text.get(*b0..*a1)) else { continue };
|
||||
let lower = w0.to_lowercase();
|
||||
let starts_upper = |w: &str| w.chars().next().is_some_and(char::is_uppercase);
|
||||
if !gap.is_empty()
|
||||
&& gap.chars().all(char::is_whitespace)
|
||||
&& w0.chars().all(char::is_alphabetic)
|
||||
&& lower == w1.to_lowercase()
|
||||
&& !ALLOWED_REPEATS.contains(&lower.as_str())
|
||||
// `de De…`, `Bora Bora`: a capitalized second word is a name.
|
||||
&& !starts_upper(w1)
|
||||
{
|
||||
v.push(issue(*a0, *b1, format!("Mot répété{NBSP}: {}", quoted(w1)), vec![w0.to_string()]));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Is `prev`, before a period, an abbreviation, an initial, a number or a Roman numeral (after
|
||||
/// which a lower-case word doesn't start a sentence)?
|
||||
fn abbreviation_or_number(prev: &str) -> bool {
|
||||
prev.chars().count() <= 1
|
||||
|| prev.chars().any(|c| c.is_ascii_digit())
|
||||
|| prev.chars().all(|c| matches!(c, 'I' | 'V' | 'X' | 'L' | 'C' | 'D' | 'M'))
|
||||
|| ABBREVIATIONS.contains(&prev.to_lowercase().as_str())
|
||||
}
|
||||
|
||||
/// Does `gap`, between the words `prev` and `next`, end a sentence: a `.`, `!` or `?` (French
|
||||
/// puts a space before the last two), then a space, with only quotes and brackets around? Not
|
||||
/// an ellipsis, not after an abbreviation; for `!` and `?`, not after an interjection, not before
|
||||
/// a closing quote (dialogue goes on: `« Tu viens ? » demanda-t-elle`), not before an inversion
|
||||
/// or a speech verb (`Quoi ? dit-il`, `— Viens ! cria-t-on`), and never in a `dialogue` line (one
|
||||
/// opening with a dash).
|
||||
fn ends_sentence(gap: &str, prev: &str, next: &str, dialogue: bool) -> bool {
|
||||
let closing = ['"', '”', '’', '\'', ')', '»', ']'];
|
||||
let t = gap.trim_start_matches(|c: char| c.is_whitespace() || is_fixed_space(c) || closing.contains(&c));
|
||||
let Some(end) = t.chars().next() else { return false };
|
||||
if !matches!(end, '.' | '!' | '?') || t.starts_with("..") {
|
||||
return false;
|
||||
}
|
||||
if end == '.' && abbreviation_or_number(prev) {
|
||||
return false;
|
||||
}
|
||||
let exclamation = end != '.';
|
||||
let lower_next = next.to_lowercase();
|
||||
if exclamation
|
||||
&& (dialogue
|
||||
|| INTERJECTIONS.contains(&prev.to_lowercase().as_str())
|
||||
|| INVERTED.iter().any(|p| lower_next.ends_with(p))
|
||||
|| SPEECH_VERBS.contains(&lower_next.as_str()))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
let rest = t.trim_start_matches(['.', '!', '?']);
|
||||
let after_marks = rest.trim_start_matches(is_fixed_space);
|
||||
if after_marks.starts_with(closing) {
|
||||
if exclamation {
|
||||
return false;
|
||||
}
|
||||
return ends_sentence_after(after_marks.trim_start_matches(|c: char| closing.contains(&c) || is_fixed_space(c)));
|
||||
}
|
||||
ends_sentence_after(rest)
|
||||
}
|
||||
|
||||
/// Is `rest` (after the final mark) a space, then nothing but opening quotes and brackets?
|
||||
fn ends_sentence_after(rest: &str) -> bool {
|
||||
rest.starts_with(char::is_whitespace)
|
||||
&& rest.trim_start().chars().all(|c| c.is_whitespace() || is_fixed_space(c) || matches!(c, '"' | '“' | '«' | '(' | '[' | '—' | '–'))
|
||||
}
|
||||
|
||||
fn sentence_capitals(text: &str, ws: &[(usize, usize)], v: &mut Vec<Issue>) {
|
||||
let dialogue = text.trim_start().starts_with(['—', '–']);
|
||||
for pair in ws.windows(2) {
|
||||
let [(a0, b0), (a1, b1)] = pair else { continue };
|
||||
let (Some(prev), Some(gap), Some(w)) = (text.get(*a0..*b0), text.get(*b0..*a1), text.get(*a1..*b1)) else { continue };
|
||||
let word_like = w.chars().all(|c| c.is_alphabetic() || matches!(c, '\'' | '’' | '-'));
|
||||
if word_like && w.chars().next().is_some_and(char::is_lowercase) && ends_sentence(gap, prev, w, dialogue) && !is_address(token_at(text, *a1))
|
||||
{
|
||||
let fixed: String = w.chars().next().map(|f| f.to_uppercase().chain(w.chars().skip(1)).collect()).unwrap_or_default();
|
||||
v.push(issue(*a1, *b1, "Majuscule attendue en début de phrase".into(), vec![fixed]));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Does `w` (lower case) start with a sound that makes `le`, `de`, `que`… elide?
|
||||
fn takes_elision(w: &str) -> bool {
|
||||
let Some(first) = w.chars().next() else { return false };
|
||||
if NO_ELISION.contains(&w) {
|
||||
return false;
|
||||
}
|
||||
match first {
|
||||
'a' | 'à' | 'â' | 'ä' | 'e' | 'é' | 'è' | 'ê' | 'ë' | 'i' | 'î' | 'ï' | 'o' | 'ô' | 'ö' | 'u' | 'ù' | 'û' | 'ü' | 'œ' | 'æ' => {
|
||||
true
|
||||
}
|
||||
'h' => MUTE_H.contains(&w),
|
||||
// Only `y` itself (`j’y vais`): `le yaourt`, `de yoga`.
|
||||
'y' => w == "y",
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn elisions(text: &str, ws: &[(usize, usize)], v: &mut Vec<Issue>) {
|
||||
for pair in ws.windows(2) {
|
||||
let [(a0, b0), (a1, b1)] = pair else { continue };
|
||||
let (Some(w0), Some(w1), Some(gap)) = (text.get(*a0..*b0), text.get(*a1..*b1), text.get(*b0..*a1)) else { continue };
|
||||
if gap.is_empty() || !gap.chars().all(|c| c == ' ') {
|
||||
continue;
|
||||
}
|
||||
let lower0 = w0.to_lowercase();
|
||||
let Some((_, elided)) = ELIDING.iter().find(|(full, _)| *full == lower0) else { continue };
|
||||
// `le`/`Le` only, not `LE`; never before a proper noun (`de Orléans`) or a capital.
|
||||
let capital = w0.chars().next().is_some_and(char::is_uppercase);
|
||||
if w0.chars().skip(1).any(char::is_uppercase) || !w1.chars().next().is_some_and(char::is_lowercase) {
|
||||
continue;
|
||||
}
|
||||
if !w1.chars().all(|c| c.is_alphabetic() || c == '-') {
|
||||
continue;
|
||||
}
|
||||
let lower1 = w1.to_lowercase();
|
||||
// `le un`, `la une` (the number, the front page); a single letter only where it is a word
|
||||
// the elision is certain for (`n’a`, `qu’a`, `j’y`).
|
||||
if matches!(lower0.as_str(), "le" | "la") && matches!(lower1.as_str(), "un" | "une") {
|
||||
continue;
|
||||
}
|
||||
let certain = (lower1 == "y" && matches!(lower0.as_str(), "je" | "me" | "te" | "se" | "ne"))
|
||||
|| (lower1 == "a" && matches!(lower0.as_str(), "ne" | "que"));
|
||||
if lower1.chars().count() == 1 && !certain {
|
||||
continue;
|
||||
}
|
||||
if !takes_elision(&lower1) {
|
||||
continue;
|
||||
}
|
||||
let head = if capital { crate::affix::capitalize(elided) } else { (*elided).to_string() };
|
||||
let fixed = format!("{head}’{w1}");
|
||||
v.push(issue(*a0, *b1, format!("Élision{NBSP}: {}", quoted(&fixed)), vec![fixed]));
|
||||
}
|
||||
}
|
||||
|
||||
/// Grammar issues in a paragraph of French text (France's typography), in order.
|
||||
pub fn check(text: &str) -> Vec<Issue> {
|
||||
check_in(text, ProofLang::from_tag(Some("fr-FR")))
|
||||
}
|
||||
|
||||
/// Grammar issues in a French Language pack locale, with Quebec typography in `fr-CA`.
|
||||
/// Other locales have no French issues; the original locale is retained on every issue.
|
||||
pub fn check_in(text: &str, lang: ProofLang) -> Vec<Issue> {
|
||||
let Some(locale) = lang.pack_locale().filter(|l| l.primary() == "fr") else { return Vec::new() };
|
||||
let ws: Vec<(usize, usize)> = words(text);
|
||||
let mut v = Vec::new();
|
||||
high_punctuation(text, crate::lang::region(locale.as_str()).is_some_and(|r| r.eq_ignore_ascii_case("CA")), &mut v);
|
||||
guillemets(text, &mut v);
|
||||
straight_quotes(text, &mut v);
|
||||
repeated_words(text, &ws, &mut v);
|
||||
sentence_capitals(text, &ws, &mut v);
|
||||
elisions(text, &ws, &mut v);
|
||||
for i in &mut v {
|
||||
i.lang = lang;
|
||||
}
|
||||
v.sort_by_key(|i| (i.start, i.end));
|
||||
v
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The issues of `text` as (flagged text, first suggestion).
|
||||
fn found(text: &str) -> Vec<(String, String)> {
|
||||
check(text)
|
||||
.iter()
|
||||
.map(|i| (text.get(i.start..i.end).unwrap_or("?").to_string(), i.suggestions.first().cloned().unwrap_or_default()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn quiet(text: &str) {
|
||||
let v = check(text);
|
||||
assert!(v.is_empty(), "{text:?} → {v:?}");
|
||||
}
|
||||
|
||||
fn one(text: &str, flagged: &str, fix: &str) {
|
||||
assert_eq!(found(text), [(flagged.to_string(), fix.to_string())], "{text:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_space_before_high_punctuation() {
|
||||
one("Bonjour!", "!", "\u{202F}!");
|
||||
one("Tu viens?", "?", "\u{202F}?");
|
||||
one("Il pleut; il vente.", ";", "\u{202F};");
|
||||
one("Note: rien.", ":", "\u{A0}:");
|
||||
// `?!`: only the first mark needs the space.
|
||||
one("Quoi?!", "?", "\u{202F}?");
|
||||
assert!(check("Bonjour!").iter().all(|i| i.lang == ProofLang::from_tag(Some("fr-FR")) && i.message.contains("fine insécable")));
|
||||
assert!(check("Note: rien.").iter().all(|i| i.message.contains("insécable") && !i.message.contains("fine")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_ordinary_space_stays_quiet_until_french_autoformat() {
|
||||
// Typing can't produce a non-breaking space yet, so an ordinary one is accepted.
|
||||
quiet("Vraiment ? Bonjour ! Il pleut ; note : rien.");
|
||||
quiet("Il dit « oui » hier.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn quebec_has_no_space_before_semicolon_exclamation_and_question() {
|
||||
let fr_ca = |t: &str| check_in(t, ProofLang::from_tag(Some("fr-CA")));
|
||||
assert!(fr_ca("Bonjour! Tu viens? Il pleut; il vente.").is_empty(), "{:?}", fr_ca("Bonjour! Tu viens?"));
|
||||
// The colon and the guillemets still want their space in Quebec.
|
||||
let v = fr_ca("Note: «oui».");
|
||||
assert_eq!(v.len(), 3, "{v:?}");
|
||||
assert!(v.iter().all(|i| i.lang == ProofLang::from_tag(Some("fr-CA"))), "{v:?}");
|
||||
assert_eq!(check_in("Bonjour!", ProofLang::from_tag(Some("fr-FR"))).len(), 1, "France still wants the space");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn high_punctuation_stays_quiet() {
|
||||
quiet("Bonjour\u{202F}! Tu viens\u{202F}? Il pleut\u{202F}; note\u{A0}: rien.");
|
||||
quiet("Bonjour\u{A0}! Quoi\u{202F}?! Ah\u{2009}!");
|
||||
quiet("Il est 10:30.");
|
||||
quiet("Voir https://exemple.org:8080/a?b=c pour la suite.");
|
||||
quiet("Écrire à contact@exemple.org!");
|
||||
quiet("Super :) et ;-) aussi :D");
|
||||
quiet("a:b et x;y");
|
||||
quiet("Le fichier C:\\dossier est là.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_space_inside_guillemets() {
|
||||
assert_eq!(found("Il dit «oui» hier."), [("«".to_string(), "«\u{A0}".to_string()), ("»".to_string(), "\u{A0}»".to_string())]);
|
||||
quiet("Il dit «\u{A0}oui\u{A0}» hier.");
|
||||
quiet("Il dit «\u{202F}oui\u{202F}» hier.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn straight_quotes_become_guillemets() {
|
||||
one("Il a dit \"bonjour\" hier.", "\"bonjour\"", "«\u{A0}bonjour\u{A0}»");
|
||||
one("\"Bonne idée\", dit-elle.", "\"Bonne idée\"", "«\u{A0}Bonne idée\u{A0}»");
|
||||
// Inch marks and unbalanced quotes are left alone.
|
||||
quiet("Un écran de 27\" et un de 32\" ici.");
|
||||
quiet("Il a dit \"bonjour hier.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repeated_words_are_flagged() {
|
||||
one("Le le chat dort.", "Le le", "Le");
|
||||
one("Je pense que que tu viens.", "que que", "que");
|
||||
one("Il mange la la pomme.", "la la", "la");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deliberate_repetitions_stay_quiet() {
|
||||
quiet("Nous nous lavons et vous vous trompez.");
|
||||
quiet("Si si, très très bien.");
|
||||
quiet("Ah ah, bla bla.");
|
||||
quiet("Je crois que qu’il viendra.");
|
||||
quiet("Le livre de De Gaulle.");
|
||||
quiet("Nous partons à Bora Bora.");
|
||||
quiet("Ils ont dit bye bye. Tchin tchin\u{202F}!");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sentence_capitals_are_flagged() {
|
||||
one("Il pleut. il fait froid.", "il", "Il");
|
||||
one("Il pleut\u{202F}! il fait froid.", "il", "Il");
|
||||
one("Tu viens\u{202F}? on part.", "on", "On");
|
||||
one("Il pleut. l’orage arrive.", "l’orage", "L’orage");
|
||||
one("Il pleut. «\u{A0}tout va bien\u{A0}», dit-il.", "tout", "Tout");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sentence_capitals_stay_quiet() {
|
||||
quiet("Voir p. 12 et cf. la note, etc. puis la suite.");
|
||||
quiet("M. le maire est là.");
|
||||
quiet("Il attend… puis part. Il attend... puis part.");
|
||||
quiet("«\u{A0}Tu viens\u{202F}?\u{A0}» demanda-t-elle.");
|
||||
quiet("Ah\u{202F}! que c’est beau.");
|
||||
quiet("Quoi\u{202F}? dit-il.");
|
||||
quiet("Tu viens\u{202F}? demanda le voisin.");
|
||||
quiet("— Tu viens\u{202F}? reprends-tu, inquiet.");
|
||||
quiet("– Viens\u{202F}! on part.");
|
||||
quiet("Voir www.exemple.fr pour la suite.");
|
||||
quiet("En 2026. ensuite rien.");
|
||||
quiet("il commence en minuscule.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_elisions_are_flagged() {
|
||||
one("Il refuse de avoir peur.", "de avoir", "d’avoir");
|
||||
one("Je ai faim.", "Je ai", "J’ai");
|
||||
one("Voici le homme.", "le homme", "l’homme");
|
||||
one("Je sais que il viendra.", "que il", "qu’il");
|
||||
one("Le arbre est grand.", "Le arbre", "L’arbre");
|
||||
one("Il ne a pas faim.", "ne a", "n’a");
|
||||
one("Bon, je y vais.", "je y", "j’y");
|
||||
one("Il se est levé.", "se est", "s’est");
|
||||
one("Voici la école.", "la école", "l’école");
|
||||
one("Il me aime.", "me aime", "m’aime");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn aspirated_h_and_other_exceptions_stay_quiet() {
|
||||
for w in [
|
||||
"hibou",
|
||||
"héros",
|
||||
"haricot",
|
||||
"hasard",
|
||||
"haut",
|
||||
"hall",
|
||||
"hamac",
|
||||
"hangar",
|
||||
"hockey",
|
||||
"homard",
|
||||
"huit",
|
||||
"hérisson",
|
||||
"hors-jeu",
|
||||
"onze",
|
||||
"oui",
|
||||
"yaourt",
|
||||
] {
|
||||
quiet(&format!("Voici le {w}."));
|
||||
}
|
||||
// hiérarchie has an aspirated h, despite the mute h in hiéroglyphe:
|
||||
// https://www.academie-francaise.fr/le-hieroglyphe-pour-lhieroglyphe
|
||||
for w in ["honte", "hache", "harpe", "hausse", "hiérarchie"] {
|
||||
quiet(&format!("Voici la {w}."));
|
||||
}
|
||||
quiet("Une question de hiérarchie.");
|
||||
quiet("Le onze de France. À la une du journal.");
|
||||
quiet("Il vient de Orléans.");
|
||||
quiet("De a à z, le a et le b.");
|
||||
quiet("Le y et le x sont des variables. La dérivée de y.");
|
||||
quiet("Je mange de la viande.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hostile_input_never_panics() {
|
||||
for t in ["", " ", "!", " !", "«", "»", " »", "« ", "\"", "\"\"", ". a", "de", "de ", "\u{202F}?", "é!é", "« » « »", ":", " :", "a :)"]
|
||||
{
|
||||
let _ = check(t);
|
||||
}
|
||||
// A long run of marks in one token: linear, not one token scan per mark.
|
||||
let run = format!("mot{}", "?".repeat(200_000));
|
||||
assert_eq!(check(&run).len(), 1);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,193 @@
|
||||
//! French proofing through the shared Language pack cache.
|
||||
//!
|
||||
//! A French locale remains [`ProofLang::Pack`]: dictionary selection, installation, reloads,
|
||||
//! memory limits and hyphenation use the same paths as other packs. This module adds French
|
||||
//! elisions, accent-oriented suggestions and conservative typography, including Quebec spacing.
|
||||
//! No dictionary or hyphenation data is bundled here.
|
||||
|
||||
pub mod grammar;
|
||||
mod suggest;
|
||||
#[cfg(test)]
|
||||
#[allow(clippy::unwrap_used)]
|
||||
pub(crate) mod test_dict;
|
||||
|
||||
use crate::affix::AffixDict;
|
||||
use crate::{Options, ProofLang};
|
||||
|
||||
pub use suggest::suggest_in;
|
||||
|
||||
/// Words after whose elision a user-dictionary entry is accepted. Ordinary words and their
|
||||
/// elisions are always decided by the dictionary (`l'arbre`, but not `l'arbres`).
|
||||
const ELISIONS: &[&str] = &["l", "d", "j", "m", "t", "s", "n", "c", "qu", "jusqu", "lorsqu", "puisqu", "quoiqu", "presqu", "quelqu"];
|
||||
|
||||
/// Check French spelling in the installed France/generic pack, with the default options.
|
||||
/// Regional callers should use [`crate::is_correct_in`] with their full locale.
|
||||
pub fn is_correct(word: &str) -> bool {
|
||||
crate::is_correct_in(word, ProofLang::from_tag(Some("fr-FR")))
|
||||
}
|
||||
|
||||
/// French suggestions from the installed France/generic pack, best first, at most `max`.
|
||||
/// Regional callers should use [`crate::suggest_in`] with their full locale.
|
||||
pub fn suggest(word: &str, max: usize) -> Vec<String> {
|
||||
crate::suggest_in(word, max, ProofLang::from_tag(Some("fr-FR")))
|
||||
}
|
||||
|
||||
/// Check against a supplied dictionary, without changing any installed pack or cache.
|
||||
pub fn is_correct_with(d: &AffixDict, word: &str) -> bool {
|
||||
check(d, &crate::composed(word), &Options::default())
|
||||
}
|
||||
|
||||
pub(crate) fn check(d: &AffixDict, word: &str, opts: &Options) -> bool {
|
||||
let Some(w) = crate::spelling::word_to_check(d, word, opts) else { return true };
|
||||
// A regional dictionary may store the curly apostrophe without ICONV. Honor that entry
|
||||
// before the French ASCII-apostrophe fallback.
|
||||
if known(d, w) {
|
||||
return true;
|
||||
}
|
||||
let original = w;
|
||||
let w = w.replace('’', "'");
|
||||
if known(d, &w) {
|
||||
return true;
|
||||
}
|
||||
if let Some((head, tail)) = w.split_once('\'')
|
||||
&& ELISIONS.contains(&head.to_lowercase().as_str())
|
||||
&& !tail.is_empty()
|
||||
&& !tail.contains('\'')
|
||||
&& crate::in_user_dictionary(&tail.to_lowercase())
|
||||
{
|
||||
return true;
|
||||
}
|
||||
original.contains('-') && original.split('-').all(|p| !p.is_empty() && (p.chars().count() <= 1 || known(d, p) || known(d, &p.replace('’', "'"))))
|
||||
}
|
||||
|
||||
fn known(d: &AffixDict, word: &str) -> bool {
|
||||
d.check(word) || crate::in_user_dictionary(&word.to_lowercase())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[allow(clippy::unwrap_used)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn original_french_words_and_elisions() {
|
||||
let d = test_dict::dictionary();
|
||||
for word in [
|
||||
"maison",
|
||||
"maisons",
|
||||
"école",
|
||||
"écoles",
|
||||
"grand",
|
||||
"grande",
|
||||
"grands",
|
||||
"grandes",
|
||||
"aimer",
|
||||
"aime",
|
||||
"aimons",
|
||||
"aiment",
|
||||
"suis",
|
||||
"est",
|
||||
"sommes",
|
||||
"sont",
|
||||
"a",
|
||||
"avons",
|
||||
"ont",
|
||||
"Paris",
|
||||
"Montréal",
|
||||
"œuvre",
|
||||
"sœur",
|
||||
"aujourd'hui",
|
||||
"aujourd’hui",
|
||||
"peut-être",
|
||||
"arc-en-ciel",
|
||||
"km",
|
||||
"Maison",
|
||||
"MAISON",
|
||||
"SNCF",
|
||||
"2026",
|
||||
"très",
|
||||
"rivière",
|
||||
"l'arbre",
|
||||
"l’arbre",
|
||||
"L'arbre",
|
||||
"d'enfants",
|
||||
"l'homme",
|
||||
"qu'elle",
|
||||
"qu’il",
|
||||
"j'aime",
|
||||
"j’ai",
|
||||
"d'avoir",
|
||||
"l'école",
|
||||
"jusqu'à",
|
||||
"maison-école",
|
||||
"est-il",
|
||||
"a-t-il",
|
||||
] {
|
||||
assert!(is_correct_with(&d, word), "{word}");
|
||||
}
|
||||
for word in [
|
||||
"maisonn",
|
||||
"ecole",
|
||||
"arbrre",
|
||||
"beaucoups",
|
||||
"aimmer",
|
||||
"soeur",
|
||||
"Km",
|
||||
"grandd",
|
||||
"aim",
|
||||
"l'hasard",
|
||||
"l'chat",
|
||||
"l'arbres",
|
||||
"l'arbrre",
|
||||
"qu'maison",
|
||||
"maison-ecole",
|
||||
"maison--école",
|
||||
] {
|
||||
assert!(!is_correct_with(&d, word), "{word}");
|
||||
}
|
||||
assert!(is_correct_with(&d, "e\u{301}cole"));
|
||||
assert!(!is_correct_with(&d, "maison\u{338}"), "a trailing mark is part of the word");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn curly_apostrophes_in_compounds_keep_exact_dictionary_parts() {
|
||||
let d = AffixDict::from_hunspell("SET UTF-8\n", "4\naujourd’hui\nécole\ncafé\nl'école\n", &[]).unwrap();
|
||||
for word in ["aujourd’hui-école", "aujourd’hui-e\u{301}cole", "café-aujourd’hui", "l’école-aujourd’hui"] {
|
||||
assert!(is_correct_with(&d, word), "{word}");
|
||||
}
|
||||
for word in ["aujourd’hui-écolle", "aujourd’hui--école", "aujourd'hui-école"] {
|
||||
assert!(!is_correct_with(&d, word), "{word}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn explicit_curly_apostrophe_without_iconv() {
|
||||
let d = AffixDict::from_hunspell("SET UTF-8\n", "1\naujourd’hui\n", &[]).unwrap();
|
||||
assert!(is_correct_with(&d, "aujourd’hui"));
|
||||
assert!(!is_correct_with(&d, "aujourd’huui"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn user_word_after_an_elision() {
|
||||
let d = test_dict::dictionary();
|
||||
let word = "zxqmotadaptfr";
|
||||
assert!(!is_correct_with(&d, word) && !is_correct_with(&d, &format!("l'{word}")));
|
||||
crate::add_word(word);
|
||||
assert!(is_correct_with(&d, word) && is_correct_with(&d, &format!("l'{word}")) && is_correct_with(&d, &format!("d’{word}")));
|
||||
assert!(!is_correct_with(&d, &format!("chat'{word}")));
|
||||
crate::remove_word(word);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn options_and_missing_dictionary() {
|
||||
let d = test_dict::dictionary();
|
||||
let defaults = Options::default();
|
||||
let strict = Options { ignore_uppercase: false, ignore_numbers: false, ..defaults };
|
||||
assert!(check(&d, "SNCF", &defaults) && check(&d, "mp3x", &defaults));
|
||||
assert!(!check(&d, "SNCF", &strict) && !check(&d, "mp3x", &strict));
|
||||
assert!(check(&d, "MAISON", &strict) && check(&d, "2026", &strict));
|
||||
for word in ["école", "l'arbre", "xqzv", "SNCF", ""] {
|
||||
assert!(check(&AffixDict::default(), word, &strict));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
//! French tuning of the shared, bounded affix suggestion engine.
|
||||
|
||||
use crate::affix::AffixDict;
|
||||
use crate::affix_suggest::{Split, Tuning};
|
||||
|
||||
/// Short words often run into the next one (`et la`, `de Paris`, `pour moi`).
|
||||
const LEADING_WORDS: &[&str] = &[
|
||||
"à", "a", "au", "aux", "de", "du", "des", "le", "la", "les", "un", "une", "et", "ou", "où", "en", "y", "ce", "se", "ne", "je", "tu", "il", "on",
|
||||
"nous", "vous", "ils", "elle", "par", "pour", "sur", "sous", "dans", "avec", "sans", "chez", "vers", "entre", "très", "plus", "moins", "bien",
|
||||
"pas", "que", "qui", "mais", "donc", "car", "si", "son", "sa", "ses", "mon", "ma", "mes", "ton", "ta", "tes", "est", "tout", "tous", "toute",
|
||||
"comme", "quand", "puis", "encore", "déjà", "jamais", "toujours",
|
||||
];
|
||||
/// Short words often run into the one before (`dis-moi` without the hyphen, `là-bas`, or a
|
||||
/// dropped space: `pourmoi` → `pour moi`).
|
||||
const TRAILING_WORDS: &[&str] =
|
||||
&["moi", "toi", "lui", "elle", "nous", "vous", "eux", "le", "la", "les", "en", "y", "là", "ci", "il", "on", "ils", "elles", "est", "et"];
|
||||
|
||||
const TUNING: Tuning = Tuning {
|
||||
split: Split::Near { leading: LEADING_WORDS, trailing: TRAILING_WORDS },
|
||||
alphabet: "esaitnrulodcpmévqfbghjàxèêyçôîâûùœ",
|
||||
..Tuning::DEFAULT
|
||||
};
|
||||
|
||||
/// French suggestions from a supplied dictionary; keeps the writer's apostrophe and case.
|
||||
/// Without a typed apostrophe, keep the dictionary's OCONV output unchanged.
|
||||
pub fn suggest_in(d: &AffixDict, word: &str, max: usize) -> Vec<String> {
|
||||
let word = crate::composed(word);
|
||||
let apostrophe = if word.contains('’') {
|
||||
Some('’')
|
||||
} else if word.contains('\'') {
|
||||
Some('\'')
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let mut out = Vec::new();
|
||||
for text in crate::affix_suggest::suggest_in(d, &word, max, &TUNING) {
|
||||
let styled: String = text.chars().map(|c| if c == '\'' || c == '’' { apostrophe.unwrap_or(c) } else { c }).collect();
|
||||
// A pack may spell only one apostrophe and have no ICONV. Keep its valid output
|
||||
// rather than turn it back into the spelling mistake the caller is correcting.
|
||||
let text = if styled.split(' ').all(|p| d.can_suggest_cased(p, p)) { styled } else { text };
|
||||
if text != word && !out.contains(&text) {
|
||||
out.push(text);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[allow(clippy::unwrap_used)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn french_accents_confusions_splits_and_elisions() {
|
||||
let d = super::super::test_dict::dictionary();
|
||||
for (typo, expected) in [
|
||||
("ecole", "école"),
|
||||
("riviere", "rivière"),
|
||||
("soeur", "sœur"),
|
||||
("maisonn", "maison"),
|
||||
("pourmoi", "pour moi"),
|
||||
("l'arbrre", "l'arbre"),
|
||||
("l’arbrre", "l’arbre"),
|
||||
] {
|
||||
let got = suggest_in(&d, typo, 8);
|
||||
assert!(got.iter().any(|s| s == expected), "{typo}: {got:?}");
|
||||
}
|
||||
assert!(suggest_in(&d, "e\u{301}cole", 4).iter().all(|s| s != "école"));
|
||||
assert!(suggest_in(&d, "ecole", 0).is_empty());
|
||||
assert!(suggest_in(&d, &"a".repeat(41), 4).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn curly_dictionary_without_iconv_remains_suggestible() {
|
||||
let d = AffixDict::from_hunspell("SET UTF-8\n", "1\naujourd’hui\n", &[]).unwrap();
|
||||
assert!(suggest_in(&d, "aujourd’huui", 4).iter().any(|s| s == "aujourd’hui"));
|
||||
assert!(suggest_in(&d, "aujourd'hui", 4).iter().any(|s| s == "aujourd’hui"));
|
||||
}
|
||||
|
||||
/// The French wrapper must retain the shared engine's missing-apostrophe fix and the
|
||||
/// dictionary's output conversion when the writer supplied no apostrophe style.
|
||||
#[test]
|
||||
fn missing_apostrophe_keeps_dictionary_output_and_shared_ranking() {
|
||||
let aff = "SET UTF-8\nFLAG long\nWORDCHARS -’'\nICONV 1\nICONV ’ '\nOCONV 1\nOCONV ' ’\n\
|
||||
PFX j' Y 1\nPFX j' 0 j' .\nPFX l' Y 1\nPFX l' 0 l' .\n";
|
||||
let d = AffixDict::from_hunspell(aff, "5\nadore/j'\nami/l'\njade\nj\nl\n", &[]).unwrap();
|
||||
for (word, want, space) in
|
||||
[("Jadore", "J’adore", "J adore"), ("jadore", "j’adore", "j adore"), ("JADORE", "J’ADORE", "J ADORE"), ("lami", "l’ami", "l ami")]
|
||||
{
|
||||
for max in [0, 1, 6] {
|
||||
let got = suggest_in(&d, word, max);
|
||||
assert!(got.len() <= max, "{word}: {got:?}");
|
||||
if max == 0 {
|
||||
assert!(got.is_empty());
|
||||
} else {
|
||||
assert_eq!(got.first().map(String::as_str), Some(want), "{word}: {got:?}");
|
||||
if let Some(at) = got.iter().position(|s| s == space) {
|
||||
assert!(at > 0, "elision before split: {word}: {got:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for (word, want) in [("j'adorr", "j'adore"), ("j’adorr", "j’adore"), ("J'adorr", "J'adore")] {
|
||||
let got = suggest_in(&d, word, 6);
|
||||
assert!(got.iter().any(|s| s == want), "typed apostrophe: {word}: {got:?}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn output_conversion_preserves_typed_apostrophe() {
|
||||
let d = AffixDict::from_hunspell("SET UTF-8\nICONV 1\nICONV ’ '\nOCONV 1\nOCONV ' ’\n", "1\nl'arbre\n", &[]).unwrap();
|
||||
assert!(suggest_in(&d, "l'arbrre", 4).iter().any(|s| s == "l'arbre"));
|
||||
assert!(suggest_in(&d, "l’arbrre", 4).iter().any(|s| s == "l’arbre"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
//! A tiny synthetic French Hunspell dictionary for tests (written for these tests; not taken
|
||||
//! from any real dictionary). Real dictionaries are installed on the system, never bundled.
|
||||
//!
|
||||
//! It exercises what the Dicollecte French dictionary uses: two-character flags (`FLAG long`),
|
||||
//! stems that need an affix (`NEEDAFFIX`, with an empty suffix for the singular), elisions as
|
||||
//! prefixes licensed by a suffix's continuation (`l'arbre` but not `l'arbres`), irregular verbs
|
||||
//! through `FULLSTRIP` (`être` → `suis`), `KEEPCASE` (`km`), `NOSUGGEST` (`bah`), `ICONV` (`’`
|
||||
//! read as `'`) and the
|
||||
//! suggestion tables (`TRY`, `MAP` for accents, `REP` for common confusions).
|
||||
|
||||
pub const AFF: &str = "SET UTF-8
|
||||
FLAG long
|
||||
NEEDAFFIX ()
|
||||
KEEPCASE ||
|
||||
NOSUGGEST --
|
||||
FULLSTRIP
|
||||
ICONV 1
|
||||
ICONV ’ '
|
||||
TRY esaitnrulodcpmévqfbghjàxèêyçôîâûùœ
|
||||
MAP 6
|
||||
MAP aàâä
|
||||
MAP eéèêë
|
||||
MAP iîï
|
||||
MAP oôö
|
||||
MAP uùûü
|
||||
MAP cç
|
||||
REP 3
|
||||
REP oe œ
|
||||
REP ph f
|
||||
REP f ph
|
||||
PFX L' Y 1
|
||||
PFX L' 0 l' [aàâeéèêiîoôuœh]
|
||||
PFX D' Y 1
|
||||
PFX D' 0 d' [aàâeéèêiîoôuœh]
|
||||
PFX Q' Y 1
|
||||
PFX Q' 0 qu' [aàâeéèêiîoôuœh]
|
||||
PFX J' Y 1
|
||||
PFX J' 0 j' [aàâeéèêiîoôuœhy]
|
||||
SFX S. Y 2
|
||||
SFX S. 0 0/L'D'Q' [^sxz]
|
||||
SFX S. 0 s/D'Q' [^sxz]
|
||||
SFX S= Y 2
|
||||
SFX S= 0 0 [^sxz]
|
||||
SFX S= 0 s [^sxz]
|
||||
SFX F. Y 4
|
||||
SFX F. 0 0/L'D'Q' .
|
||||
SFX F. 0 e/L'D'Q' .
|
||||
SFX F. 0 s/D'Q' .
|
||||
SFX F. 0 es/D'Q' .
|
||||
SFX Er Y 6
|
||||
SFX Er er er/D' er
|
||||
SFX Er er e/J' er
|
||||
SFX Er er es er
|
||||
SFX Er er ons er
|
||||
SFX Er er ez er
|
||||
SFX Er er ent er
|
||||
SFX Et N 7
|
||||
SFX Et être être être
|
||||
SFX Et être suis être
|
||||
SFX Et être es être
|
||||
SFX Et être est être
|
||||
SFX Et être sommes être
|
||||
SFX Et être êtes être
|
||||
SFX Et être sont être
|
||||
SFX Av N 7
|
||||
SFX Av avoir avoir/D' avoir
|
||||
SFX Av avoir ai/J' avoir
|
||||
SFX Av avoir as avoir
|
||||
SFX Av avoir a avoir
|
||||
SFX Av avoir avons avoir
|
||||
SFX Av avoir avez avoir
|
||||
SFX Av avoir ont/Q' avoir
|
||||
";
|
||||
|
||||
/// Stems (the count line first, as in Hunspell files).
|
||||
pub const DIC: &str = "150
|
||||
maison/S.()
|
||||
arbre/S.()
|
||||
enfant/S.()
|
||||
ami/S.()
|
||||
amie/S.()
|
||||
homme/S.()
|
||||
école/S.()
|
||||
café/S.()
|
||||
chat/S.()
|
||||
rivière/S.()
|
||||
ville/S.()
|
||||
heure/S.()
|
||||
histoire/S.()
|
||||
sœur/S.()
|
||||
œuvre/S.()
|
||||
photo/S.()
|
||||
téléphone/S.()
|
||||
année/S.()
|
||||
vie/S.()
|
||||
livre/S.()
|
||||
table/S.()
|
||||
porte/S.()
|
||||
femme/S.()
|
||||
garçon/S.()
|
||||
fille/S.()
|
||||
père/S.()
|
||||
mère/S.()
|
||||
langue/S.()
|
||||
mot/S.()
|
||||
texte/S.()
|
||||
phrase/S.()
|
||||
exemple/S.()
|
||||
jour/S.()
|
||||
nuit/S.()
|
||||
monde/S.()
|
||||
hasard/S=()
|
||||
haricot/S=()
|
||||
grand/F.()
|
||||
petit/F.()
|
||||
noir/F.()
|
||||
aimer/Er()
|
||||
parler/Er()
|
||||
chanter/Er()
|
||||
arriver/Er()
|
||||
être/Et()
|
||||
avoir/Av()
|
||||
il/Q'
|
||||
ils/Q'
|
||||
elle/Q'
|
||||
elles/Q'
|
||||
on/Q'L'
|
||||
un/D'Q'L'
|
||||
une/D'Q'L'
|
||||
en/Q'J'
|
||||
y/J'
|
||||
à/Q'
|
||||
jusqu'à
|
||||
aujourd'hui/D'
|
||||
peut-être
|
||||
arc-en-ciel
|
||||
km/||
|
||||
bah/--
|
||||
Paris
|
||||
France
|
||||
Montréal
|
||||
le
|
||||
la
|
||||
les
|
||||
de
|
||||
des
|
||||
du
|
||||
et
|
||||
à
|
||||
je
|
||||
tu
|
||||
nous
|
||||
vous
|
||||
que
|
||||
qui
|
||||
ne
|
||||
pas
|
||||
se
|
||||
me
|
||||
te
|
||||
ce
|
||||
ça
|
||||
avec
|
||||
pour
|
||||
dans
|
||||
sur
|
||||
sous
|
||||
très
|
||||
près
|
||||
beaucoup
|
||||
si
|
||||
oui
|
||||
non
|
||||
mais
|
||||
ou
|
||||
où
|
||||
dit
|
||||
demanda
|
||||
quand
|
||||
comme
|
||||
plus
|
||||
bien
|
||||
tout
|
||||
toujours
|
||||
jamais
|
||||
ici
|
||||
là
|
||||
alors
|
||||
donc
|
||||
aussi
|
||||
encore
|
||||
déjà
|
||||
mon
|
||||
ma
|
||||
mes
|
||||
son
|
||||
sa
|
||||
ses
|
||||
notre
|
||||
votre
|
||||
leur
|
||||
cette
|
||||
ces
|
||||
au
|
||||
aux
|
||||
par
|
||||
entre
|
||||
vers
|
||||
chez
|
||||
sans
|
||||
moi
|
||||
toi
|
||||
lui
|
||||
rien
|
||||
chose
|
||||
temps
|
||||
fois
|
||||
eau
|
||||
voici
|
||||
voilà
|
||||
bonjour
|
||||
merci
|
||||
beau
|
||||
belle
|
||||
la
|
||||
etc
|
||||
M
|
||||
p
|
||||
";
|
||||
|
||||
/// The original synthetic fixture, parsed independently of the installed pack cache.
|
||||
pub fn dictionary() -> crate::affix::AffixDict {
|
||||
crate::affix::AffixDict::from_hunspell(AFF, DIC, &[]).unwrap()
|
||||
}
|
||||
@@ -42,6 +42,9 @@
|
||||
//! existing English fallback.
|
||||
//! - **German** ([`ProofLang::De`]): hyphenation by `hyph-utf8`'s German patterns (reformed
|
||||
//! spelling, MIT); no spelling or grammar marks yet.
|
||||
//! - **French** (a [`ProofLang::Pack`] locale; module [`fr`]): the pack dictionary with French
|
||||
//! elisions and accent-oriented suggestions, plus conservative typography and basic checks.
|
||||
//! France and Canada retain their regional tags; Quebec uses its own punctuation spacing.
|
||||
//! - **Any other language** ([`ProofLang::Pack`], module [`packs`]): spelling and suggestions
|
||||
//! from its Hunspell dictionary (an installed Language pack, the dictionaries folder or the
|
||||
//! system's) with default tuning, hyphenation from its pack, no grammar rules; without a
|
||||
@@ -73,6 +76,7 @@ pub mod dict;
|
||||
pub mod en_variants;
|
||||
mod english;
|
||||
pub mod es;
|
||||
pub mod fr;
|
||||
pub mod hr;
|
||||
pub mod hunspell;
|
||||
pub mod hyphen;
|
||||
@@ -976,6 +980,7 @@ fn grammar_in(text: &str, lang: ProofLang, capital_first: bool) -> Vec<Issue> {
|
||||
match lang {
|
||||
ProofLang::En => grammar_en(text, capital_first),
|
||||
ProofLang::EnGb | ProofLang::EnAu | ProofLang::EnCa => grammar_en(text, capital_first).into_iter().map(|i| Issue { lang, ..i }).collect(),
|
||||
ProofLang::Pack(locale) if locale.primary() == "fr" => fr::grammar::check_in(text, lang),
|
||||
// `pt-pt` shares Portuguese's rules, its issues in its own language.
|
||||
_ => language(lang.built_in())
|
||||
.map_or_else(Vec::new, |l| (l.grammar)(text, lang, capital_first).into_iter().map(|i| Issue { lang, ..i }).collect()),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! Proofing for any language by its locale ([`crate::ProofLang::Pack`]): its Hunspell dictionary (the
|
||||
//! dictionaries folder, then an installed Language pack, then the system's), read by the shared
|
||||
//! affix engine, with default suggestion tuning and no grammar rules.
|
||||
//! affix engine. Most use default suggestion tuning and no grammar rules; French locales add
|
||||
//! the spelling tuning and conservative typography in [`crate::fr`] without a separate cache.
|
||||
//!
|
||||
//! The loaded dictionaries share a memory budget ([`MAX_LOADED_BYTES`], estimated from their
|
||||
//! files); the one used least recently goes first, but never one used during the current
|
||||
@@ -337,6 +338,9 @@ pub(crate) fn is_correct(word: &str, locale: PackLocale, opts: &crate::Options)
|
||||
if other_script(w, &scripts) {
|
||||
return true;
|
||||
}
|
||||
if locale.primary() == "fr" {
|
||||
return crate::fr::check(&d, w, opts);
|
||||
}
|
||||
let known = |w: &str| d.check(w) || crate::in_user_dictionary(&w.to_lowercase());
|
||||
if w.contains('-') && !known(w) {
|
||||
return w.split('-').all(|p| p.chars().count() <= 1 || known(p));
|
||||
@@ -350,7 +354,7 @@ pub(crate) fn suggest(word: &str, max: usize, locale: PackLocale) -> Vec<String>
|
||||
if d.is_empty() {
|
||||
return Vec::new();
|
||||
}
|
||||
crate::affix_suggest::suggest_in(&d, word, max, &Tuning::DEFAULT)
|
||||
if locale.primary() == "fr" { crate::fr::suggest_in(&d, word, max) } else { crate::affix_suggest::suggest_in(&d, word, max, &Tuning::DEFAULT) }
|
||||
}
|
||||
|
||||
/// The pack locales whose hyphenation patterns serve a [`crate::ProofLang::Pack`] language: the locale,
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
//! Frozen public output from the pre-French shared affix engine at upstream
|
||||
//! e6c5a701ddff5be16e266028168c70658aae39d9, using original synthetic fixtures.
|
||||
//! Do not regenerate expectations from the new implementation: they establish the baseline.
|
||||
//! Legacy WCAFF1 bytes remain frozen, but are explicitly refused by the WCAFF2 engine.
|
||||
|
||||
use craft_proof::affix::{AffixDict, capitalize};
|
||||
use std::fmt::Write;
|
||||
|
||||
#[path = "fixtures/affix_compatibility_cases.rs"]
|
||||
mod fixtures;
|
||||
|
||||
fn snapshot(case: &fixtures::Case, dict: &AffixDict) -> String {
|
||||
let mut out = format!("{}\nlen={} rules={}\n", case.name, dict.len(), dict.rule_count());
|
||||
writeln!(out, "try={:?} key={:?} map={:?} rep={:?} comments={:?}", dict.try_chars(), dict.key(), dict.map(), dict.rep(), dict.comments())
|
||||
.unwrap();
|
||||
let mut stems: Vec<_> = dict.stems().collect();
|
||||
stems.sort_unstable();
|
||||
for stem in &stems {
|
||||
let mut forms = dict.forms(stem);
|
||||
forms.sort_unstable();
|
||||
forms.dedup();
|
||||
writeln!(out, "forms {stem:?}: {forms:?}").unwrap();
|
||||
}
|
||||
let mut probes: Vec<String> = case.probes.iter().map(|word| (*word).to_string()).collect();
|
||||
for stem in stems {
|
||||
probes.extend(dict.forms(stem));
|
||||
}
|
||||
let originals = probes.clone();
|
||||
for word in originals {
|
||||
probes.extend([word.to_uppercase(), word.to_lowercase(), capitalize(&word)]);
|
||||
}
|
||||
probes.sort_unstable();
|
||||
probes.dedup();
|
||||
for word in probes {
|
||||
writeln!(
|
||||
out,
|
||||
"{word:?}: contains={} check={} suggest={} raw_suggest={}",
|
||||
dict.contains(&word),
|
||||
dict.check(&word),
|
||||
dict.can_suggest(&word),
|
||||
dict.contains_suggestible(&word)
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_french_affix_outputs_match_pre_french_baseline() {
|
||||
let expected = include_str!("fixtures/affix_compatibility.expected");
|
||||
let mut actual = String::new();
|
||||
for case in fixtures::cases() {
|
||||
let dict = AffixDict::from_hunspell(&case.aff, &case.dic, &["original compatibility fixture"]).unwrap();
|
||||
let result = snapshot(&case, &dict);
|
||||
let restored = AffixDict::from_bytes(&dict.to_bytes().unwrap()).unwrap();
|
||||
assert_eq!(snapshot(&case, &restored), result, "current compact round trip: {}", case.name);
|
||||
actual.push_str(&result);
|
||||
}
|
||||
// craft-libs 55332dd predates this adaptation and supports a prefix plus a two-suffix
|
||||
// chain (native test affix::tests::a_prefix_with_a_suffix_chain). The old Indonesian
|
||||
// fixture's bermain + kan + lah therefore became a word, in these three cases only.
|
||||
// Keep the frozen historical file untouched and assert every explicit transition once.
|
||||
// Normalize only checkout line endings; the frozen fixture and its observations stay unchanged.
|
||||
let mut expected = expected.replace("\r\n", "\n");
|
||||
let mut transitions = 0;
|
||||
for word in ["BERMAINKANLAH", "Bermainkanlah", "bermainkanlah"] {
|
||||
let before = format!("{word:?}: contains=false check=false suggest=false raw_suggest=false\n");
|
||||
let after = if word == "bermainkanlah" {
|
||||
format!("{word:?}: contains=true check=true suggest=true raw_suggest=true\n")
|
||||
} else {
|
||||
format!("{word:?}: contains=false check=true suggest=true raw_suggest=false\n")
|
||||
};
|
||||
assert_eq!(expected.matches(&before).count(), 1, "one historical observable: {word}");
|
||||
expected = expected.replacen(&before, &after, 1);
|
||||
transitions += 1;
|
||||
}
|
||||
assert_eq!(transitions, 3, "the documented upstream exception has exactly three observables");
|
||||
assert_eq!(actual, expected, "shared engine output changed beyond the explicit upstream chain exception");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn legacy_compact_data_is_refused_without_reinterpreting_homonyms() {
|
||||
let encoded = include_str!("fixtures/affix_compatibility.compact");
|
||||
let cases = fixtures::cases();
|
||||
let lines: Vec<_> = encoded.lines().collect();
|
||||
assert_eq!(lines.len(), cases.len());
|
||||
for (case, hex) in cases.iter().zip(lines) {
|
||||
assert_eq!(hex.len() % 2, 0);
|
||||
let bytes: Vec<_> =
|
||||
hex.as_bytes().as_chunks::<2>().0.iter().map(|pair| u8::from_str_radix(std::str::from_utf8(pair).unwrap(), 16).unwrap()).collect();
|
||||
let raw = miniz_oxide::inflate::decompress_to_vec(&bytes).unwrap();
|
||||
assert!(raw.starts_with(b"WCAFF1\n"), "fixture must contain WCAFF1 data: {}", case.name);
|
||||
assert!(AffixDict::from_bytes(&bytes).is_err(), "WCAFF1 must be regenerated as WCAFF2: {}", case.name);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
4d8bc10a825014446b7bbf62a07dd62744e5ae120b2ca2e02aafb8bceac54b295d067e91db76ea7f2516d16666ce30138c47ae3ba41e8c95a35cf884c89caf1c4b282789531ce4112756d1cadf809591a47e6a4bb39107aef2d64d5924e44f3dd42fd8ac4d09ca823c778d101b0c7011853e2d1be696536cf7e9ee57280dcd8dfe2dd4f97b88306faa2a0753b0f0274bea046c6f19dfd989a8a34dec70485d6b3e7c157553da617a03
|
||||
1d8c410e82301045bb9e534ce2deea11a8d8c48d10304196058b990418534aa25ec7852770e9aa17b3b27c79ffbf6a9768bd8515b2a32b8da6c796879bf1d4504ffe811dddfdec2c9c8a1a8d259ec33bbcc2077456a8439aee8f5556a418be90eb33e658e3069dc535941193888c462ae485d5a2a768ffa71244c32379968902d19a290e4174e6699dcc41d0308f17962aa67f
|
||||
1dc7410ac2301005d0ace7141fdc1b3d42aa64a9a15d5497d392ca40eac818416f6fe8eaf1c65388f1483ba8c9439e5c30ebfae22a9314a93f2cf2ad1fcb94e20d09771c60823d0dada15521d06d1d2e6064308dd7fe3c90eb75657233bfd97789dcc2967db3c864ea03fd01
|
||||
1d8a3d0a02311406b77ea7f8c0dee8114437651453ac220a51a33cc88fc408e62c9e451bf55e1bb6199861baf94cca298d10135f39188753f43793f9c88e73c1859ff9912cade4060a5b4c102cc6a4abea410b7687b21f425b0f83df0b86bae57aa1a9f1ce9cbf6fa1a8b9e714bd50b51551f1ffd860444b3d
|
||||
1d8c310e82401444b1dd53fcc45ef408282cb11008ab414b346a3641310413ed0cad95072034f450488c46f10a7f6fe4aeddcc9bcc0b4606a503d28538e15bbe0f2358c5bb4398f2258f787a860d3fa5c7644d268607d8881ba1ae3f1c9ba6e504ae6f0296c471d9ccb62d3605cc8947e780052ca00ff8c1067a842952fd89b8caae6e8c6878c716df3a56a4f3c01abfe222ed998ea5222f6c552e64d64426e7a794d53ae612fc00
|
||||
2dc84b0e8230100050d6738a49dc8b1e0114589800a11864398522c3a768a951120f2f212edf2b4e5e181e618793e13b6b1ab09ac60759963cb05db0e18f7d1905795622c9aa56dc6bb804253edfcad8e54b73dd4016a45877d8419c886b140522c718d2f08629967840a90cee41acf636f7a45dff3ffe3603b5ab8b243b0b702455e47ae08cc4daf552700cb534f37a31fc00
|
||||
35c7b98e82600045e1a9790a92e967dc97d20df71515b51345455114c1ed6d2616c6c2cad8d8fe2f36e67269cec9a7e5328a1296be65db31e7e6666cc9137bbd1dbba66e5aa67b9667e6c9f51c435295813c961b72e8d31f4887746a024da82934a50cc8a066d08c9a43736a012d281332a925b4a456d08ab2208b5a436b6a036d281bb2a92db4a576d08e722087da437bca855cca833cea001da82374a44ed0893a4367ea025da80c94a1b25096ca41392a0fe5a90254a01448a18a50912a4125aa0c95a90a54a1aa5095aa4135aa0ed5a906d4a09a50936a412daa0db5a90ed4a15448a5ba5097ea413daa0ff5290dd2a80134a086d0901a41232a0485a83014a62250848a42512a06c5a83814a71250824a42492a05a5a83494a6c41ff8197df77d0ffcf0fd08fcf4fd0cfcf6fd0e7cf37d0b7cf57d0dfcf2fdfa586b76f2aaf4657bae63ff8a97f475301cddfe0d8523d1583c914ca533d95cbea0144be54ab5566f345bed8edaedf5b5c17034d6275363365f98cb95b5ded8db9db377bdc3f174be883f71153771170ff1142ff196fe01
|
||||
@@ -0,0 +1,497 @@
|
||||
pl: conditional strip, prefix cross products, diacritics
|
||||
len=4 rules=5
|
||||
try="aeoiułkr" key="" map=[['a', 'ą'], ['o', 'ó', 'u']] rep=[("ż", "rz"), ("u", "ó")] comments=["original compatibility fixture"]
|
||||
forms "Warszawa": ["Warszawa", "Warszawą"]
|
||||
forms "kot": ["kot", "kotem", "koty", "niekot", "niekotem", "niekoty"]
|
||||
forms "krowa": ["krowa", "krową"]
|
||||
forms "piesek": ["piesek", "piesekem", "pieseky", "pieska"]
|
||||
"": contains=false check=false suggest=false raw_suggest=false
|
||||
"KOT": contains=false check=true suggest=true raw_suggest=false
|
||||
"KOTEM": contains=false check=true suggest=true raw_suggest=false
|
||||
"KOTY": contains=false check=true suggest=true raw_suggest=false
|
||||
"KOTĄ": contains=false check=false suggest=false raw_suggest=false
|
||||
"KROWA": contains=false check=true suggest=true raw_suggest=false
|
||||
"KROWY": contains=false check=false suggest=false raw_suggest=false
|
||||
"KROWĄ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Kot": contains=false check=true suggest=true raw_suggest=false
|
||||
"Kotem": contains=false check=true suggest=true raw_suggest=false
|
||||
"Koty": contains=false check=true suggest=true raw_suggest=false
|
||||
"Kotą": contains=false check=false suggest=false raw_suggest=false
|
||||
"Krowa": contains=false check=true suggest=true raw_suggest=false
|
||||
"Krowy": contains=false check=false suggest=false raw_suggest=false
|
||||
"Krową": contains=false check=true suggest=true raw_suggest=false
|
||||
"NIE": contains=false check=false suggest=false raw_suggest=false
|
||||
"NIEKOT": contains=false check=true suggest=true raw_suggest=false
|
||||
"NIEKOTEM": contains=false check=true suggest=true raw_suggest=false
|
||||
"NIEKOTY": contains=false check=true suggest=true raw_suggest=false
|
||||
"NIEPIESKA": contains=false check=false suggest=false raw_suggest=false
|
||||
"Nie": contains=false check=false suggest=false raw_suggest=false
|
||||
"Niekot": contains=false check=true suggest=true raw_suggest=false
|
||||
"Niekotem": contains=false check=true suggest=true raw_suggest=false
|
||||
"Niekoty": contains=false check=true suggest=true raw_suggest=false
|
||||
"Niepieska": contains=false check=false suggest=false raw_suggest=false
|
||||
"PIESEK": contains=false check=true suggest=true raw_suggest=false
|
||||
"PIESEKA": contains=false check=false suggest=false raw_suggest=false
|
||||
"PIESEKEM": contains=false check=true suggest=true raw_suggest=false
|
||||
"PIESEKY": contains=false check=true suggest=true raw_suggest=false
|
||||
"PIESKA": contains=false check=true suggest=true raw_suggest=false
|
||||
"Piesek": contains=false check=true suggest=true raw_suggest=false
|
||||
"Pieseka": contains=false check=false suggest=false raw_suggest=false
|
||||
"Piesekem": contains=false check=true suggest=true raw_suggest=false
|
||||
"Pieseky": contains=false check=true suggest=true raw_suggest=false
|
||||
"Pieska": contains=false check=true suggest=true raw_suggest=false
|
||||
"WARSZAWA": contains=false check=true suggest=true raw_suggest=false
|
||||
"WARSZAWĄ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Warszawa": contains=true check=true suggest=true raw_suggest=true
|
||||
"Warszawą": contains=true check=true suggest=true raw_suggest=true
|
||||
"kot": contains=true check=true suggest=true raw_suggest=true
|
||||
"kotem": contains=true check=true suggest=true raw_suggest=true
|
||||
"koty": contains=true check=true suggest=true raw_suggest=true
|
||||
"kotą": contains=false check=false suggest=false raw_suggest=false
|
||||
"krowa": contains=true check=true suggest=true raw_suggest=true
|
||||
"krowy": contains=false check=false suggest=false raw_suggest=false
|
||||
"krową": contains=true check=true suggest=true raw_suggest=true
|
||||
"nie": contains=false check=false suggest=false raw_suggest=false
|
||||
"niekot": contains=true check=true suggest=true raw_suggest=true
|
||||
"niekotem": contains=true check=true suggest=true raw_suggest=true
|
||||
"niekoty": contains=true check=true suggest=true raw_suggest=true
|
||||
"niepieska": contains=false check=false suggest=false raw_suggest=false
|
||||
"piesek": contains=true check=true suggest=true raw_suggest=true
|
||||
"pieseka": contains=false check=false suggest=false raw_suggest=false
|
||||
"piesekem": contains=true check=true suggest=true raw_suggest=true
|
||||
"pieseky": contains=true check=true suggest=true raw_suggest=true
|
||||
"pieska": contains=true check=true suggest=true raw_suggest=true
|
||||
"warszawa": contains=false check=false suggest=false raw_suggest=false
|
||||
"warszawą": contains=false check=false suggest=false raw_suggest=false
|
||||
pt: UTF-8 flags, forbidden words, two suffixes
|
||||
len=4 rules=3
|
||||
try="aeiouçãõ" key="" map=[] rep=[] comments=["original compatibility fixture"]
|
||||
forms "bonito": ["bonita", "bonitas", "bonito", "bonitos"]
|
||||
forms "casa": ["casa", "casas"]
|
||||
forms "fazer": ["fazer", "refazer"]
|
||||
forms "imundo": []
|
||||
"BONITA": contains=false check=true suggest=true raw_suggest=false
|
||||
"BONITAS": contains=false check=true suggest=true raw_suggest=false
|
||||
"BONITASS": contains=false check=false suggest=false raw_suggest=false
|
||||
"BONITO": contains=false check=true suggest=true raw_suggest=false
|
||||
"BONITOS": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bonita": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bonitas": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bonitass": contains=false check=false suggest=false raw_suggest=false
|
||||
"Bonito": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bonitos": contains=false check=true suggest=true raw_suggest=false
|
||||
"CASA": contains=false check=true suggest=true raw_suggest=false
|
||||
"CASAA": contains=false check=false suggest=false raw_suggest=false
|
||||
"CASAS": contains=false check=true suggest=true raw_suggest=false
|
||||
"Casa": contains=false check=true suggest=true raw_suggest=false
|
||||
"Casaa": contains=false check=false suggest=false raw_suggest=false
|
||||
"Casas": contains=false check=true suggest=true raw_suggest=false
|
||||
"FAZER": contains=false check=true suggest=true raw_suggest=false
|
||||
"Fazer": contains=false check=true suggest=true raw_suggest=false
|
||||
"IMUNDO": contains=false check=false suggest=false raw_suggest=false
|
||||
"IMUNDOS": contains=false check=false suggest=false raw_suggest=false
|
||||
"Imundo": contains=false check=false suggest=false raw_suggest=false
|
||||
"Imundos": contains=false check=false suggest=false raw_suggest=false
|
||||
"RECASAS": contains=false check=false suggest=false raw_suggest=false
|
||||
"REFAZER": contains=false check=true suggest=true raw_suggest=false
|
||||
"Recasas": contains=false check=false suggest=false raw_suggest=false
|
||||
"Refazer": contains=false check=true suggest=true raw_suggest=false
|
||||
"bonita": contains=true check=true suggest=true raw_suggest=true
|
||||
"bonitas": contains=true check=true suggest=true raw_suggest=true
|
||||
"bonitass": contains=false check=false suggest=false raw_suggest=false
|
||||
"bonito": contains=true check=true suggest=true raw_suggest=true
|
||||
"bonitos": contains=true check=true suggest=true raw_suggest=true
|
||||
"casa": contains=true check=true suggest=true raw_suggest=true
|
||||
"casaa": contains=false check=false suggest=false raw_suggest=false
|
||||
"casas": contains=true check=true suggest=true raw_suggest=true
|
||||
"fazer": contains=true check=true suggest=true raw_suggest=true
|
||||
"imundo": contains=false check=false suggest=false raw_suggest=false
|
||||
"imundos": contains=false check=false suggest=false raw_suggest=false
|
||||
"recasas": contains=false check=false suggest=false raw_suggest=false
|
||||
"refazer": contains=true check=true suggest=true raw_suggest=true
|
||||
it: stripping, non-crossing suffix, proper-name casing
|
||||
len=4 rules=3
|
||||
try="" key="" map=[] rep=[] comments=["original compatibility fixture"]
|
||||
forms "Roma": ["Roma"]
|
||||
forms "casa": ["casa", "case", "ricasa"]
|
||||
forms "fare": ["fare", "rifare"]
|
||||
forms "libro": ["libri", "libro"]
|
||||
"": contains=false check=false suggest=false raw_suggest=false
|
||||
"CASA": contains=false check=true suggest=true raw_suggest=false
|
||||
"CASE": contains=false check=true suggest=true raw_suggest=false
|
||||
"Casa": contains=false check=true suggest=true raw_suggest=false
|
||||
"Case": contains=false check=true suggest=true raw_suggest=false
|
||||
"FARE": contains=false check=true suggest=true raw_suggest=false
|
||||
"Fare": contains=false check=true suggest=true raw_suggest=false
|
||||
"LIBRI": contains=false check=true suggest=true raw_suggest=false
|
||||
"LIBRO": contains=false check=true suggest=true raw_suggest=false
|
||||
"LIBROS": contains=false check=false suggest=false raw_suggest=false
|
||||
"Libri": contains=false check=true suggest=true raw_suggest=false
|
||||
"Libro": contains=false check=true suggest=true raw_suggest=false
|
||||
"Libros": contains=false check=false suggest=false raw_suggest=false
|
||||
"R ILIBRI": contains=false check=false suggest=false raw_suggest=false
|
||||
"R ilibri": contains=false check=false suggest=false raw_suggest=false
|
||||
"RICAS A": contains=false check=false suggest=false raw_suggest=false
|
||||
"RICASA": contains=false check=true suggest=true raw_suggest=false
|
||||
"RICASE": contains=false check=false suggest=false raw_suggest=false
|
||||
"RIFARE": contains=false check=true suggest=true raw_suggest=false
|
||||
"ROMA": contains=false check=true suggest=true raw_suggest=false
|
||||
"Ricas a": contains=false check=false suggest=false raw_suggest=false
|
||||
"Ricasa": contains=false check=true suggest=true raw_suggest=false
|
||||
"Ricase": contains=false check=false suggest=false raw_suggest=false
|
||||
"Rifare": contains=false check=true suggest=true raw_suggest=false
|
||||
"Roma": contains=true check=true suggest=true raw_suggest=true
|
||||
"casa": contains=true check=true suggest=true raw_suggest=true
|
||||
"case": contains=true check=true suggest=true raw_suggest=true
|
||||
"fare": contains=true check=true suggest=true raw_suggest=true
|
||||
"libri": contains=true check=true suggest=true raw_suggest=true
|
||||
"libro": contains=true check=true suggest=true raw_suggest=true
|
||||
"libros": contains=false check=false suggest=false raw_suggest=false
|
||||
"r ilibri": contains=false check=false suggest=false raw_suggest=false
|
||||
"ricas a": contains=false check=false suggest=false raw_suggest=false
|
||||
"ricasa": contains=true check=true suggest=true raw_suggest=true
|
||||
"ricase": contains=false check=false suggest=false raw_suggest=false
|
||||
"rifare": contains=true check=true suggest=true raw_suggest=true
|
||||
"roma": contains=false check=false suggest=false raw_suggest=false
|
||||
cs: negated conditions and shared Unicode stems
|
||||
len=4 rules=3
|
||||
try="" key="" map=[] rep=[] comments=["original compatibility fixture"]
|
||||
forms "mladý": ["mladý", "nemladý"]
|
||||
forms "strom": ["nestrom", "nestromy", "strom", "stromy"]
|
||||
forms "y": ["y"]
|
||||
forms "žena": ["žena", "ženě"]
|
||||
"MLADÝ": contains=false check=true suggest=true raw_suggest=false
|
||||
"MLADÝY": contains=false check=false suggest=false raw_suggest=false
|
||||
"Mladý": contains=false check=true suggest=true raw_suggest=false
|
||||
"Mladýy": contains=false check=false suggest=false raw_suggest=false
|
||||
"NEMLADÝ": contains=false check=true suggest=true raw_suggest=false
|
||||
"NESTROM": contains=false check=true suggest=true raw_suggest=false
|
||||
"NESTROMY": contains=false check=true suggest=true raw_suggest=false
|
||||
"NEŽENĚ": contains=false check=false suggest=false raw_suggest=false
|
||||
"Nemladý": contains=false check=true suggest=true raw_suggest=false
|
||||
"Nestrom": contains=false check=true suggest=true raw_suggest=false
|
||||
"Nestromy": contains=false check=true suggest=true raw_suggest=false
|
||||
"Neženě": contains=false check=false suggest=false raw_suggest=false
|
||||
"STROM": contains=false check=true suggest=true raw_suggest=false
|
||||
"STROMY": contains=false check=true suggest=true raw_suggest=false
|
||||
"Strom": contains=false check=true suggest=true raw_suggest=false
|
||||
"Stromy": contains=false check=true suggest=true raw_suggest=false
|
||||
"Y": contains=false check=true suggest=true raw_suggest=false
|
||||
"YY": contains=false check=false suggest=false raw_suggest=false
|
||||
"Yy": contains=false check=false suggest=false raw_suggest=false
|
||||
"mladý": contains=true check=true suggest=true raw_suggest=true
|
||||
"mladýy": contains=false check=false suggest=false raw_suggest=false
|
||||
"nemladý": contains=true check=true suggest=true raw_suggest=true
|
||||
"nestrom": contains=true check=true suggest=true raw_suggest=true
|
||||
"nestromy": contains=true check=true suggest=true raw_suggest=true
|
||||
"neženě": contains=false check=false suggest=false raw_suggest=false
|
||||
"strom": contains=true check=true suggest=true raw_suggest=true
|
||||
"stromy": contains=true check=true suggest=true raw_suggest=true
|
||||
"y": contains=true check=true suggest=true raw_suggest=true
|
||||
"yy": contains=false check=false suggest=false raw_suggest=false
|
||||
"ŽENA": contains=false check=true suggest=true raw_suggest=false
|
||||
"ŽENY": contains=false check=false suggest=false raw_suggest=false
|
||||
"ŽENĚ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Žena": contains=false check=true suggest=true raw_suggest=false
|
||||
"Ženy": contains=false check=false suggest=false raw_suggest=false
|
||||
"Ženě": contains=false check=true suggest=true raw_suggest=false
|
||||
"žena": contains=true check=true suggest=true raw_suggest=true
|
||||
"ženy": contains=false check=false suggest=false raw_suggest=false
|
||||
"ženě": contains=true check=true suggest=true raw_suggest=true
|
||||
ru: Cyrillic flags and suggestions exclusion
|
||||
len=4 rules=2
|
||||
try="" key="" map=[['е', 'ё']] rep=[] comments=["original compatibility fixture"]
|
||||
forms "дом": ["дом", "домы"]
|
||||
forms "запрет": []
|
||||
forms "кот": ["кот", "коты", "некот", "некоты"]
|
||||
forms "тайна": ["тайна", "тайнаы"]
|
||||
"ДОМ": contains=false check=true suggest=true raw_suggest=false
|
||||
"ДОМЫ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Дом": contains=false check=true suggest=true raw_suggest=false
|
||||
"Домы": contains=false check=true suggest=true raw_suggest=false
|
||||
"ЗАПРЕТ": contains=false check=false suggest=false raw_suggest=false
|
||||
"ЗАПРЕТЫ": contains=false check=false suggest=false raw_suggest=false
|
||||
"Запрет": contains=false check=false suggest=false raw_suggest=false
|
||||
"Запреты": contains=false check=false suggest=false raw_suggest=false
|
||||
"КОТ": contains=false check=true suggest=true raw_suggest=false
|
||||
"КОТЫ": contains=false check=true suggest=true raw_suggest=false
|
||||
"КОт": contains=false check=false suggest=false raw_suggest=false
|
||||
"Кот": contains=false check=true suggest=true raw_suggest=false
|
||||
"Коты": contains=false check=true suggest=true raw_suggest=false
|
||||
"НЕДОМЫ": contains=false check=false suggest=false raw_suggest=false
|
||||
"НЕКОТ": contains=false check=true suggest=true raw_suggest=false
|
||||
"НЕКОТЫ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Недомы": contains=false check=false suggest=false raw_suggest=false
|
||||
"Некот": contains=false check=true suggest=true raw_suggest=false
|
||||
"Некоты": contains=false check=true suggest=true raw_suggest=false
|
||||
"ТАЙНА": contains=false check=true suggest=false raw_suggest=false
|
||||
"ТАЙНАЫ": contains=false check=true suggest=false raw_suggest=false
|
||||
"Тайна": contains=false check=true suggest=false raw_suggest=false
|
||||
"Тайнаы": contains=false check=true suggest=false raw_suggest=false
|
||||
"дом": contains=true check=true suggest=true raw_suggest=true
|
||||
"домы": contains=true check=true suggest=true raw_suggest=true
|
||||
"запрет": contains=false check=false suggest=false raw_suggest=false
|
||||
"запреты": contains=false check=false suggest=false raw_suggest=false
|
||||
"кОт": contains=false check=false suggest=false raw_suggest=false
|
||||
"кот": contains=true check=true suggest=true raw_suggest=true
|
||||
"коты": contains=true check=true suggest=true raw_suggest=true
|
||||
"недомы": contains=false check=false suggest=false raw_suggest=false
|
||||
"некот": contains=true check=true suggest=true raw_suggest=true
|
||||
"некоты": contains=true check=true suggest=true raw_suggest=true
|
||||
"тайна": contains=true check=true suggest=false raw_suggest=false
|
||||
"тайнаы": contains=true check=true suggest=false raw_suggest=false
|
||||
id: continuation and NOSUGGEST through derivations
|
||||
len=3 rules=3
|
||||
try="abcdeikn" key="qwerty|asdf" map=[] rep=[("dj", "j")] comments=["original compatibility fixture"]
|
||||
forms "baca": ["baca", "bacakan", "bacakanlah"]
|
||||
forms "main": ["bermain", "bermainkan", "main", "mainkan", "mainkanlah"]
|
||||
forms "rahasia": ["rahasia", "rahasiakan", "rahasiakanlah"]
|
||||
"BACA": contains=false check=true suggest=true raw_suggest=false
|
||||
"BACAKAN": contains=false check=true suggest=true raw_suggest=false
|
||||
"BACAKANLAH": contains=false check=true suggest=true raw_suggest=false
|
||||
"BERBACA": contains=false check=false suggest=false raw_suggest=false
|
||||
"BERMAIN": contains=false check=true suggest=true raw_suggest=false
|
||||
"BERMAINKAN": contains=false check=true suggest=true raw_suggest=false
|
||||
"BERMAINKANLAH": contains=false check=false suggest=false raw_suggest=false
|
||||
"Baca": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bacakan": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bacakanlah": contains=false check=true suggest=true raw_suggest=false
|
||||
"Berbaca": contains=false check=false suggest=false raw_suggest=false
|
||||
"Bermain": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bermainkan": contains=false check=true suggest=true raw_suggest=false
|
||||
"Bermainkanlah": contains=false check=false suggest=false raw_suggest=false
|
||||
"MAIN": contains=false check=true suggest=true raw_suggest=false
|
||||
"MAINKAN": contains=false check=true suggest=true raw_suggest=false
|
||||
"MAINKANLAH": contains=false check=true suggest=true raw_suggest=false
|
||||
"MAINKANLAHLAH": contains=false check=false suggest=false raw_suggest=false
|
||||
"MAINLAH": contains=false check=false suggest=false raw_suggest=false
|
||||
"Main": contains=false check=true suggest=true raw_suggest=false
|
||||
"Mainkan": contains=false check=true suggest=true raw_suggest=false
|
||||
"Mainkanlah": contains=false check=true suggest=true raw_suggest=false
|
||||
"Mainkanlahlah": contains=false check=false suggest=false raw_suggest=false
|
||||
"Mainlah": contains=false check=false suggest=false raw_suggest=false
|
||||
"RAHASIA": contains=false check=true suggest=false raw_suggest=false
|
||||
"RAHASIAKAN": contains=false check=true suggest=false raw_suggest=false
|
||||
"RAHASIAKANLAH": contains=false check=true suggest=false raw_suggest=false
|
||||
"Rahasia": contains=false check=true suggest=false raw_suggest=false
|
||||
"Rahasiakan": contains=false check=true suggest=false raw_suggest=false
|
||||
"Rahasiakanlah": contains=false check=true suggest=false raw_suggest=false
|
||||
"baca": contains=true check=true suggest=true raw_suggest=true
|
||||
"bacakan": contains=true check=true suggest=true raw_suggest=true
|
||||
"bacakanlah": contains=true check=true suggest=true raw_suggest=true
|
||||
"berbaca": contains=false check=false suggest=false raw_suggest=false
|
||||
"bermain": contains=true check=true suggest=true raw_suggest=true
|
||||
"bermainkan": contains=true check=true suggest=true raw_suggest=true
|
||||
"bermainkanlah": contains=false check=false suggest=false raw_suggest=false
|
||||
"main": contains=true check=true suggest=true raw_suggest=true
|
||||
"mainkan": contains=true check=true suggest=true raw_suggest=true
|
||||
"mainkanlah": contains=true check=true suggest=true raw_suggest=true
|
||||
"mainkanlahlah": contains=false check=false suggest=false raw_suggest=false
|
||||
"mainlah": contains=false check=false suggest=false raw_suggest=false
|
||||
"rahasia": contains=true check=true suggest=false raw_suggest=false
|
||||
"rahasiakan": contains=true check=true suggest=false raw_suggest=false
|
||||
"rahasiakanlah": contains=true check=true suggest=false raw_suggest=false
|
||||
pt-sized: 70 distinct flags across the old u128 boundary
|
||||
len=2 rules=70
|
||||
try="" key="" map=[] rep=[] comments=["original compatibility fixture"]
|
||||
forms "outro": ["outro", "outroõ"]
|
||||
forms "verbo": ["verbo", "verbo1", "verbo2", "verbo3", "verbo4", "verbo5", "verbo6", "verbo7", "verbo8", "verbo9", "verboA", "verboB", "verboC", "verboD", "verboE", "verboF", "verboG", "verboH", "verboI", "verboJ", "verboK", "verboL", "verboM", "verboN", "verboO", "verboP", "verboQ", "verboR", "verboS", "verboT", "verboU", "verboV", "verboW", "verboX", "verboY", "verboZ", "verboa", "verbob", "verboc", "verbod", "verboe", "verbof", "verbog", "verboh", "verboi", "verboj", "verbok", "verbol", "verbom", "verbon", "verboo", "verbop", "verboq", "verbor", "verbos", "verbot", "verbou", "verbov", "verbow", "verbox", "verboy", "verboz", "verboá", "verboã", "verboç", "verboé", "verboí", "verboó", "verboõ", "verboú"]
|
||||
"OUTRO": contains=false check=true suggest=true raw_suggest=false
|
||||
"OUTROA": contains=false check=false suggest=false raw_suggest=false
|
||||
"OUTROÕ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Outro": contains=false check=true suggest=true raw_suggest=false
|
||||
"Outroa": contains=false check=false suggest=false raw_suggest=false
|
||||
"Outroõ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO0": contains=false check=false suggest=false raw_suggest=false
|
||||
"VERBO1": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO2": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO3": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO4": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO5": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO6": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO7": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO8": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBO9": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOA": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOB": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOC": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOD": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOE": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOF": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOG": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOH": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOI": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOJ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOK": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOL": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOM": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBON": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOO": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOP": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOQ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOR": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOS": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOSS": contains=false check=false suggest=false raw_suggest=false
|
||||
"VERBOT": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOU": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOV": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOW": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOX": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOY": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOZ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÁ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÃ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÇ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÉ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÍ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÓ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÕ": contains=false check=true suggest=true raw_suggest=false
|
||||
"VERBOÚ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo0": contains=false check=false suggest=false raw_suggest=false
|
||||
"Verbo1": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo2": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo3": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo4": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo5": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo6": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo7": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo8": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbo9": contains=false check=true suggest=true raw_suggest=false
|
||||
"VerboA": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboB": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboC": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboD": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboE": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboF": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboG": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboH": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboI": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboJ": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboK": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboL": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboM": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboN": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboO": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboP": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboQ": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboR": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboS": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboT": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboU": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboV": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboW": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboX": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboY": contains=false check=false suggest=false raw_suggest=false
|
||||
"VerboZ": contains=false check=false suggest=false raw_suggest=false
|
||||
"Verboa": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbob": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboc": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbod": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboe": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbof": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbog": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboh": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboi": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboj": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbok": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbol": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbom": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbon": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboo": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbop": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboq": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbor": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbos": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbot": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbou": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbov": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbow": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verbox": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboy": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboz": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboß": contains=false check=false suggest=false raw_suggest=false
|
||||
"Verboá": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboã": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboç": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboé": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboí": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboó": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboõ": contains=false check=true suggest=true raw_suggest=false
|
||||
"Verboú": contains=false check=true suggest=true raw_suggest=false
|
||||
"outro": contains=true check=true suggest=true raw_suggest=true
|
||||
"outroa": contains=false check=false suggest=false raw_suggest=false
|
||||
"outroõ": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo0": contains=false check=false suggest=false raw_suggest=false
|
||||
"verbo1": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo2": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo3": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo4": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo5": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo6": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo7": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo8": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbo9": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboA": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboB": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboC": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboD": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboE": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboF": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboG": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboH": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboI": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboJ": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboK": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboL": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboM": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboN": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboO": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboP": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboQ": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboR": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboS": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboT": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboU": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboV": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboW": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboX": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboY": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboZ": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboa": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbob": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboc": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbod": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboe": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbof": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbog": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboh": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboi": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboj": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbok": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbol": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbom": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbon": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboo": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbop": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboq": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbor": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbos": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbot": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbou": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbov": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbow": contains=true check=true suggest=true raw_suggest=true
|
||||
"verbox": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboy": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboz": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboß": contains=false check=false suggest=false raw_suggest=false
|
||||
"verboá": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboã": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboç": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboé": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboí": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboó": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboõ": contains=true check=true suggest=true raw_suggest=true
|
||||
"verboú": contains=true check=true suggest=true raw_suggest=true
|
||||
@@ -0,0 +1,61 @@
|
||||
// Original, deliberately small fixtures for behavior supported before French proofing.
|
||||
// These are not extracts from installed dictionaries and do not claim full language coverage.
|
||||
pub struct Case {
|
||||
pub name: &'static str,
|
||||
pub aff: String,
|
||||
pub dic: String,
|
||||
pub probes: Vec<&'static str>,
|
||||
}
|
||||
|
||||
pub fn cases() -> Vec<Case> {
|
||||
let mut out = vec![
|
||||
Case {
|
||||
name: "pl: conditional strip, prefix cross products, diacritics",
|
||||
aff: "SET UTF-8\nTRY aeoiułkr\nPFX b Y 1\nPFX b 0 nie .\nSFX a Y 3\nSFX a 0 y [^y]\nSFX a ek ka ek\nSFX a 0 em .\nSFX c N 1\nSFX c a ą a\nMAP 2\nMAP aą\nMAP oóu\nREP 2\nREP ż rz\nREP u ó\n".into(),
|
||||
dic: "4\nkot/ab\npiesek/a\nkrowa/c\nWarszawa/c\n".into(),
|
||||
probes: vec!["kot", "koty", "kotem", "niekoty", "pieska", "pieseka", "niepieska", "krową", "krowy", "Warszawą", "warszawą", "kotą", "nie", ""],
|
||||
},
|
||||
Case {
|
||||
name: "pt: UTF-8 flags, forbidden words, two suffixes",
|
||||
aff: "SET UTF-8\nFLAG UTF-8\nFORBIDDENWORD ý\nTRY aeiouçãõ\nSFX A Y 1\nSFX A o a/B o\nSFX B Y 1\nSFX B 0 s .\nPFX P Y 1\nPFX P 0 re .\n".into(),
|
||||
dic: "4\nbonito/AB\ncasa/B\nfazer/P\nimundo/ýB\n".into(),
|
||||
probes: vec!["bonito", "bonita", "bonitos", "bonitas", "bonitass", "casa", "casas", "casaa", "refazer", "recasas", "imundo", "imundos", "Imundo", "IMUNDO"],
|
||||
},
|
||||
Case {
|
||||
name: "it: stripping, non-crossing suffix, proper-name casing",
|
||||
aff: "SET UTF-8\nSFX A Y 1\nSFX A o i o\nSFX B N 1\nSFX B a e a\nPFX P Y 1\nPFX P 0 ri .\n".into(),
|
||||
dic: "4\nlibro/A\ncasa/BP\nfare/P\nRoma\n".into(),
|
||||
probes: vec!["libri", "libros", "case", "ricas a", "ricasa", "ricase", "rifare", "r ilibri", "Roma", "roma", "ROMA", ""],
|
||||
},
|
||||
Case {
|
||||
name: "cs: negated conditions and shared Unicode stems",
|
||||
aff: "SET UTF-8\nPFX N Y 1\nPFX N 0 ne .\nSFX S Y 1\nSFX S 0 y [^y]\nSFX E N 1\nSFX E a ě a\n".into(),
|
||||
dic: "4\nstrom/NS\nžena/E\nmladý/N\ny/S\n".into(),
|
||||
probes: vec!["stromy", "nestromy", "nestrom", "ženě", "ženy", "neženě", "nemladý", "mladýy", "yy", "Ženě", "STROMY"],
|
||||
},
|
||||
Case {
|
||||
name: "ru: Cyrillic flags and suggestions exclusion",
|
||||
aff: "SET UTF-8\nFLAG UTF-8\nNOSUGGEST Н\nFORBIDDENWORD Ф\nSFX Я Y 1\nSFX Я 0 ы .\nPFX П Y 1\nPFX П 0 не .\nMAP 1\nMAP её\n".into(),
|
||||
dic: "4\nкот/ЯП\nдом/Я\nтайна/НЯ\nзапрет/ФЯ\n".into(),
|
||||
probes: vec!["кот", "коты", "некоты", "домы", "недомы", "тайна", "тайнаы", "ТАЙНАЫ", "запрет", "запреты", "КОТЫ", "кОт"],
|
||||
},
|
||||
Case {
|
||||
name: "id: continuation and NOSUGGEST through derivations",
|
||||
aff: "SET UTF-8\nNOSUGGEST N\nKEY qwerty|asdf\nTRY abcdeikn\nREP 1\nREP dj j\nPFX P Y 1\nPFX P 0 ber .\nSFX A Y 1\nSFX A 0 kan/B .\nSFX B Y 1\nSFX B 0 lah .\n".into(),
|
||||
dic: "3\nmain/PA\nbaca/A\nrahasia/NA\n".into(),
|
||||
probes: vec!["main", "bermain", "mainkan", "mainkanlah", "bermainkanlah", "mainlah", "mainkanlahlah", "bacakanlah", "berbaca", "rahasia", "rahasiakan", "RAHASIAKANLAH"],
|
||||
},
|
||||
];
|
||||
let flags: Vec<char> = ('a'..='z').chain('A'..='Z').chain('0'..='9').chain(['á', 'é', 'í', 'ó', 'ú', 'ç', 'ã', 'õ']).collect();
|
||||
let mut aff = String::from("SET UTF-8\nFLAG UTF-8\n");
|
||||
for flag in &flags {
|
||||
aff.push_str(&format!("SFX {flag} N 1\nSFX {flag} 0 {flag} .\n"));
|
||||
}
|
||||
out.push(Case {
|
||||
name: "pt-sized: 70 distinct flags across the old u128 boundary",
|
||||
aff,
|
||||
dic: format!("2\nverbo/{}\noutro/õ\n", flags.iter().collect::<String>()),
|
||||
probes: vec!["verboa", "verboz", "verboA", "verboZ", "verbo0", "verbo9", "verboç", "verboõ", "verboß", "outroõ", "outroa"],
|
||||
});
|
||||
out
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
//! Explicit manual real-dictionary gate. Verify the pinned runtime inputs as documented in
|
||||
//! crates/proof/README.md before running with `--ignored`. Missing data is a failure,
|
||||
//! never a silently successful test that checked nothing.
|
||||
|
||||
#![cfg(not(target_arch = "wasm32"))]
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used)]
|
||||
|
||||
use craft_proof::{IssueKind, Options, ProofLang, fr};
|
||||
|
||||
#[test]
|
||||
#[ignore = "requires a real French dictionary in WORDCRAFT_DICTIONARIES"]
|
||||
fn real_french_dictionary_spelling_suggestions_and_dispatch() {
|
||||
let directory = std::env::var_os("WORDCRAFT_DICTIONARIES").expect("set WORDCRAFT_DICTIONARIES to the test dictionary folder");
|
||||
craft_proof::set_dictionaries_folder(Some(std::path::PathBuf::from(&directory)));
|
||||
let french = ProofLang::from_tag(Some("fr-FR"));
|
||||
let installed = craft_proof::pack_dictionary_file(french).expect("the real French dictionary must be installed");
|
||||
assert_eq!(installed.parent(), Some(std::path::Path::new(&directory)), "do not silently use another system dictionary");
|
||||
let aff = installed.with_extension("aff");
|
||||
let dictionary = craft_proof::hunspell::load(&aff, &installed).expect("the real French dictionary loads");
|
||||
assert!(dictionary.len() > 50_000, "expected Dicollecte, not the synthetic fixture: {}", dictionary.len());
|
||||
for word in [
|
||||
"maison",
|
||||
"maisons",
|
||||
"Paris",
|
||||
"mangeons",
|
||||
"finissaient",
|
||||
"étaient",
|
||||
"l'arbre",
|
||||
"l’arbre",
|
||||
"qu'elles",
|
||||
"école",
|
||||
"œuvre",
|
||||
"aujourd'hui",
|
||||
"chevaux",
|
||||
"heureuse",
|
||||
"va-t-il",
|
||||
] {
|
||||
assert!(fr::is_correct(word), "correct French word: {word}");
|
||||
}
|
||||
for (typo, expected) in [("maisonn", "maison"), ("ecole", "école"), ("l’ecole", "l’école"), ("documant", "document")] {
|
||||
assert!(!fr::is_correct(typo), "misspelling: {typo}");
|
||||
let suggestions = fr::suggest(typo, 8);
|
||||
assert!(suggestions.iter().any(|s| s == expected), "{typo} -> {suggestions:?}, expected {expected}");
|
||||
}
|
||||
let text = "Une maisonn près de l’école.";
|
||||
for lang in [ProofLang::from_tag(Some("fr-FR")), ProofLang::from_tag(Some("fr-CA"))] {
|
||||
let issues = craft_proof::check_text(text, &[(0..text.len(), lang)]);
|
||||
assert_eq!(issues.len(), 1, "{lang:?}: {issues:?}");
|
||||
let issue = &issues[0];
|
||||
assert_eq!(&text[issue.start..issue.end], "maisonn");
|
||||
assert_eq!(issue.lang, lang);
|
||||
assert_eq!(issue.message, "Faute d’orthographe possible");
|
||||
}
|
||||
check_combining_marks_and_compounds(&dictionary);
|
||||
assert!(craft_proof::dictionary_load_error(ProofLang::from_tag(Some("fr-FR"))).is_none());
|
||||
}
|
||||
|
||||
fn check_combining_marks_and_compounds(dictionary: &craft_proof::affix::AffixDict) {
|
||||
let spelling = |text: &str, spans: &[(std::ops::Range<usize>, ProofLang)]| {
|
||||
craft_proof::check_text_with(text, spans, &Options::default(), false, false)
|
||||
.into_iter()
|
||||
.filter(|i| i.kind == IssueKind::Spelling)
|
||||
.map(|i| (i.start, i.end, i.lang))
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
let mut failures = Vec::new();
|
||||
for lang in [ProofLang::from_tag(Some("fr-FR")), ProofLang::from_tag(Some("fr-CA"))] {
|
||||
for text in
|
||||
["école", "e\u{301}cole", "l’école", "l’e\u{301}cole", "café", "cafe\u{301}", "porte-monnaie", "va-t-il", "a-t-il", "maison-école"]
|
||||
{
|
||||
assert!(
|
||||
fr::is_correct_with(dictionary, text) || ["va-t-il", "a-t-il", "maison-école"].contains(&text),
|
||||
"real dictionary recognizes {text:?}"
|
||||
);
|
||||
let issues = spelling(text, &[(0..text.len(), lang)]);
|
||||
if !issues.is_empty() {
|
||||
failures.push(format!("correct {lang:?} {text:?}: {issues:?}"));
|
||||
}
|
||||
}
|
||||
for text in ["porte--monnaie", "va--t-il", "maison--école", "e\u{301}colllle", "cafffe\u{301}"] {
|
||||
assert!(!fr::is_correct_with(dictionary, text), "the explicit dictionary entry must not already accept {text:?}");
|
||||
let issues = spelling(text, &[(0..text.len(), lang)]);
|
||||
let expected = vec![(0, text.len(), lang)];
|
||||
if issues != expected {
|
||||
failures.push(format!("typo {lang:?} {text:?}: {issues:?}, wanted {expected:?}"));
|
||||
}
|
||||
}
|
||||
let text = "e\u{301}cole cafe\u{301} xyzxyz porte--monnaie";
|
||||
let foreign = text.find("xyzxyz").unwrap();
|
||||
let compound = text.find("porte").unwrap();
|
||||
let spans = [(0..foreign, lang), (foreign..compound, ProofLang::Other), (compound..text.len(), lang)];
|
||||
let issues = spelling(text, &spans);
|
||||
if issues != [(compound, text.len(), lang)] {
|
||||
failures.push(format!("mixed {lang:?}: {issues:?}"));
|
||||
}
|
||||
assert!(spelling(text, &[(0..text.len(), ProofLang::Other)]).is_empty());
|
||||
// An English word remains in its own dictionary next to the original French bytes.
|
||||
let text = "e\u{301}cole hello cafe\u{301}";
|
||||
let en = text.find("hello").unwrap();
|
||||
let spans = [(0..en, lang), (en..en + 5, ProofLang::En), (en + 5..text.len(), lang)];
|
||||
let issues = spelling(text, &spans);
|
||||
if !issues.is_empty() {
|
||||
failures.push(format!("English boundary {lang:?}: {issues:?}"));
|
||||
}
|
||||
}
|
||||
assert!(failures.is_empty(), "{}", failures.join("\n"));
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
//! French uses ordinary Language packs and keeps each regional locale at the public API.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used)]
|
||||
|
||||
use craft_proof::affix;
|
||||
use craft_proof::{IssueKind, Options, ProofLang, check_text_with, is_correct_in_with, suggest_in};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
#[path = "../src/fr/test_dict.rs"]
|
||||
mod fixture;
|
||||
|
||||
struct Tools(PathBuf);
|
||||
impl Drop for Tools {
|
||||
fn drop(&mut self) {
|
||||
craft_proof::set_dictionaries_folder(None);
|
||||
craft_proof::packs::set_packs_dir(None);
|
||||
craft_proof::remove_word("zxqrechargefr");
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn pair(dir: &Path, name: &str, aff: &str, dic: &str) {
|
||||
std::fs::create_dir_all(dir).unwrap();
|
||||
std::fs::write(dir.join(format!("{name}.aff")), aff).unwrap();
|
||||
std::fs::write(dir.join(format!("{name}.dic")), dic).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn french_packs_regions_reload_and_original_ranges() {
|
||||
assert!(!fixture::dictionary().is_empty());
|
||||
let root = std::env::temp_dir().join(format!("craft-fr-packs-{}", std::process::id()));
|
||||
let _tools = Tools(root.clone());
|
||||
let pack = root.join("packs/fr");
|
||||
pair(&pack, "spelling", fixture::AFF, fixture::DIC);
|
||||
std::fs::write(pack.join("pack.toml"), "locale = 'fr'\nname = 'Français'\nlicence = 'MIT'\n").unwrap();
|
||||
std::fs::write(pack.join("hyphenation.pat"), "ma1ison\n").unwrap();
|
||||
let folder = root.join("dictionaries");
|
||||
std::fs::create_dir_all(&folder).unwrap();
|
||||
craft_proof::packs::set_packs_dir(Some(root.join("packs")));
|
||||
craft_proof::set_dictionaries_folder(Some(folder.clone()));
|
||||
let defaults = Options::default();
|
||||
let strict = Options { ignore_uppercase: false, ignore_numbers: false, ..defaults };
|
||||
check_locales(defaults, strict);
|
||||
check_typography(strict);
|
||||
let fr = ProofLang::from_tag(Some("fr-FR"));
|
||||
let ca = ProofLang::from_tag(Some("fr-CA"));
|
||||
check_user_word(fr, ca, strict);
|
||||
check_overrides(folder, fr, ca, strict);
|
||||
}
|
||||
|
||||
fn check_locales(defaults: Options, strict: Options) {
|
||||
for tag in ["fr", "fr-FR", "fr-CA", "fr-BE", "fr-CA-u-nu-latn", "fr-u-ca-gregory"] {
|
||||
let lang = ProofLang::from_tag(Some(tag));
|
||||
assert!(matches!(lang, ProofLang::Pack(_)), "{tag}");
|
||||
assert!(craft_proof::has_dictionary(Some(tag)), "{tag}");
|
||||
assert!(craft_proof::pack_dictionary_file(lang).unwrap().ends_with("fr/spelling.dic"), "{tag}");
|
||||
for word in ["école", "e\u{301}cole", "l'arbre", "l’arbre", "maison-école"] {
|
||||
assert!(is_correct_in_with(word, lang, &strict), "{tag}: {word}");
|
||||
}
|
||||
assert!(!is_correct_in_with("maison--école", lang, &strict));
|
||||
assert!(!is_correct_in_with("l'arbres", lang, &strict));
|
||||
assert!(is_correct_in_with("SNCF", lang, &defaults));
|
||||
assert!(!is_correct_in_with("SNCF", lang, &strict));
|
||||
assert!(is_correct_in_with("mp3x", lang, &defaults));
|
||||
assert!(!is_correct_in_with("mp3x", lang, &strict));
|
||||
assert!(suggest_in("ecole", 6, lang).iter().any(|s| s == "école"));
|
||||
let text = "🙂 maison\u{338} école";
|
||||
let issues = check_text_with(text, &[(0..text.len(), lang)], &strict, false, false);
|
||||
assert_eq!(issues.len(), 1, "{tag}: {issues:?}");
|
||||
assert_eq!(&text[issues[0].start..issues[0].end], "maison\u{338}");
|
||||
assert_eq!(issues[0].lang, lang);
|
||||
assert_eq!(issues[0].kind, IssueKind::Spelling);
|
||||
assert_eq!(issues[0].message, "Faute d’orthographe possible");
|
||||
}
|
||||
}
|
||||
|
||||
fn check_typography(strict: Options) {
|
||||
let fr = ProofLang::from_tag(Some("fr-FR"));
|
||||
let ca = ProofLang::from_tag(Some("fr-CA"));
|
||||
for lang in [fr, ca] {
|
||||
assert_eq!(craft_proof::hyphen::hyphen_points_in("maison", &craft_proof::hyphen::Limits::default(), lang), [2]);
|
||||
}
|
||||
for tag in ["fr-FR", "fr-BE", "fr-u-ca-gregory", "fr-FR-u-rg-cazzzz"] {
|
||||
let issues = craft_proof::check_grammar_in("Bonjour!", ProofLang::from_tag(Some(tag)));
|
||||
assert_eq!(issues.len(), 1, "{tag}: {issues:?}");
|
||||
assert_eq!(issues[0].suggestions, ["\u{202f}!"]);
|
||||
}
|
||||
assert!(craft_proof::check_grammar_in("Bonjour!", ca).is_empty());
|
||||
assert!(craft_proof::check_grammar_in("Bonjour!", ProofLang::from_tag(Some("fr-Latn-CA"))).is_empty());
|
||||
assert!(craft_proof::check_grammar_in("Bonjour!", ProofLang::from_tag(Some("nl-NL"))).is_empty());
|
||||
assert!(craft_proof::fr::grammar::check_in("Bonjour!", ProofLang::En).is_empty());
|
||||
let text = "Bonjour! Salut!";
|
||||
let issues = check_text_with(text, &[(0..8, fr), (8..text.len(), ca)], &strict, true, false);
|
||||
let spacing: Vec<_> = issues.iter().filter(|i| i.kind == IssueKind::Grammar && i.message.contains("fine insécable")).collect();
|
||||
assert_eq!(spacing.len(), 1, "{issues:?}");
|
||||
assert_eq!(spacing[0].lang, fr);
|
||||
}
|
||||
|
||||
fn check_user_word(fr: ProofLang, ca: ProofLang, strict: Options) {
|
||||
craft_proof::add_word("zxqrechargefr");
|
||||
for lang in [fr, ca] {
|
||||
assert!(is_correct_in_with("l’zxqrechargefr", lang, &strict));
|
||||
}
|
||||
craft_proof::remove_word("zxqrechargefr");
|
||||
assert!(!is_correct_in_with("l’zxqrechargefr", ca, &strict));
|
||||
}
|
||||
|
||||
fn check_overrides(folder: PathBuf, fr: ProofLang, ca: ProofLang, strict: Options) {
|
||||
// Regional overrides keep the generic pack fallback and the shared reload generation.
|
||||
pair(&folder, "fr_FR", "SET UTF-8\n", "1\nhexagonefixture\n");
|
||||
pair(&folder, "fr_CA", "SET UTF-8\n", "1\nquebecfixture\n");
|
||||
let generation = craft_proof::dictionary_generation();
|
||||
craft_proof::set_dictionaries_folder(Some(folder.clone()));
|
||||
assert!(craft_proof::dictionary_generation() > generation);
|
||||
assert!(is_correct_in_with("hexagonefixture", fr, &strict));
|
||||
assert!(!is_correct_in_with("hexagonefixture", ca, &strict));
|
||||
assert!(is_correct_in_with("quebecfixture", ca, &strict));
|
||||
assert!(!is_correct_in_with("quebecfixture", fr, &strict));
|
||||
assert!(craft_proof::pack_dictionary_file(ca).unwrap().ends_with("fr_CA.dic"));
|
||||
std::fs::remove_file(folder.join("fr_CA.dic")).unwrap();
|
||||
std::fs::remove_file(folder.join("fr_CA.aff")).unwrap();
|
||||
craft_proof::set_dictionaries_folder(Some(folder));
|
||||
assert!(is_correct_in_with("école", ca, &strict), "removed regional override falls back to fr pack");
|
||||
assert!(craft_proof::pack_dictionary_file(ca).unwrap().ends_with("fr/spelling.dic"));
|
||||
}
|
||||
Reference in new issue
Block a user