From 050b6b0da33ae99444aad26a2df8d745586ccd5f Mon Sep 17 00:00:00 2001 From: StrajnarFilip <46705237+StrajnarFilip@users.noreply.github.com> Date: Sun, 8 Nov 2026 03:44:01 +0100 Subject: [PATCH] Arabic letters join across a change of formatting (#4093) (#5691) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Arabic letters join across a change of formatting (#4093) Each run of a paragraph was shaped on its own, so a letter at a run's edge took the form of a word's start or end. A coloured letter stood apart from its word, and a word edited with Track Changes on read with a space in it (ال ذكاء). Runs are now shaped with the text shown on either side of them, through the shaper's pre- and post-context (fonts::ShapeContext). Runs left out of the layout (hidden text, deletions in a final view) are skipped, so letters join what is on the page. Only paragraphs with a joining script do the work. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01LvG6E2QwHfTULQtWYgVEXk * #4093 review: no letter joins a drop cap formatted on its own A drop cap letter in a run of its own (coloured, say) was still passed to the next run as the text before it, so that run's first letter joined the drop cap, which is drawn apart from the word. The shown text now has a floor past the drop cap, and no run takes context from before it. The drop cap test also covers the letter in a run of its own. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01LvG6E2QwHfTULQtWYgVEXk --------- Co-authored-by: Filip Strajnar Co-authored-by: Claude Opus 5.5 Co-authored-by: Martin Ball --- ROADMAP.md | 2 + crates/fonts/src/lib.rs | 62 ++++++++++-- crates/fonts/src/script.rs | 22 +++++ crates/layout/src/para/builder.rs | 23 ++++- crates/layout/src/para/paragraph.rs | 58 ++++++++++- crates/layout/src/tests/bidi.rs | 146 ++++++++++++++++++++++++++++ docs/gaps.md | 4 +- docs/layout-parity.md | 3 + docs/localization-parity.md | 3 +- 9 files changed, 309 insertions(+), 14 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 9445deb18c..3c31857b74 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -171,6 +171,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm - 2026-11-04: RTF import skips the declared Unicode fallback count even when it exceeds eight fallback units, so longer ANSI alternatives do not leak into the document text. Skipping stays bounded by the available input and ends at a group boundary. Readiness estimates unchanged. - 2026-11-05: Linux: a WordCraft started by an AppImage app no longer takes that app's file for its own (#5653). The AppImage runtime's `APPIMAGE`, `APPDIR`, `ARGV0` and `OWD` pass to whatever the app starts, so an installed WordCraft opened through `xdg-open` saw another program's `APPIMAGE`, and its self-update replaced that program's file. The app now starts over without those variables unless its executable is under `APPDIR`, so the update goes through the package or plain file it really is. Should starting over fail, the self-update is off in that window and Help › Check for Updates says why. Readiness estimates unchanged. - 2026-11-05: A Word template (`.dotx`, `.dotm`, `.dot`) opened from the file manager, the command line or the Finder starts a new, untitled document based on it, as Word's default verb for a template does (#3433). It opened the template itself, so Save (and AutoSave) wrote over it. The new document is attached to the template and Save asks where to write; a protected template asks for its password and still starts a new document. File › Open and a file dropped on the window still open a template itself, to edit it. +- 2026-11-05: Arabic letters join across a change of formatting (gap 16, #4093). Each run was shaped on its own, so a letter at a run's edge took the form of a word's start or end: a coloured letter stood apart, and a word edited with Track Changes on read with a space in it (ال ذكاء). Each run is now shaped knowing the text shown on either side, past hidden or deleted text left out of the view, as for Syriac, Mongolian and the other joining scripts. Readiness estimates unchanged. - 2026-10-28: Ink replay, eraser picking, lasso selection and arrow-candidate discovery reuse the paragraph prefixes they have already scanned (#3614). Page item order, stroke geometry and saved data retain their behavior. This narrows the stability work; ink import fidelity, native pen input and broader corpus qualification remain open. - 2026-10-29: Page drawing finds text-box warps without rescanning the paragraph for every box (#3614). Only requested offsets are indexed, and each paragraph walk stops after its last requested character. Original Linux discovery, complete display-output and CLI/MCP save/reopen controls; native Word comparison untested. Readiness estimates unchanged. - 2026-10-28: Protected typing boundary lookup counts preceding objects once and indexes editable permission IDs (#3614); caret movement and enforcement policy are unchanged. Existing mutation and protection checks retain their costs; this does not close the broader performance gap. @@ -825,6 +826,7 @@ MATH glyphs that reference the same construction now share its immutable variant > 2026-11-05 · Linux self-update ignores another program's AppImage variables (#5653); recorded in the progress log, readiness unchanged. > 2026-11-05 · A template opened from the file manager starts a new document instead of being edited and saved over (#3433); recorded in the progress log, readiness unchanged. +> 2026-11-05 · Arabic letters join across a change of formatting and past hidden or deleted text (#4093); recorded in the progress log, readiness unchanged. > > 2026-11-03 · Arabic dialogs read right to left (rtl::Table); recorded in the progress log, readiness unchanged. > diff --git a/crates/fonts/src/lib.rs b/crates/fonts/src/lib.rs index b99e03b8ff..7305f5af5b 100644 --- a/crates/fonts/src/lib.rs +++ b/crates/fonts/src/lib.rs @@ -21,7 +21,7 @@ pub use fontdb::{ pub use harfrust::Feature; use harfrust::{Direction, ShapeOptions, ShapePlan, ShapePlanKey, Shaper, Tag, UnicodeBuffer}; pub use kurbo::BezPath; -pub use script::{fallback_rank, is_emoji, needs_layout_tables, script_fallbacks}; +pub use script::{fallback_rank, is_emoji, joins_cursively, needs_layout_tables, script_fallbacks}; use skrifa::MetadataProvider; use skrifa::instance::Size; pub use ui_fallback::{ @@ -185,13 +185,34 @@ fn direction_runs(text: &str) -> Vec<(std::ops::Range, bool)> { /// order: right-to-left runs are shaped right to left, then their clusters put back in text /// order (each cluster's glyphs keep the shaper's order); line layout reorders them visually. pub fn shape(face: &FontFace, text: &str, features: &[Feature], map: impl Fn(char) -> char) -> Vec { + shape_in(face, text, features, map, ShapeContext::default()) +} + +/// The text shown on either side of what is being shaped. Only a letter's form depends on it: the +/// letters of an Arabic word join across a change of formatting, so each piece has to know what +/// it continues (#4093). Nothing outside the shaped text is drawn or measured. +#[derive(Clone, Copy, Debug, Default)] +pub struct ShapeContext<'a> { + pub before: &'a str, + pub after: &'a str, +} + +impl<'a> ShapeContext<'a> { + /// This context for the part `r` of `text`: what's beyond `text` where `r` reaches its edge. + fn at_edges_of(self, text: &str, r: &std::ops::Range) -> ShapeContext<'a> { + ShapeContext { before: if r.start == 0 { self.before } else { "" }, after: if r.end == text.len() { self.after } else { "" } } + } +} + +/// [`shape`] of a piece of longer text (see [`ShapeContext`]). +pub fn shape_in(face: &FontFace, text: &str, features: &[Feature], map: impl Fn(char) -> char, around: ShapeContext<'_>) -> Vec { if !text.chars().any(is_rtl) { - return shape_dir(face, text, features, &map, false); + return shape_dir(face, text, features, &map, false, around); } let mut out = Vec::with_capacity(text.len()); for (r, rtl) in direction_runs(text) { let Some(sub) = text.get(r.clone()) else { continue }; - let mut g = shape_run(face, sub, features, &map, rtl); + let mut g = shape_run_in(face, sub, features, &map, rtl, around.at_edges_of(text, &r)); for x in &mut g { x.cluster += r.start; } @@ -205,7 +226,19 @@ pub fn shape(face: &FontFace, text: &str, features: &[Feature], map: impl Fn(cha /// mark positioning. Glyphs come back in logical order like [`shape`]: clusters in text order, /// each cluster's glyphs in the shaper's (visual) order. pub fn shape_run(face: &FontFace, text: &str, features: &[Feature], map: impl Fn(char) -> char, rtl: bool) -> Vec { - let g = shape_dir(face, text, features, &map, rtl); + shape_run_in(face, text, features, map, rtl, ShapeContext::default()) +} + +/// [`shape_run`] of a piece of longer text (see [`ShapeContext`]). +pub fn shape_run_in( + face: &FontFace, + text: &str, + features: &[Feature], + map: impl Fn(char) -> char, + rtl: bool, + around: ShapeContext<'_>, +) -> Vec { + let g = shape_dir(face, text, features, &map, rtl, around); if !rtl { return g; } @@ -221,7 +254,14 @@ pub fn shape_run(face: &FontFace, text: &str, features: &[Feature], map: impl Fn groups.into_iter().flatten().collect() } -fn shape_dir(face: &FontFace, text: &str, features: &[Feature], map: &impl Fn(char) -> char, rtl: bool) -> Vec { +fn shape_dir( + face: &FontFace, + text: &str, + features: &[Feature], + map: &impl Fn(char) -> char, + rtl: bool, + around: ShapeContext<'_>, +) -> Vec { let mut out = Vec::with_capacity(text.len()); let shaped = face.hb().map(|hb| { let shaper = face.shaper.shaper(&hb).instance(face.instance.as_ref()).build(); @@ -234,6 +274,10 @@ fn shape_dir(face: &FontFace, text: &str, features: &[Feature], map: &impl Fn(ch for (i, c) in sub.char_indices() { buf.add(map(c), (r.start + i) as u32); } + // After the text: adding to a buffer clears what follows it. + let around = around.at_edges_of(text, &r); + buf.set_pre_context(around.before); + buf.set_post_context(around.after); buf.guess_segment_properties(); buf.set_direction(if rtl { Direction::RightToLeft } else { Direction::LeftToRight }); let plan = shape_plan(face, &shaper, &buf, features); @@ -386,7 +430,10 @@ mod tests { gb.glyph_infos().iter().zip(gb.glyph_positions()).map(|(i, p)| (i.glyph_id, i.cluster, p.x_advance, p.x_offset, p.y_offset)).collect() }; let kept = |face: &FontFace, text: &str, features: &[Feature], rtl: bool| -> Vec<(u32, u32, i32, i32, i32)> { - shape_dir(face, text, features, &|c| c, rtl).iter().map(|g| (g.gid, g.cluster as u32, g.x_advance, g.x_offset, g.y_offset)).collect() + shape_dir(face, text, features, &|c| c, rtl, ShapeContext::default()) + .iter() + .map(|g| (g.gid, g.cluster as u32, g.x_advance, g.x_offset, g.y_offset)) + .collect() }; let db = FontDb::global(); let faces = [db.face(DEFAULT_FAMILY, "Regular"), db.face(DEFAULT_FAMILY, "Italic"), db.face("Source Sans 3", "Regular")]; @@ -421,7 +468,8 @@ mod tests { for n in 0..3 * MAX_SHAPE_PLANS as u32 { let features = [feature_value(b"kern", n % 2), feature_value(b"liga", n)]; let rtl = n % 3 == 0; - let kept: Vec<(u32, i32)> = shape_dir(&face, "office", &features, &|c| c, rtl).iter().map(|g| (g.gid, g.x_advance)).collect(); + let kept: Vec<(u32, i32)> = + shape_dir(&face, "office", &features, &|c| c, rtl, ShapeContext::default()).iter().map(|g| (g.gid, g.x_advance)).collect(); assert_eq!(kept, fresh(&features, rtl), "{n}"); assert!(kept_shape_plans() <= MAX_SHAPE_PLANS, "{}", kept_shape_plans()); } diff --git a/crates/fonts/src/script.rs b/crates/fonts/src/script.rs index e97de89ce7..51c9d252c5 100644 --- a/crates/fonts/src/script.rs +++ b/crates/fonts/src/script.rs @@ -18,6 +18,28 @@ fn real_script(c: char) -> Option