craft-captions: export warnings name STL's ? and captions cut to fit (#959)

* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks

Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0,
U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a
leading or trailing <br/> was trimmed away.

* craft-captions: TTML/DFXP export drops characters XML 1.0 forbids

Ported from storytold/filmcraft#4467. A control character in cue text made the
exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the
review of that pull request asked for.

* craft-captions: WebVTT cue-text arrow escape is reversible

Ported from storytold/filmcraft#4437. A literal --&gt; in cue text came back as
--> after a write and parse; the writer now adds one amp; level and the reader
removes exactly one.

* craft-captions: a WebVTT speaker name with > no longer spills into the cue text

Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in
place of > inside the voice tag. Also from that pull request's review: a line
break in a name is written as a space, and a name ending in - is kept apart
from the tag's closing >.

* craft-captions: an SRT text line that looks like a timing line survives a round trip

Ported from storytold/filmcraft#4448. The writer marks such a line with a
leading zero-width space and the reader removes it, so the cue is not split.

* craft-captions: back-to-back SCC captions with the same text stay separate cues

Ported from storytold/filmcraft#3808. Every End of Caption now closes the open
cue. MCC's CEA-608 path uses the same decoder.

* craft-captions: an SCC pop-on load may run across the previous caption's erase

Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely
follows another popped on about a second late, because its load had to be one
unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed
between them.

* craft-captions: CEA-708 export starts each caption at its window's top left

Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now
sends DeleteWindows before DefineWindow in place of the ClearWindows after it:
a window made anew is empty with the pen at its top left in any decoder, for
the same number of bytes, so cue timing is unchanged.

* craft-captions: TTML/DFXP export keeps literal < and > text

Ported from storytold/filmcraft#2830. The writer treated every < with a later >
as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its
middle. Only a well-formed tag is markup now; a > inside a quoted attribute
value does not end the tag.

* craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written

Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could
collide with the style ID, and a generated c1 could take the ID of a later cue.
Valid explicit IDs are reserved first; missing, invalid or repeated ones get a
replacement that is in no one's way. IDs are checked as XML NCNames.

* craft-captions: compare the craft-xml and roxmltree views of TTML documents

The TTML reader moved from roxmltree to craft-xml. This dev-only test checks
that both hand it the same elements, attributes and text, by namespace, on
TTML documents and on the writer's output. roxmltree becomes a dev-dependency,
which the cue ID tests use too.

* craft-captions: use craft-timeline's exact time instead of a private copy

craft-captions carried a 290-line subset of the tick, frame rate and timecode
field code because craft-libs had none when it was added. craft-timeline now
has the whole module with the same tick unit. craft_captions::time is a
re-export of craft_timeline::time, so every public path still resolves and
the types are now the same ones craft-timeline's callers hold.

* craft-captions: SCC and MCC time codes count from the document's start timecode

Ported from the caption-format part of storytold/filmcraft#4131
(filmcraft#4130). Document::start_timecode is the label of time zero; the SCC
and MCC writers count from the 29.97 frame with the same label, so the label
carries across rates. Readers note the rate and drop-frame mode of a file's
labels in Document::labels, and Document::count_from counts cues back from a
start time code. charset_warnings names the time codes the file has, which
that pull request's review found missing.

* craft-captions: MCC export can write a packet on every frame

Ported from the caption-format part of storytold/filmcraft#4037
(filmcraft#4024). WriteOptions::every_frame_until fills every 29.97 fps frame
before that time that has no caption data with a packet of CEA-608 nulls and
CEA-708 padding, as SDI and frame-wrapped MXF carry one per frame. Off by
default, at most 24 hours (mcc::MAX_PADDED_FRAMES), and write_to streams the
file line by line.

* craft-captions: README has its MCC writer section, with the every-frame option

The section's text was cut off when the crate was added; only a stray heading
marker was left.

* craft-captions: export warnings name STL's ? and text cut to fit

Ported from storytold/filmcraft#4157 (filmcraft#4155, filmcraft#4156).
charset_warnings now names the characters EBU STL writes as ? (from the
writer's own encoder, at the STL file's time code) and each caption that
CEA-608's 15 rows or the CEA-708 window's 12 rows cut, with how much it lost.
The CEA-708 writer loads only the 12 rows its window holds. The time codes
count from the document's start time code, as the files do.

---------

Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
Co-authored-by: SeeRay11 <searay112023@gmail.com>
Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com>
Co-authored-by: JonRC <jonarc06@gmail.com>
Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com>

crates/captions/README.md

@@ -111,10 +111,12 @@ offset for centring, then text; control codes doubled) into non-displayed memory
 Of Caption exactly on the caption's first frame. The load is placed in the latest run of free
 frames before the caption, so back-to-back captions load while the previous one is on screen. An
 Erase Displayed Memory clears the caption at its out point unless the next caption replaces it.
-Text is wrapped at 32 columns, bottom-aligned (last row 15) and centred. Characters come from the
+Text is wrapped at 32 columns, bottom-aligned (last row 15) and centred; text past 15 rows is
+cut. Characters come from the
 608 basic, special and extended sets (extended characters are preceded by a basic fallback
 character, as the standard requires); a few have a close stand-in (’ as ', – as -) and others are
-dropped. `charset_warnings` names each dropped or replaced character. If there is no room to load a caption
+dropped. `charset_warnings` names each dropped or replaced character and each caption cut to 15
+rows, with how many characters it lost. If there is no room to load a caption
 before its in point (for example a caption at 00:00:00:00), it appears as soon as it is loaded.
 
 ### MCC writer
@@ -123,7 +125,9 @@ The writer merges the scheduled CEA-608 and CEA-708 frames in timestamp order. F
 caption data are omitted, including during a displayed caption; its explicit erase/hide command
 still lands at its out point. CDP sequence counters advance per emitted packet, wrapping at 16 bits.
 CEA-708 text uses the G0 and G1 (Latin-1) sets and part of G2 (… ‘ ’ “ ” • ™ and a few more);
-other characters are sent as `?`, and `charset_warnings` names them.
+other characters are sent as `?`, and `charset_warnings` names them. The CEA-708 window holds 12
+rows and the writer loads only those (`mcc::window_rows`), so a longer caption loses its last rows
+there, and `charset_warnings` names it too.
 Export work and output size therefore follow the scheduled caption data, not the time between
 cues or their duration.
 
@@ -133,3 +137,10 @@ from the start to that time: CEA-608 null pairs and CEA-708 padding. The packets
 so they line up with the video only at that rate; the caller decides whether to offer it. It is
 off by default, covers at most 24 hours (`mcc::MAX_PADDED_FRAMES`), and can be streamed
 (`write_to`).
+
+### EBU STL writer
+
+Text is encoded in ISO/IEC 6937 (character code table `00`). Characters it has no code for, such as
+the ellipsis, the em dash, the euro sign and most scripts other than Latin, are written as `?`;
+`charset_warnings` names each one with how often it occurs and the first caption that has it, at the
+STL file's own time code.

crates/captions/src/lib.rs

@@ -338,51 +338,117 @@ pub fn write(doc: &Document, format: Format, opts: WriteOptions) -> Vec<u8> {
 
 /// How many characters [`charset_warnings`] names one by one per caption stream; the rest share one warning.
 const LISTED_CHARS: usize = 20;
+/// How many cut captions [`charset_warnings`] names one by one per caption stream; the rest share one warning.
+const LISTED_CUTS: usize = 20;
+/// The rows of 32 columns CEA-608 shows ([`scc::caption_rows`] keeps at most this many).
+const ROWS_608: usize = 15;
 
-/// What writing `doc` as `format` does to text the caption data cannot carry. SCC and MCC drop
-/// the characters CEA-608 has no code for (…) or send a close one (’ as '), and MCC's CEA-708
-/// service sends `?` for the characters it has no code for. One warning per such character, in
-/// order of first use, with how often and the first caption that has it. Other formats: none.
+/// What writing `doc` as `format` loses of its text, the characters the caption data cannot carry
+/// and the text cut to fit. EBU STL writes `?` for the characters ISO/IEC 6937 has no code for.
+/// SCC and MCC drop the characters CEA-608 has no code for (…) or send a close one (’ as '), and
+/// MCC's CEA-708 service sends `?` for the characters it has no code for. One warning per such
+/// character, in order of first use, with how often and the first caption that has it. CEA-608
+/// shows at most 15 rows of 32 characters and MCC's CEA-708 window 12 rows: one warning per
+/// caption cut to fit, with how many characters it lost (spaces not counted) and where the lost
+/// text starts. Other formats: none.
 pub fn charset_warnings(doc: &Document, format: Format, opts: WriteOptions) -> Vec<String> {
     let mut out = Vec::new();
+    // the time code the file has for a caption: counted from the start time code, as the writers do
+    let origin = label_frame(doc.start_timecode, scc::RATE, opts.drop_frame);
+    let at_608 = |t: Tick| scc::format_timecode(scc::frame_of(t).saturating_add(origin), opts.drop_frame);
+    // the rows each writer loads, so the warnings name exactly what the files leave out
     if matches!(format, Format::Scc | Format::Mcc) {
-        lossy_chars(doc, "CEA-608", cea608::shown_as, opts.drop_frame, &mut out);
+        lossy_chars(doc, "CEA-608", |text| changed_in_rows(&scc::caption_rows(text), cea608::shown_as), at_608, &mut out);
+        cut_rows(doc, &format!("CEA-608 shows {ROWS_608} rows of 32 characters"), scc::caption_rows, at_608, &mut out);
     }
     if format == Format::Mcc {
-        lossy_chars(doc, "CEA-708", |c| Some(mcc::shown_as_708(c)), opts.drop_frame, &mut out);
+        let shown_as = |c| Some(mcc::shown_as_708(c));
+        lossy_chars(doc, "CEA-708", |text| changed_in_rows(&mcc::window_rows(text), shown_as), at_608, &mut out);
+        let limit = format!("The CEA-708 window holds {} rows of 32 characters", mcc::WINDOW_ROWS);
+        cut_rows(doc, &limit, mcc::window_rows, at_608, &mut out);
+    }
+    if format == Format::Stl {
+        let unencodable = |text: &str| -> Vec<(char, Option<char>)> { stl::unencodable(text).into_iter().map(|c| (c, Some('?'))).collect() };
+        lossy_chars(doc, "EBU STL (ISO/IEC 6937)", unencodable, |t| stl::timecode(t, opts.rate), &mut out);
     }
     out
 }
 
-/// Warnings for the characters of `doc` that `shown_as` does not give back unchanged.
-fn lossy_chars(doc: &Document, stream: &str, shown_as: impl Fn(char) -> Option<char>, drop_frame: bool, out: &mut Vec<String>) {
-    // (character, what a decoder shows instead, count, frame of the first caption with it)
-    let mut lost: Vec<(char, Option<char>, usize, i64)> = Vec::new();
+/// The characters of the `rows` a writer loads that `shown_as` changes, with what a decoder shows
+/// instead.
+fn changed_in_rows(rows: &[String], shown_as: impl Fn(char) -> Option<char>) -> Vec<(char, Option<char>)> {
+    rows.iter().flat_map(|r| r.chars()).map(|c| (c, shown_as(c))).filter(|(c, shown)| *shown != Some(*c)).collect()
+}
+
+/// `n` of `what`, singular for one.
+fn count(n: usize, what: &str) -> String {
+    if n == 1 { format!("1 {what}") } else { format!("{n} {what}s") }
+}
+
+/// Warnings for the captions of `doc` whose text the writer's `rows_of` (the rows it loads for a
+/// caption's text) does not hold in full, as `limit` says: how many characters each loses, not
+/// counting spaces, and the start of the lost text.
+fn cut_rows(doc: &Document, limit: &str, rows_of: impl Fn(&str) -> Vec<String>, at: impl Fn(Tick) -> String, out: &mut Vec<String>) {
+    let visible = |s: &str| s.chars().filter(|c| !c.is_whitespace()).count();
+    let mut cues: Vec<&Cue> = doc.cues.iter().filter(|c| c.end > c.start).collect();
+    cues.sort_by_key(|c| c.start);
+    // (start of the caption, the text it loses)
+    let mut cut: Vec<(Tick, String)> = Vec::new();
+    for cue in cues {
+        let text = plain_text(&cue.text);
+        // the rows keep a prefix of the text: the lost part starts at the first character past them
+        let kept: usize = rows_of(&cue.text).iter().map(|r| visible(r)).sum();
+        if let Some((i, _)) = text.char_indices().filter(|(_, c)| !c.is_whitespace()).nth(kept) {
+            cut.push((cue.start, text[i..].to_string()));
+        }
+    }
+    for (start, lost) in cut.iter().take(LISTED_CUTS) {
+        // the first characters of the lost text, on one line
+        let lost_text = lost.split_whitespace().collect::<Vec<_>>().join(" ");
+        let mut from: String = lost_text.chars().take(24).collect();
+        if from.len() < lost_text.len() {
+            from.push('…');
+        }
+        let n = visible(lost);
+        let what = if n == 1 { "its last character".to_string() } else { format!("its last {n} characters") };
+        out.push(format!("{limit}: the caption at {} lost {what}, from \"{from}\"", at(*start)));
+    }
+    if let Some(rest) = cut.get(LISTED_CUTS..).filter(|rest| !rest.is_empty()) {
+        let n = rest.iter().fold(0usize, |n, (_, lost)| n.saturating_add(visible(lost)));
+        let captions = if rest.len() == 1 { "caption was" } else { "captions were" };
+        out.push(format!("{limit}: {} more {captions} cut to fit too, losing {}", rest.len(), count(n, "character")));
+    }
+}
+
+/// Warnings for the characters of `doc` that the writer changes (`lost_in` gives them for a
+/// caption's text, with what a decoder shows instead), timed by `at`.
+fn lossy_chars(
+    doc: &Document,
+    stream: &str,
+    lost_in: impl Fn(&str) -> Vec<(char, Option<char>)>,
+    at: impl Fn(Tick) -> String,
+    out: &mut Vec<String>,
+) {
+    // (character, what a decoder shows instead, count, start of the first caption with it)
+    let mut lost: Vec<(char, Option<char>, usize, Tick)> = Vec::new();
     let mut index: std::collections::HashMap<char, usize> = std::collections::HashMap::new();
     let mut cues: Vec<&Cue> = doc.cues.iter().filter(|c| c.end > c.start).collect();
     cues.sort_by_key(|c| c.start);
     for cue in cues {
-        // the text the SCC and MCC writers encode: plain text in 32-column rows
-        for c in scc::wrap_rows(&plain_text(&cue.text)).iter().flat_map(|r| r.chars()) {
-            let shown = shown_as(c);
-            if shown == Some(c) {
-                continue;
-            }
+        for (c, shown) in lost_in(&cue.text) {
             match index.get(&c).and_then(|&i| lost.get_mut(i)) {
                 Some(entry) => entry.2 = entry.2.saturating_add(1),
                 None => {
                     index.insert(c, lost.len());
-                    lost.push((c, shown, 1, scc::frame_of(cue.start)));
+                    lost.push((c, shown, 1, cue.start));
                 }
             }
         }
     }
     let times = |n: usize| if n == 1 { "once".to_string() } else { format!("{n} times") };
-    for &(c, shown, n, frame) in lost.iter().take(LISTED_CHARS) {
+    for &(c, shown, n, start) in lost.iter().take(LISTED_CHARS) {
         let what = shown.map_or_else(|| "dropped".to_string(), |s| format!("written as \"{s}\""));
-        // the time code the file has for it: counted from the start time code, as the writers do
-        let at = scc::format_timecode(frame.saturating_add(label_frame(doc.start_timecode, scc::RATE, drop_frame)), drop_frame);
-        out.push(format!("{stream} has no \"{c}\" (U+{:04X}): {what} {}, first in the caption at {at}", c as u32, times(n)));
+        out.push(format!("{stream} has no \"{c}\" (U+{:04X}): {what} {}, first in the caption at {}", c as u32, times(n), at(start)));
     }
     if let Some(rest) = lost.get(LISTED_CHARS..).filter(|rest| !rest.is_empty()) {
         let n = rest.iter().fold(0usize, |n, e| n.saturating_add(e.2));
@@ -603,6 +669,91 @@ mod tests {
         assert_eq!(doc.warnings, ["more than 2 cues: the rest were skipped"]);
     }
 
+    /// EBU STL writes `?` for what ISO/IEC 6937 has no code for, and names it with the time code
+    /// the STL file gives the caption (25 fps, or 29.97 for a 29.97 or 30 fps sequence).
+    #[test]
+    fn charset_warnings_name_what_stl_cannot_carry() {
+        let at = |s: i64| Tick(TICKS_PER_SECOND * s / 5);
+        let cue = |s: i64, text: &str| Cue { start: at(s), end: at(s + 10), text: text.into(), ..Default::default() };
+        let doc = Document { cues: vec![cue(6, "Espere… custa 5 €, não?"), cue(20, "<i>Mais €</i> — Ação ♪")], ..Default::default() };
+        // (sequence rate, time code of the first caption, of the second)
+        for (rate, first, second) in [(None, "00:00:01:05", "00:00:04:00"), (Some(FrameRate::FPS_29_97), "00:00:01:06", "00:00:04:00")] {
+            let opts = WriteOptions { rate, ..Default::default() };
+            let warnings = charset_warnings(&doc, Format::Stl, opts);
+            assert_eq!(
+                warnings,
+                [
+                    format!("EBU STL (ISO/IEC 6937) has no \"…\" (U+2026): written as \"?\" once, first in the caption at {first}"),
+                    format!("EBU STL (ISO/IEC 6937) has no \"€\" (U+20AC): written as \"?\" 2 times, first in the caption at {first}"),
+                    format!("EBU STL (ISO/IEC 6937) has no \"—\" (U+2014): written as \"?\" once, first in the caption at {second}"),
+                ],
+                "{rate:?}"
+            );
+            // what the warnings name is what the file loses
+            let back = stl::parse(&write(&doc, Format::Stl, opts)).unwrap();
+            assert_eq!(back.cues[0].text, "Espere? custa 5 ?, não?");
+        }
+        let clean = Document { cues: vec![cue(0, "Ação “já” ♪ ½ ― Øre")], ..Default::default() };
+        assert!(charset_warnings(&clean, Format::Stl, WriteOptions::default()).is_empty());
+    }
+
+    /// CEA-608 shows 15 rows of 32 characters and the MCC writer's CEA-708 window 12: a longer
+    /// caption loses its last rows, and the warning names the caption and exactly what the file
+    /// leaves out, short rows included.
+    #[test]
+    fn charset_warnings_name_captions_cut_to_fit() {
+        let visible = |s: &str| s.chars().filter(|c| !c.is_whitespace()).count();
+        // rows of exactly 32 characters, numbered so the lost part is visible
+        let full = |k: usize| format!("row{k:02}{}", "x".repeat(27));
+        let rows = |n: usize, last: &[&str]| (1..=n).map(full).chain(last.iter().map(|r| r.to_string())).collect::<Vec<_>>();
+        let doc = |rows: &[String], cues: i64| Document {
+            cues: (0..cues)
+                .map(|i| Cue {
+                    start: Tick((2 * i + 1) * TICKS_PER_SECOND),
+                    end: Tick((2 * i + 2) * TICKS_PER_SECOND),
+                    text: rows.join("\n"),
+                    ..Default::default()
+                })
+                .collect(),
+            ..Default::default()
+        };
+        let row13 = format!("row13{}…", "x".repeat(19));
+        let row16 = format!("row16{}…", "x".repeat(19));
+        // (case, rows of text, what SCC loses, what MCC's CEA-708 loses: characters and the start)
+        let cases = [
+            ("12 full rows", rows(12, &[]), (0, ""), (0, "")),
+            ("12 rows, the last short", rows(11, &["end"]), (0, ""), (0, "")),
+            ("a short row 12, then one character", rows(11, &["end", "z"]), (0, ""), (1, "z")),
+            ("13 full rows", rows(13, &[]), (0, ""), (32, row13.as_str())),
+            ("17 full rows", rows(17, &[]), (64, row16.as_str()), (160, row13.as_str())),
+        ];
+        let cut = |limit: &str, (n, from): (usize, &str)| {
+            let what = if n == 1 { "its last character".to_string() } else { format!("its last {n} characters") };
+            (n > 0).then(|| format!("{limit}: the caption at 00:00:01;00 lost {what}, from \"{from}\""))
+        };
+        let opts = WriteOptions::default();
+        for (case, rows, scc_lost, dtv_lost) in cases {
+            let doc = doc(&rows, 1);
+            let c608 = cut("CEA-608 shows 15 rows of 32 characters", scc_lost);
+            let c708 = cut("The CEA-708 window holds 12 rows of 32 characters", dtv_lost);
+            assert_eq!(charset_warnings(&doc, Format::Scc, opts), Vec::from_iter(c608.clone()), "{case}, SCC");
+            assert_eq!(charset_warnings(&doc, Format::Mcc, opts), Vec::from_iter(c608.into_iter().chain(c708)), "{case}, MCC");
+            for f in [Format::Srt, Format::WebVtt, Format::Stl, Format::Ttml, Format::Dfxp] {
+                assert!(charset_warnings(&doc, f, opts).is_empty(), "{case}, {f:?}");
+            }
+            // what the warnings count is what the files leave out
+            let all = visible(&rows.concat());
+            let scc = scc::parse(&String::from_utf8(write(&doc, Format::Scc, opts)).unwrap()).unwrap();
+            assert_eq!(all - visible(&scc.cues[0].text), scc_lost.0, "{case}, SCC file");
+            let dtv = mcc::parse(&mcc::write_with(&doc, true, false, true, "test")).unwrap();
+            assert_eq!(all - visible(&dtv.cues[0].text), dtv_lost.0, "{case}, CEA-708 in the MCC file");
+        }
+        // many cut captions: a bounded list
+        let w = charset_warnings(&doc(&rows(16, &[]), 23), Format::Scc, opts);
+        assert_eq!(w.len(), LISTED_CUTS + 1);
+        assert_eq!(w[LISTED_CUTS], "CEA-608 shows 15 rows of 32 characters: 3 more captions were cut to fit too, losing 96 characters");
+    }
+
     #[test]
     fn normalize_cuts_overlaps() {
         let c = |s: i64, e: i64| Cue { start: Tick(s), end: Tick(e), text: "x".into(), ..Default::default() };

crates/captions/src/mcc.rs

@@ -314,6 +314,18 @@ impl Dtv {
     }
 }
 
+/// The rows of the CEA-708 window a caption is loaded into.
+pub const WINDOW_ROWS: usize = 12;
+
+/// The rows the CEA-708 writer loads for caption `text`: the CEA-608 rows (plain text in 32-column
+/// rows), as many as the window holds. The rest is left out: past a window's last row, decoders
+/// differ (one scrolls the first rows away, another runs the text into the last row).
+pub fn window_rows(text: &str) -> Vec<String> {
+    let mut rows = scc::caption_rows(text);
+    rows.truncate(WINDOW_ROWS);
+    rows
+}
+
 /// Schedule CEA-708 service 1 pop-on captions (see the module docs).
 fn schedule_708(doc: &Document) -> BTreeMap<i64, Vec<(bool, u8, u8)>> {
     let mut d = Dtv { frames: BTreeMap::new() };
@@ -325,7 +337,7 @@ fn schedule_708(doc: &Document) -> BTreeMap<i64, Vec<(bool, u8, u8)>> {
         let win = (i % 2) as u8;
         let s = scc::frame_of(c.start).max(cursor);
         let e = scc::frame_of(c.end);
-        let rows = scc::wrap_rows(&crate::plain_text(&c.text));
+        let rows = window_rows(&c.text);
         let load = packets(&load_atoms(win, &rows), &mut seq);
         let mut f = (s - load.len() as i64).max(cursor);
         let mut last = f;

crates/captions/src/scc.rs

@@ -364,10 +364,15 @@ pub fn wrap_rows(text: &str) -> Vec<String> {
     rows
 }
 
+/// The rows the CEA-608 writers (SCC, MCC field 1) load for caption `text`: its plain text in at
+/// most 15 rows of 32 columns. What does not fit is left out.
+pub fn caption_rows(text: &str) -> Vec<String> {
+    wrap_rows(&crate::plain_text(text))
+}
+
 /// Byte pairs (without parity) that load `text` into non-displayed memory.
 fn load_pairs(text: &str) -> Vec<(u8, u8)> {
-    let plain = crate::plain_text(text);
-    let rows = wrap_rows(&plain);
+    let rows = caption_rows(text);
     let mut out = vec![(0x14, misc::RCL), (0x14, misc::RCL), (0x14, misc::ENM), (0x14, misc::ENM)];
     let first_row = 16 - rows.len();
     for (i, row) in rows.iter().enumerate() {

crates/captions/src/stl.rs

@@ -209,7 +209,27 @@ fn decode_tf(tf: &[u8], out: &mut String, italic: &mut bool, underline: &mut boo
 
 /// Caption text → text-field bytes (`<i>`, `<u>` become control codes; other tags dropped).
 fn encode_text(text: &str) -> Vec<u8> {
+    encode_text_noting(text, &mut Vec::new())
+}
+
+/// The characters of `text` that ISO/IEC 6937 has no code for, in order: [`write`] sends each
+/// as `?`.
+pub fn unencodable(text: &str) -> Vec<char> {
+    let mut lost = Vec::new();
+    encode_text_noting(text, &mut lost);
+    lost
+}
+
+/// [`encode_text`], noting in `lost` each character it writes as `?`.
+fn encode_text_noting(text: &str, lost: &mut Vec<char>) -> Vec<u8> {
     let mut out = Vec::new();
+    let mut put = |out: &mut Vec<u8>, ch: char| match encode_6937(ch) {
+        Some(b) => out.extend(b),
+        None => {
+            lost.push(ch);
+            out.push(b'?');
+        }
+    };
     let mut rest = text;
     while let Some(ch) = rest.chars().next() {
         let is_tag_start = ch == '<'
@@ -243,7 +263,7 @@ fn encode_text(text: &str) -> Vec<u8> {
                 if ch == '\n' {
                     out.push(0x8a);
                 } else {
-                    out.extend(encode_6937(ch).unwrap_or_else(|| vec![b'?']));
+                    put(&mut out, ch);
                 }
             }
             break;
@@ -251,7 +271,7 @@ fn encode_text(text: &str) -> Vec<u8> {
         if ch == '\n' {
             out.push(0x8a);
         } else {
-            out.extend(encode_6937(ch).unwrap_or_else(|| vec![b'?']));
+            put(&mut out, ch);
         }
         rest = &rest[ch.len_utf8()..];
     }
@@ -364,13 +384,26 @@ fn put(buf: &mut [u8], at: usize, len: usize, s: &str) {
     }
 }
 
+/// The frame rate and disk format code [`write`] uses for a programme at `rate`: 29.97 fps
+/// (`STL30.01`) for 29.97 and 30, else 25 fps (`STL25.01`).
+fn file_rate(rate: Option<FrameRate>) -> (FrameRate, &'static str) {
+    match rate {
+        Some(r) if r == FrameRate::FPS_29_97 || r == FrameRate::FPS_30 => (FrameRate::FPS_29_97, "STL30.01"),
+        _ => (FrameRate::FPS_25, "STL25.01"),
+    }
+}
+
+/// The time code [`write`] gives a caption starting at `t` in a programme at `rate`.
+pub fn timecode(t: Tick, rate: Option<FrameRate>) -> String {
+    let (rate, _) = file_rate(rate);
+    let (_, h, m, s, f) = frames_to_fields(rate.frame_at(rate.snap_nearest(t)), rate, false);
+    format!("{h:02}:{m:02}:{s:02}:{f:02}")
+}
+
 /// Write `doc` at `rate` (25 fps → `STL25.01`; 29.97 / 30 → `STL30.01`; others are written at 25).
 /// `program` goes in the GSI block's originator field (32 characters).
 pub fn write(doc: &Document, rate: Option<FrameRate>, program: &str) -> Vec<u8> {
-    let (rate, dfc) = match rate {
-        Some(r) if r == FrameRate::FPS_29_97 || r == FrameRate::FPS_30 => (FrameRate::FPS_29_97, "STL30.01"),
-        _ => (FrameRate::FPS_25, "STL25.01"),
-    };
+    let (rate, dfc) = file_rate(rate);
     // TTI blocks first (the GSI counts them)
     let mut ttis: Vec<[u8; TTI]> = Vec::new();
     let mut n_subs = 0u16;