Files
craft-libs/crates/audio/src/peaks.rs
T
Aleksander HelgakerandClaude Opus 5.5 38740b43a9 craft-audio: core types, WAV/AIFF read and write, peaks, symphonia (#441)
* craft-audio: core types, WAV/AIFF read and write, peaks, symphonia

The new crate without any codec module: what every later codec plugs
into.

- Shared types: `AudioBuffer`, `AudioInfo`, `ChannelLayout`/`Speaker`,
  `AudioError`, `EncodeOptions` (built with `new` and `with_*`),
  `EncodeFormat`, `BitDepth`, `BwfInfo`, `SampleFormat`.
- WAV/BWF/RF64 and AIFF/AIFF-C reading and writing from SoundCraft's
  `soundcraft-audio-io`, kept over EffectCraft's and FilmCraft's
  smaller writers (only it has BWF, dither, any speaker mask and
  streaming without a known length). `encode` and `FileEncoder` behind
  `pcm-encode`; `lossless-encode` and `encode` turn it on.
- `decode`, `probe`, `tracks` and the streaming, seekable `Decoder`
  (feature `decode-native`). The native path reads WAV, AIFF, and PCM in
  MP4/MOV and Matroska through craft-media's demuxers, packet by packet,
  with edit lists and `CodecDelay`/`DiscardPadding` applied.
  `TrackDecoder` decodes PCM; each codec PR adds its codec to it.
- symphonia 0.6 (MPL-2.0, used unmodified) behind the `symphonia`
  feature, part of `decode`: the fallback for every format with no
  native decoder yet. Its size fields are checked before it reads, and
  its panics are caught where they unwind.
- `peaks`: the min/max/RMS pyramid for waveform drawing.

Workspace: craft-media is a workspace dependency without default
features, so an encode-only craft-audio does not build the MP4
demuxer, and `symphonia-bundle-flac` builds without overflow checks in
dev profiles (its 32-bit LPC restore wraps on valid input).

CI: `audio.yml`, with clippy per feature, rustdoc, WASM, MSRV, and the
tests with ffmpeg oracles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes (CodeRabbit on #441)

- Reading: AIFF that declares frames without an SSND chunk is Malformed; a Matroska PCM track's
  length counts from its first frame; seek targets saturate; an empty FLAC has one track; a
  failed header read is an Io error; deinterleave allocates fallibly; concealed silence is capped
  like a decoded packet, and symphonia's uses the damaged packet's own duration when known.
- Peaks sum squares in f64, so loud float samples give a finite RMS.
- Writing: .aifc maps to AIFF; BWF text is cut at a character boundary; a WAV byte rate past
  32 bits is an error; the FileEncoder docs say nothing may follow the sink's position.
- README: the Limits list says which codecs go through symphonia and which MP4 edit lists apply.
- Tests for each, alias and dither coverage, and a FLAC case in the routing oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, second round (CodeRabbit on #441)

- Every header read is checked: `Native::open`, `native::tracks` and the no-symphonia error read
  it with `head_checked`. craft-media demuxer errors that are I/O failures (not a file ending
  early) are returned as `AudioError::Io` instead of sending the file to the other path.
- A Matroska track's presented length runs from its first frame plus `CodecDelay` (unclamped,
  in timestamp ticks with the delay rounded to one) to its end.
- symphonia's concealment allocates fallibly, and a damaged first packet is concealed when the
  demuxer gives its duration. `deinterleave` reserves its channel list fallibly too.
- `TrackDecoder::conceal` documents its cap at the decode limit.
- Tests: an I/O failure past the first 64 KiB, codec delay with the first block at -20, 0 and
  50 ms (checked with `probe`), a seek back after an overshoot, and the loud-sample RMS value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: TrackDecoder::conceal returns a Result

Concealment allocates fallibly and fails with TooLarge when the silence would pass the 2^30-sample
decode limit, instead of returning less than the packet's length. The native path, the README,
the packets example and the doc example pass the error on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, third round (CodeRabbit on #441)

- Concealed silence is held to the decode limit inside pcm::silence, so symphonia's path checks
  frames × channels too, and symphonia's decoded audio is copied into fallibly allocated planes.
- TrackDecoder::reset forgets the last packet's length.
- Without symphonia, a damaged MP4, Matroska or Ogg file (truncated, invalid, no moov) is
  Malformed instead of Unsupported; a feature the demuxer does not read still names the build.
- Tests: an MP4 with no moov in a native-only build, conceal after a reset and past the limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, fourth round (CodeRabbit on #441)

- A file the native demuxers find damaged is Malformed with symphonia too, when symphonia calls
  it unsupported; before, an ftyp-only MP4 was Unsupported in symphonia builds.
- symphonia's decoded blocks are held to the 2^30-sample decode limit before allocating.
- Symph::seek forgets the last packet's length, so a lost packet after a seek is not concealed
  with the length of one before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, fifth round (CodeRabbit on #441)

- The damaged-file test covers Matroska as well as MP4, and fails when either branch of
  native::damage is removed.
- native::damage leaves out Ogg: ogg::open reports no damage (it skips damaged pages and calls a
  file with none it can read not Ogg), and symphonia reports damaged Ogg as malformed itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: run the ffmpeg oracle tests on ubuntu-26.04

ubuntu-latest's apt ships ffmpeg 6.1.1, which writes MP2 and E-AC-3 in
Matroska with ends a few milliseconds early, so the routing tests on the
later codec PRs cannot pass on it. ubuntu-26.04 ships ffmpeg 8.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 10:25:57 +01:00

286 lines
13 KiB
Rust

//! Multi-resolution overview for waveform drawing.
//!
//! Level 0 sums up each block of [`BLOCK`] samples (min, max and sum of squares); each following
//! level merges [`FANOUT`] blocks of the level below, until a level has a single block. Range
//! queries combine the coarsest whole blocks that fit inside the range and read the partial blocks
//! at its edges from the samples, so a query is exact and costs O(log n) regardless of zoom.
/// Samples per level-0 block.
pub const BLOCK: usize = 32;
/// Blocks of level `n` merged into one block of level `n + 1`.
pub const FANOUT: usize = 4;
/// Upper bound on the number of spans [`Peaks::spans`] will produce.
pub const MAX_COLUMNS: usize = 1 << 20;
/// Multi-resolution overview of one channel for waveform drawing.
#[derive(Debug, Clone, Default, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
pub struct Peaks {
len: usize,
levels: Vec<Vec<Block>>,
}
/// What a run of samples sums up to; NaN samples count for nothing.
#[derive(Debug, Clone, Copy, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
struct Block {
min: f32,
max: f32,
/// In f64: the square of a loud float sample, or the sum of many, can overflow f32.
squares: f64,
}
impl Block {
const EMPTY: Block = Block { min: f32::INFINITY, max: f32::NEG_INFINITY, squares: 0.0 };
fn point(s: f32) -> Block {
Block { min: s, max: s, squares: if s.is_finite() { f64::from(s) * f64::from(s) } else { 0.0 } }
}
fn of(samples: &[f32]) -> Block {
samples.iter().fold(Block::EMPTY, |b, &s| b.merge(Block::point(s)))
}
fn merge(self, o: Block) -> Block {
Block { min: self.min.min(o.min), max: self.max.max(o.max), squares: self.squares + o.squares }
}
/// (min, max), or (0, 0) when nothing was summed up.
fn range(self) -> (f32, f32) {
if self.min > self.max { (0.0, 0.0) } else { (self.min, self.max) }
}
}
impl Peaks {
/// Build the overview of `samples` (one channel). NaN samples are ignored.
pub fn build(samples: &[f32]) -> Peaks {
if samples.is_empty() {
return Peaks::default();
}
let mut levels: Vec<Vec<Block>> = vec![samples.chunks(BLOCK).map(Block::of).collect()];
while let Some(last) = levels.last().filter(|l| l.len() > 1) {
let next = last.chunks(FANOUT).map(|c| c.iter().fold(Block::EMPTY, |a, &b| a.merge(b))).collect();
levels.push(next);
}
Peaks { len: samples.len(), levels }
}
/// Number of samples the overview was built from.
pub fn len_samples(&self) -> usize {
self.len
}
/// Number of resolution levels (0 for an empty overview).
pub fn num_levels(&self) -> usize {
self.levels.len()
}
/// (min, max) over the sample range `[start, end)`, clamped to the data. `samples` must be the
/// data the overview was built from. Returns `(0.0, 0.0)` for an empty range.
pub fn minmax(&self, samples: &[f32], start: usize, end: usize) -> (f32, f32) {
self.sum(samples, start, end).range()
}
fn sum(&self, samples: &[f32], start: usize, end: usize) -> Block {
let end = end.min(self.len).min(samples.len());
let (first, last) = (start.div_ceil(BLOCK), end / BLOCK);
let raw = |a: usize, b: usize| Block::of(samples.get(a..b).unwrap_or_default());
if first >= last {
return raw(start, end);
}
raw(start, first * BLOCK).merge(raw(last * BLOCK, end)).merge(self.blocks(first, last))
}
/// The level-0 blocks `[lo, hi)`, summed from the coarsest blocks that fit.
fn blocks(&self, mut lo: usize, mut hi: usize) -> Block {
let mut acc = Block::EMPTY;
for (lvl, level) in self.levels.iter().enumerate() {
let mut add = |i: usize| acc = acc.merge(level.get(i).copied().unwrap_or(Block::EMPTY));
if lvl + 1 == self.levels.len() || hi.saturating_sub(lo) < 2 * FANOUT {
(lo..hi).for_each(add);
break;
}
while !lo.is_multiple_of(FANOUT) {
add(lo);
lo += 1;
}
while !hi.is_multiple_of(FANOUT) {
hi -= 1;
add(hi);
}
lo /= FANOUT;
hi /= FANOUT;
}
acc
}
/// The spans of sample positions `[start + c * step, start + (c + 1) * step + overlap]` that
/// reach the data, clamped to it.
fn windows(&self, samples: &[f32], start: f64, step: f64, overlap: f64, cols: usize) -> impl Iterator<Item = Option<(f64, f64)>> {
let n = self.len.min(samples.len());
let valid = n > 0 && start.is_finite() && step.is_finite() && step > 0.0 && overlap.is_finite() && overlap >= 0.0;
let last = n.saturating_sub(1) as f64;
(0..cols.min(MAX_COLUMNS)).map(move |c| {
let (a, b) = (start + step * c as f64, start + step * (c + 1) as f64 + overlap);
(valid && b >= 0.0 && a <= last).then(|| (a.max(0.0), b.min(last)))
})
}
/// (min, max) of the line through `samples` over each of `cols` spans of sample positions
/// `[start + c * step, start + (c + 1) * step + overlap]`; fractional positions interpolate
/// between samples. Neighbouring spans share at least their boundary, so a waveform drawn from
/// them stays connected at any zoom. `None` for spans outside the data.
pub fn spans(&self, samples: &[f32], start: f64, step: f64, overlap: f64, cols: usize) -> Vec<Option<(f32, f32)>> {
let at = |t: f64| {
let i = t.floor();
let a = samples.get(i as usize).copied().unwrap_or(0.0);
let b = samples.get(i as usize + 1).copied().unwrap_or(a);
Block::point(a + (b - a) * (t - i) as f32)
};
self.windows(samples, start, step, overlap, cols)
.map(|w| w.map(|(a, b)| self.sum(samples, a.ceil() as usize, b.floor() as usize + 1).merge(at(a)).merge(at(b)).range()))
.collect()
}
/// The RMS of `samples` over each span [`Peaks::spans`] gives, including the samples just
/// outside a span's ends.
pub fn rms_spans(&self, samples: &[f32], start: f64, step: f64, overlap: f64, cols: usize) -> Vec<Option<f32>> {
self.windows(samples, start, step, overlap, cols)
.map(|w| {
w.map(|(a, b)| {
let (s, e) = (a.floor() as usize, (b.ceil() as usize).saturating_add(1).min(self.len).min(samples.len()));
(self.sum(samples, s, e).squares / e.saturating_sub(s).max(1) as f64).sqrt() as f32
})
})
.collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
fn brute(samples: &[f32], start: usize, end: usize) -> (f32, f32) {
let s = &samples[start..end];
(s.iter().copied().fold(f32::INFINITY, f32::min), s.iter().copied().fold(f32::NEG_INFINITY, f32::max))
}
fn signal(n: usize) -> Vec<f32> {
(0..n).map(|i| ((i as f32 * 0.037).sin() * (i as f32 * 0.0011).cos()) + if i % 997 == 0 { 0.5 } else { 0.0 }).collect()
}
#[test]
fn every_range_is_exact() {
let x = signal(100_003);
let p = Peaks::build(&x);
for &(a, b) in &[(0, 32), (0, 100_003), (32, 64_000), (3200, 3232), (96, 99_968), (64, 100_003), (5, 6), (31, 33), (1000, 1031)] {
assert_eq!(p.minmax(&x, a, b), brute(&x, a, b), "range {a}..{b}");
}
let mut seed = 12345u64;
for _ in 0..2000 {
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
let a = (seed >> 33) as usize % 100_003;
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
let b = a + 1 + (seed >> 33) as usize % (100_003 - a);
assert_eq!(p.minmax(&x, a, b), brute(&x, a, b), "range {a}..{b}");
}
}
#[test]
fn degenerate_inputs() {
let p = Peaks::build(&[]);
assert_eq!(p.len_samples(), 0);
assert_eq!(p.minmax(&[], 0, 10), (0.0, 0.0));
assert_eq!(p.spans(&[], 0.0, 1.0, 0.0, 4), vec![None; 4]);
let x = [0.5, -0.25];
let p = Peaks::build(&x);
assert_eq!(p.minmax(&x, 0, 2), (-0.25, 0.5));
assert_eq!(p.minmax(&x, 5, 3), (0.0, 0.0));
assert_eq!(p.minmax(&x, 0, usize::MAX), (-0.25, 0.5));
assert_eq!(p.minmax(&x[..1], 0, 2), (0.5, 0.5));
for (start, step, overlap) in [
(f64::NAN, 1.0, 0.0),
(0.0, 0.0, 0.0),
(0.0, -1.0, 0.0),
(0.0, f64::INFINITY, 0.0),
(f64::NEG_INFINITY, 1.0, 0.0),
(0.0, 1.0, -0.5),
(0.0, 1.0, f64::NAN),
] {
assert_eq!(p.spans(&x, start, step, overlap, 3), vec![None; 3]);
}
assert_eq!(p.spans(&x, 0.0, 1.0, 0.0, 0).len(), 0);
assert_eq!(p.spans(&x, 0.0, 1.0, 0.0, usize::MAX).len(), MAX_COLUMNS);
let x = [f32::NAN, 1.0];
let p = Peaks::build(&x);
assert_eq!(p.minmax(&x, 0, 2), (1.0, 1.0));
assert_eq!(p.spans(&x, 0.0, 1.0, 0.0, 1), vec![Some((1.0, 1.0))]);
assert!(p.rms_spans(&x, 0.0, 1.0, 0.0, 1)[0].is_some_and(f32::is_finite));
}
#[test]
fn loud_finite_samples_give_a_finite_rms() {
let x = [1e20f32, -1e20, 3e38, 0.5];
let p = Peaks::build(&x);
let rms = p.rms_spans(&x, 0.0, 4.0, 0.0, 1)[0].unwrap();
let want = (x.iter().map(|&v| f64::from(v) * f64::from(v)).sum::<f64>() / 4.0).sqrt() as f32;
assert!(rms.is_finite() && (rms / want - 1.0).abs() < 1e-6, "{rms} vs {want}");
}
#[test]
fn rms_spans_match_the_samples() {
let x = signal(20_000);
let p = Peaks::build(&x);
let rms = |s: usize, e: usize| (x[s..e].iter().map(|&v| f64::from(v) * f64::from(v)).sum::<f64>() / (e - s) as f64).sqrt() as f32;
for (step, overlap) in [(1.5, 0.0), (37.0, 37.0), (1000.0, 0.0), (4096.0, 512.0)] {
let cols = ((19_999.0 - overlap) / step) as usize;
for (c, got) in p.rms_spans(&x, 0.0, step, overlap, cols).into_iter().enumerate() {
let (a, b) = (c as f64 * step, ((c + 1) as f64 * step + overlap).min(19_999.0));
let (got, want) = (got.unwrap(), rms(a.floor() as usize, (b.ceil() as usize + 1).min(20_000)));
assert!((got - want).abs() <= want * 1e-4 + 1e-6, "span {c} at step {step}: {got} vs {want}");
}
}
let sine: Vec<f32> = (0..48_000).map(|i| (i as f32 * 0.01).sin()).collect();
let p = Peaks::build(&sine);
let r = p.rms_spans(&sine, 0.0, 47_000.0, 0.0, 1)[0].unwrap();
assert!((r - std::f32::consts::FRAC_1_SQRT_2).abs() < 1e-3, "{r}");
assert_eq!(p.rms_spans(&sine, f64::NAN, 1.0, 0.0, 2), vec![None, None]);
assert_eq!(p.rms_spans(&sine, 50_000.0, 1.0, 0.0, 1), vec![None]);
}
#[test]
fn spans_cover_their_samples_and_touch_their_neighbours() {
let x = signal(10_000);
let p = Peaks::build(&x);
for step in [0.3, 1.0, 7.5, 100.0, 333.3] {
let cols = (9_999.0 / step) as usize;
let spans = p.spans(&x, 0.0, step, 0.0, cols);
for (c, s) in spans.iter().enumerate() {
let (lo, hi) = s.unwrap_or_else(|| panic!("span {c} at step {step}"));
let (a, b) = ((c as f64 * step).ceil() as usize, ((c + 1) as f64 * step).floor() as usize);
if a <= b {
let truth = brute(&x, a, b + 1);
assert!(lo <= truth.0 && hi >= truth.1, "span {c} at step {step}");
}
if let Some(Some((nlo, nhi))) = spans.get(c + 1) {
assert!(*nlo <= hi && *nhi >= lo, "spans {c} and {} at step {step} don't touch", c + 1);
}
}
let all = spans.iter().flatten().fold((f32::INFINITY, f32::NEG_INFINITY), |a, s| (a.0.min(s.0), a.1.max(s.1)));
assert_eq!(all, brute(&x, 0, ((cols as f64 * step).floor() as usize + 1).min(10_000)), "step {step}");
}
}
#[test]
fn spans_follow_the_line_between_samples() {
let x = [0.0, 1.0, -1.0, 0.5];
let p = Peaks::build(&x);
assert_eq!(p.spans(&x, 0.0, 0.25, 0.0, 4), vec![Some((0.0, 0.25)), Some((0.25, 0.5)), Some((0.5, 0.75)), Some((0.75, 1.0))]);
assert_eq!(p.spans(&x, 0.5, 1.0, 0.0, 1), vec![Some((0.0, 1.0))]);
let spans = vec![None, Some((0.0, 0.0)), Some((0.0, 1.0)), Some((-1.0, 1.0)), Some((-1.0, 0.5)), Some((0.5, 0.5)), None];
assert_eq!(p.spans(&x, -2.0, 1.0, 0.0, 7), spans);
assert_eq!(p.spans(&x, 0.0, 1.0, 1.0, 3), vec![Some((-1.0, 1.0)), Some((-1.0, 1.0)), Some((-1.0, 0.5))]);
assert_eq!(p.spans(&x, 0.0, 1.0, 0.5, 3), vec![Some((0.0, 1.0)), Some((-1.0, 1.0)), Some((-1.0, 0.5))]);
}
}