craft-audio: review fixes, round 1 (#443)

Raw ADTS and TrackDecoder:
- Channel configuration 0 (a PCE in the stream, as ffmpeg's -aac_pce and
  AudioToolbox's 7.x write) decodes; it was refused.
- Frames with several raw data blocks are refused; only their first block
  was decoded, silently losing the rest.
- A frame that changes the sample rate, profile or channel configuration is
  concealed; it played at the wrong speed.
- A source that fails during the frame scan is AudioError::Io; it looked
  like the end of the file.
- A stream after an ID3 tag longer than the file head is found.
- conceal counts all the ADTS frames of a packet, and a packet of more than
  1024 frames is refused before it is decoded into memory.

AAC decoder:
- Indexed channel configurations place elements by arrival order, not tag,
  as ffmpeg does; tagged differently, channels were swapped.
- A PCE in an access unit that fails no longer changes the layout, which
  left every later packet failing.
- Non-finite samples are zeroed before the output and the overlap.

MP4: probe and decode agree: the samples' durations end the track when no
single edit does, and an edit cannot outlast the media.

The routing oracle fails on a fixture ffmpeg cannot write when oracles are
required. The README's Limits names HE-AAC v2 as mono, multi-block ADTS,
iTunSMPB, Matroska's tick precision and AAC in CAF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alehelandClaude Opus 5.5 committed 2026-11-04 22:02:29 +01:00
1 parent 7d27e2d0a6
commit d7a40bc110
5 files changed
+421 -55

No files matched your search

+9
View File
@@ -281,13 +281,20 @@ match craft_audio::decode(&bytes, None) {
- HE-AAC: the AAC-LC core is decoded. `decode`, `Decoder` and `TrackDecoder` upsample it 2×;
`aac::Decoder` alone outputs the core rate. SBR is not reconstructed, so there is no content
above the core's Nyquist frequency.
- HE-AAC v2: parametric stereo is not reconstructed either, so the mono core decodes as mono.
- AAC: Main, SSR and LTP profiles, coupling channels and 960-sample frames are unsupported.
- Raw ADTS: frames with several raw data blocks are unsupported. A frame that changes the stream's
sample rate, profile or channel configuration decodes as silence and counts as damaged.
- MP4 gapless information outside an edit list is not read: a file from Apple's encoders with
`iTunSMPB` and no `elst` decodes with its priming and padding.
- AAC channel configuration 7 and program-config layouts, and AIFF with more than four channels,
decode with an unordered layout, in the codec's order.
- A Matroska track with no block durations (AAC from ffmpeg 6) ends at its `DURATION` tag, when
that ends inside the last packet. Otherwise all its audio is presented, including any padding.
- ffmpeg before 9 ends some codecs' tracks in MP4 and Matroska up to 11 ms early. The audio past
the end a file declares is not presented.
- A Matroska track's end comes from its timestamps, so it is known to one timestamp tick (usually
1 ms) unless the last block has `DiscardPadding`.
- A FLAC layout that decoders would not assume for the channel count depends on its
`WAVEFORMATEXTENSIBLE_CHANNEL_MASK` comment. This includes the usual layouts for 5 and 6 channels:
without the comment, back surrounds are read as side surrounds. It includes the usual 7-channel
@@ -328,6 +335,8 @@ Notes while symphonia is in use:
streams through `Decoder`.
- symphonia allocates sizes taken from the file. Container size fields of WAV, FLAC, MP4 and CAF
files are checked against the file length before symphonia reads them.
- AAC in CAF: the priming is not trimmed and `Decoder::seek` is not sample-exact, as CAF goes
through symphonia.
- A panic inside symphonia is caught and returned as `AudioError::Malformed` only where panics
unwind: native builds with the default `panic = "unwind"`. On `wasm32-unknown-unknown`, and in
an app built with `panic = "abort"`, such a panic aborts. The native decoders do not panic.
+89 -4
View File
@@ -60,7 +60,8 @@ impl Decoder {
}
/// Create a decoder for an ADTS stream from its first header. With channel configuration 0 the
/// channel layout is taken from the program config element carried in the first frame.
/// channel layout is taken from the program config element carried in the first frame that
/// decodes with it.
pub fn from_adts(header: &crate::aac::config::AdtsHeader) -> Result<Decoder> {
Self::from_config(header.to_asc())
}
@@ -178,9 +179,17 @@ impl Decoder {
if allow_reconfig && !saw_element && self.asc.channel_config == 0 && self.asc.pce.as_ref() != Some(&pce) {
let layout = pce.layout();
if !layout.is_empty() && layout.iter().map(|e| e.0.channels()).sum::<usize>() <= 64 {
self.asc.pce = Some(pce);
// A damaged PCE must not outlive its access unit: the layout changes
// only when the access unit decodes with it.
let pce_before = self.asc.pce.replace(pce);
let before = (std::mem::take(&mut self.layout), std::mem::take(&mut self.offsets), std::mem::take(&mut self.channels));
self.set_layout(layout);
return self.decode_block(au, false);
let decoded = self.decode_block(au, false);
if decoded.is_err() {
self.asc.pce = pce_before;
(self.layout, self.offsets, self.channels) = before;
}
return decoded;
}
}
}
@@ -214,7 +223,11 @@ impl Decoder {
fn find_slot(&self, ty: ElementType, tag: u8, used: &[bool]) -> Result<usize> {
let compatible = |t: ElementType| t == ty || (ty != ElementType::Cpe && t != ElementType::Cpe);
if let Some(i) = self.layout.iter().enumerate().position(|(i, &(t, g))| t == ty && g == tag && !used[i]) {
// A program config names its elements' tags. An indexed configuration does not: its elements
// are placed by the order they arrive in, whatever their tags.
if self.asc.channel_config == 0
&& let Some(i) = self.layout.iter().enumerate().position(|(i, &(t, g))| t == ty && g == tag && !used[i])
{
return Ok(i);
}
if let Some(i) = self.layout.iter().enumerate().position(|(i, &(t, _))| t == ty && !used[i]) {
@@ -557,6 +570,11 @@ fn synth(state: &mut ChannelState, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
}
}
}
// A hostile stream can overflow f32 (PNS at full energy through a resonant TNS filter); that must
// not reach the output or the next frame's overlap.
for s in buf.iter_mut().filter(|s| !s.is_finite()) {
*s = 0.0;
}
let out: Vec<f32> = (0..1024).map(|n| state.overlap[n] + buf[n]).collect();
state.overlap.copy_from_slice(&buf[1024..]);
state.prev_shape = cur;
@@ -566,6 +584,7 @@ fn synth(state: &mut ChannelState, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
#[cfg(test)]
mod tests {
use super::*;
use craft_media::bitstream::BitWriter;
/// Implicitly signalled HE-AAC (an AAC-LC config) is recognised from the SBR fill element.
#[test]
@@ -585,4 +604,70 @@ mod tests {
assert!(dec.sbr());
assert_eq!(dec.sample_rate(), 22_050);
}
/// An access unit whose elements are all empty: `elements` are (id, tag), then ID_END.
fn silent_au(pce: Option<&ProgramConfig>, elements: &[(u32, u32)]) -> Vec<u8> {
let mut bw = BitWriter::new();
if let Some(pce) = pce {
bw.write_bits(ID_PCE, 3);
pce.write(&mut bw);
}
for &(id, tag) in elements {
bw.write_bits(id, 3);
bw.write_bits(tag, 4);
if id == ID_CPE {
bw.write_bits(0, 1); // common_window
}
for _ in 0..if id == ID_CPE { 2 } else { 1 } {
bw.write_bits(0, 8); // global_gain
bw.write_bits(0, 11); // ics_info: long window, max_sfb 0, no predictor
bw.write_bits(0, 3); // no pulse, TNS or gain control data
}
}
bw.write_bits(ID_END, 3);
bw.finish()
}
/// Indexed configurations place elements by arrival order, whatever their tags (as ffmpeg does).
#[test]
fn indexed_configurations_ignore_element_tags() {
let dec = Decoder::from_config(AudioSpecificConfig::lc(4, 5)).unwrap();
// SCE, CPE, CPE: the first CPE to arrive is L/R even when tagged 1
assert_eq!(dec.find_slot(ElementType::Cpe, 1, &[true, false, false]), Ok(1));
assert_eq!(dec.find_slot(ElementType::Cpe, 0, &[true, true, false]), Ok(2));
}
#[test]
fn program_configs_place_elements_by_tag() {
let pce = ProgramConfig { front: vec![(true, 3), (true, 5)], ..Default::default() };
let asc = AudioSpecificConfig { pce: Some(pce), ..AudioSpecificConfig::lc(4, 0) };
let dec = Decoder::from_config(asc).unwrap();
assert_eq!(dec.find_slot(ElementType::Cpe, 5, &[false, false]), Ok(1));
}
/// A PCE in an access unit that then fails leaves the layout as it was.
#[test]
fn a_damaged_in_band_pce_does_not_change_the_layout() {
let two = ProgramConfig { front: vec![(true, 0)], ..Default::default() };
let six = ProgramConfig { front: vec![(false, 0), (true, 0)], back: vec![(true, 1)], lfe: vec![0], ..Default::default() };
let mut dec = Decoder::from_config(AudioSpecificConfig { pce: Some(two.clone()), ..AudioSpecificConfig::lc(4, 0) }).unwrap();
let good = silent_au(None, &[(ID_CPE, 0)]);
assert_eq!(dec.decode(&good).unwrap().len(), 2);
// the new layout's elements are missing from this access unit
assert!(dec.decode(&silent_au(Some(&six), &[(ID_CPE, 0)])).is_err());
assert_eq!((dec.channels(), dec.config().pce.as_ref()), (2, Some(&two)));
assert_eq!(dec.decode(&good).unwrap().len(), 2);
// one that decodes does change it
let full = silent_au(Some(&six), &[(ID_SCE, 0), (ID_CPE, 0), (ID_CPE, 1), (ID_LFE, 0)]);
assert_eq!(dec.decode(&full).unwrap().len(), 6);
assert_eq!(dec.channels(), 6);
}
#[test]
fn non_finite_spectra_do_not_reach_the_output_or_the_overlap() {
let mut state = ChannelState { overlap: vec![0.0; 1024], prev_shape: WindowShape::Sine };
let info = IcsInfo::parse(&mut BitReader::new(&[0; 2]), 4).unwrap();
let out = synth(&mut state, &info, &vec![f32::INFINITY; 1024]);
assert!(out.iter().chain(&state.overlap).all(|s| s.is_finite()));
}
}
+155 -40
View File
@@ -242,8 +242,11 @@ impl<'a> Native<'a> {
Ok(head) => head,
Err(e) => return Some(Err(e)),
};
let at = after_id3(&head);
let body = head.get(at..).unwrap_or_default();
let at = after_id3(src);
// A tag longer than the head (cover art) hides what follows it, so look there.
let past_tag = usize::try_from(at).is_ok_and(|at| at >= head.len() && at > 0);
let sniffed = if past_tag { super::detect_container(&src.bytes_upto(at, 64 * 1024), None) } else { super::detect_container(&head, None) };
let body = usize::try_from(at).ok().and_then(|at| head.get(at..)).unwrap_or_default();
// The bytes decide the container when they say; the detection (which may come from the
// extension) only picks among raw streams.
// Formats with one track: any other track index is an error, as for a multi-track file.
@@ -251,13 +254,12 @@ impl<'a> Native<'a> {
(Some(n), Some(Ok(_))) if n > 0 => Some(Err(AudioError::Unsupported(format!("no audio track {n}: the file has one")))),
(_, opened) => opened,
};
match super::detect_container(&head, None).or(detected) {
match sniffed.or(detected) {
Some(Container::Wav) if wav::is_wav(&head) => single(open_wav(src)),
Some(Container::Aiff) if aiff::is_aiff(&head) => single(open_aiff(src)),
Some(c @ (Container::Mp4 | Container::QuickTime)) if head.get(4..8) == Some(b"ftyp") => open_mp4(src, c, track),
Some(c @ (Container::Matroska | Container::WebM)) if body.starts_with(&[0x1A, 0x45, 0xDF, 0xA3]) => open_mkv(src, c, track),
Some(Container::Adts) => single(open_adts(src, &head).map(Ok)),
None => single(open_adts(src, &head).map(Ok)),
Some(Container::Adts) | None => single(open_adts(src, at)),
_ => None,
}
}
@@ -378,11 +380,11 @@ impl<'a> Native<'a> {
}
}
/// Where a stream starts: after any ID3v2 tags.
fn after_id3(b: &[u8]) -> usize {
let mut at = 0;
while let Some(len) = b.get(at..).and_then(id3v2_len) {
at = at.saturating_add(len);
/// Where a stream starts: after any ID3v2 tags, which may be longer than the head of the file.
fn after_id3(src: &Src<'_>) -> u64 {
let mut at = 0u64;
while let Some(len) = id3v2_len(&src.bytes_upto(at, 10)) {
at = at.saturating_add(len as u64);
}
at
}
@@ -476,6 +478,8 @@ struct Scanner<'s, 'a, F> {
frame_len: F,
/// The window's start and bytes.
window: (u64, Vec<u8>),
/// The source failed to read: the scan is over.
failed: Option<AudioError>,
}
impl<F: Fn(&[u8]) -> Option<(usize, u32)>> Scanner<'_, '_, F> {
@@ -485,11 +489,17 @@ impl<F: Fn(&[u8]) -> Option<(usize, u32)>> Scanner<'_, '_, F> {
/// The length and decoded frames of the frame at `p`, if one starts there and fits.
fn len_at(&mut self, p: u64) -> Option<(usize, u32)> {
if p >= self.total {
if p >= self.total || self.failed.is_some() {
return None;
}
if p < self.window.0 || p + Self::HEADER > self.window.0 + self.window.1.len() as u64 {
self.window = (p, self.src.bytes_upto(p, Self::WINDOW));
match self.src.read_upto(p, Self::WINDOW) {
Ok(bytes) => self.window = (p, bytes),
Err(e) => {
self.failed = Some(e);
return None;
}
}
}
let off = usize::try_from(p - self.window.0).ok()?;
let (n, frames) = self.window.1.get(off..).and_then(&self.frame_len)?;
@@ -508,11 +518,15 @@ impl<F: Fn(&[u8]) -> Option<(usize, u32)>> Scanner<'_, '_, F> {
/// Frame lengths of a raw stream of self-delimiting frames from `at`: `frame_len(rest)` gives the
/// length and decoded frames of a frame starting at `rest`, or `None`. The first frame must be
/// followed by a second (or the end of the data), so a stray sync word does not make a stream;
/// damage later is skipped by scanning for the next pair. Reads the stream in windows.
fn raw_frames(src: &Src<'_>, at: u64, frame_len: impl Fn(&[u8]) -> Option<(usize, u32)>) -> Option<Vec<(u64, u32, u32)>> {
/// damage later is skipped by scanning for the next pair. Reads the stream in windows. `None` when
/// no first frame is found; a source that fails to read is an error, not the end of the stream.
fn raw_frames(src: &Src<'_>, at: u64, frame_len: impl Fn(&[u8]) -> Option<(usize, u32)>) -> Result<Option<Vec<(u64, u32, u32)>>> {
let total = src.size();
let mut sc = Scanner { src, total, frame_len, window: (u64::MAX, Vec::new()) };
let first = (at..total.min(at.saturating_add(MAX_SYNC_SCAN))).find(|&p| sc.paired(p))?;
let mut sc = Scanner { src, total, frame_len, window: (u64::MAX, Vec::new()), failed: None };
let first = (at..total.min(at.saturating_add(MAX_SYNC_SCAN))).find(|&p| sc.failed.is_some() || sc.paired(p));
let Some(first) = first.filter(|_| sc.failed.is_none()) else {
return sc.failed.map_or(Ok(None), Err);
};
let mut frames = Vec::new();
let mut p = first;
while p < total {
@@ -521,13 +535,13 @@ fn raw_frames(src: &Src<'_>, at: u64, frame_len: impl Fn(&[u8]) -> Option<(usize
frames.push((p, n as u32, f));
p += n as u64;
}
None => match (p + 1..total).find(|&q| sc.paired(q)) {
Some(q) => p = q,
None => break,
None => match (p + 1..total).find(|&q| sc.failed.is_some() || sc.paired(q)) {
Some(q) if sc.failed.is_none() => p = q,
_ => break,
},
}
}
Some(frames)
sc.failed.map_or(Ok(Some(frames)), Err)
}
/// The first `n` frames of a raw stream.
@@ -535,25 +549,44 @@ fn raw_lookahead(src: &Src<'_>, frames: &[(u64, u32, u32)], n: usize) -> Vec<Vec
frames.iter().take(n).filter_map(|&(at, len, _)| src.bytes(at, len as usize)).collect()
}
/// A raw ADTS AAC stream: its frames' payloads (without the ADTS header) are the packets.
/// `head` is the start of the file, as [`Native::open`] read it.
fn open_adts<'a>(src: &Src<'a>, head: &[u8]) -> Option<Native<'a>> {
let frames = raw_frames(src, after_id3(head) as u64, |rest| {
aac::AdtsHeader::parse(rest).ok().filter(|h| h.frame_length > h.header_length).map(|h| (h.frame_length, aac::FRAME_LEN as u32))
})?;
let &(at, _, _) = frames.first()?;
let h = aac::AdtsHeader::parse(&src.bytes_upto(at, 16)).ok()?;
let core = h.to_asc().sample_rate;
let params = CodecParams::new(CodecId::Aac).with_extradata(h.to_asc().to_bytes()).with_sample_rate(core);
/// A raw ADTS AAC stream from `at`: its frames' payloads (without the ADTS header) are the packets.
/// Frames with several raw data blocks do not count as frames: their audio would be lost, so a
/// stream of them is not opened here. A frame that changes the stream's sample rate, profile or
/// channel configuration is damage: it decodes as silence of its length, not at the wrong speed.
fn open_adts<'a>(src: &Src<'a>, at: u64) -> Option<Result<Native<'a>>> {
let frames = raw_frames(src, at, |rest| {
aac::AdtsHeader::parse(rest)
.ok()
.filter(|h| h.frame_length > h.header_length && h.raw_data_blocks == 1)
.map(|h| (h.frame_length, aac::FRAME_LEN as u32))
});
let frames = match frames {
Ok(frames) => frames?,
Err(e) => return Some(Err(e)),
};
let header_at = |p: u64| src.read_upto(p, 16).map(|b| aac::AdtsHeader::parse(&b).ok());
let &(start, _, _) = frames.first()?;
let first = match header_at(start) {
Ok(first) => first?,
Err(e) => return Some(Err(e)),
};
let core = first.to_asc().sample_rate;
// Without extradata the configuration comes from the first header (and, for channel
// configuration 0, a program config element in the stream).
let params = CodecParams::new(CodecId::Aac).with_sample_rate(core);
let lookahead = raw_lookahead(src, &frames, crate::track::LOOKAHEAD_PACKETS);
// The payloads: each frame less its own header.
let payloads: Vec<(u64, u32, u32)> = frames
.iter()
.filter_map(|&(p, n, f)| {
let hl = aac::AdtsHeader::parse(&src.bytes_upto(p, 16)).ok()?.header_length as u32;
(n > hl).then_some((p + u64::from(hl), n - hl, f))
})
.collect();
let lookahead = raw_lookahead(src, &payloads, crate::track::LOOKAHEAD_PACKETS);
let mut payloads = Vec::with_capacity(frames.len());
for &(p, n, f) in &frames {
let h = match header_at(p) {
Ok(h) => h,
Err(e) => return Some(Err(e)),
};
let Some(h) = h else { continue };
let same = (h.object_type, h.sf_index, h.channel_config) == (first.object_type, first.sf_index, first.channel_config);
let skip = if same { h.header_length as u32 } else { n };
payloads.push((p + u64::from(skip), n - skip, f));
}
let mut n = native(Container::Adts, Packets::Raw { frames: payloads, src: src.clone(), next: 0 }, &params, &lookahead, 0, None, None)?;
// HE-AAC decodes twice the core frame length.
let per_frame = rescale(aac::FRAME_LEN as u64, u64::from(core), u64::from(n.info.sample_rate));
@@ -564,7 +597,7 @@ fn open_adts<'a>(src: &Src<'a>, head: &[u8]) -> Option<Native<'a>> {
n.info.frames = frames.len() as u64 * per_frame;
n.frames_known = true;
}
Some(n)
Some(Ok(n))
}
/// The `n`-th audio track (the first when `None`) among `kinds`, as an index into them.
@@ -617,8 +650,11 @@ fn open_mp4<'a>(src: &Src<'a>, container: Container, track: Option<usize>) -> Op
}
_ => (0, None),
};
// The samples' durations end the track when no edit does, and an edit cannot outlast the media.
let media: u64 = t.samples.iter().map(|s| u64::from(s.duration)).sum();
let frames = limit.unwrap_or_else(|| rescale(media, u64::from(timescale), u64::from(rate)).saturating_sub(skip));
let stored = rescale(media, u64::from(timescale), u64::from(rate)).saturating_sub(skip);
let frames = limit.map_or(stored, |l| l.min(stored));
let limit = if media > 0 { Some(frames) } else { limit };
let packets = Packets::Mp4 { file: Box::new(file), src: src.clone(), track, next: 0, timescale, rate };
native(container, packets, &params, &lookahead, skip, limit, Some(frames)).map(Ok)
}
@@ -844,3 +880,82 @@ mod tag_tests {
assert_eq!(last(3000), None, "far before it");
}
}
#[cfg(test)]
mod adts_tests {
use crate::{AudioError, Container, DecodeOptions, Decoder, decode};
use craft_media::ByteSource;
/// A frame of mono silence at sampling frequency index `sf_index`.
fn frame(sf_index: u8) -> Vec<u8> {
// ID_SCE, tag 0, an empty channel stream, ID_END
let au = [0x00, 0x00, 0x00, 0x07];
[&crate::aac::AdtsHeader::write(2, sf_index, 1, au.len(), 0x7FF)[..], &au].concat()
}
#[test]
fn a_stream_decodes_past_damage_between_frames() {
let good = frame(3).repeat(4);
let file = [&good[..], &[0x12; 100], &good].concat();
let (info, buf) = decode(&file, None).unwrap();
assert_eq!((info.container, buf.frames()), (Container::Adts, 8 * 1024));
}
#[test]
fn frames_with_several_raw_data_blocks_do_not_lose_audio_silently() {
let mut one = frame(3);
one[6] |= 1; // two raw data blocks
let file = one.repeat(8);
// No native frames: the file is refused, not decoded at half its length.
assert!(
!matches!(decode(&file, Some("aac")), Ok((info, _)) if info.codec == crate::CodecId::Aac && info.container == Container::Adts && info.damaged == 0)
);
}
#[test]
fn a_frame_that_changes_the_stream_format_is_damage() {
let file = [frame(3).repeat(4), frame(4).repeat(3)].concat();
let (info, buf) = decode(&file, None).unwrap();
assert_eq!((info.sample_rate, info.damaged, buf.frames()), (48_000, 3, 7 * 1024));
}
#[test]
fn a_stream_is_found_after_an_id3_tag_longer_than_the_head() {
let size = 300_000u32;
let tag = [
b"ID3\x03\x00\x00".to_vec(),
[size >> 21, size >> 14, size >> 7, size].iter().map(|v| (v & 0x7F) as u8).collect(),
vec![0; size as usize],
]
.concat();
let file = [tag, frame(3).repeat(4)].concat();
let (info, buf) = decode(&file, None).unwrap();
assert_eq!((info.container, buf.frames()), (Container::Adts, 4 * 1024));
}
/// Bytes that fail to read past `good`.
struct Failing {
bytes: Vec<u8>,
good: u64,
}
impl ByteSource for Failing {
fn len(&self) -> u64 {
self.bytes.len() as u64
}
fn read_at(&self, offset: u64, buf: &mut [u8]) -> std::io::Result<()> {
if offset + buf.len() as u64 > self.good {
return Err(std::io::Error::other("the source failed"));
}
self.bytes.as_slice().read_at(offset, buf)
}
}
#[test]
fn a_source_failing_during_the_frame_scan_is_an_io_error() {
let bytes = frame(3).repeat(100_000);
// Past the head, so the failure comes during the scan rather than format detection.
let opened = Decoder::open(Failing { bytes, good: 300_000 }, Some("aac"), &DecodeOptions::default());
assert!(matches!(opened, Err(AudioError::Io(_))), "{:?}", opened.err());
}
}
+107 -8
View File
@@ -97,7 +97,7 @@ impl TrackDecoder {
let h = lookahead.first().and_then(|p| aac::AdtsHeader::parse(p).ok()).ok_or_else(|| {
AudioError::Malformed("AAC track without an AudioSpecificConfig (extradata), and its packets are not ADTS".into())
})?;
h.to_asc().to_bytes()
adts_config(&h, &payloads)?
} else {
params.extradata.clone()
};
@@ -195,7 +195,10 @@ impl TrackDecoder {
let data = packet.unwrap_or_default();
let frames = match &mut self.inner {
#[cfg(feature = "aac-decode")]
Inner::Aac { up, .. } => Some(aac::FRAME_LEN * if up.is_some() { 2 } else { 1 }),
Inner::Aac { up, .. } => {
let units = adts_frames(data).map_or(1, |f| f.iter().map(|(h, _)| usize::from(h.raw_data_blocks)).sum::<usize>());
Some(units.min(MAX_ADTS_FRAMES_PER_PACKET) * aac::FRAME_LEN * if up.is_some() { 2 } else { 1 })
}
Inner::Pcm { kind, channels } => Some(data.len() / (kind.bytes() * *channels).max(1)),
};
let frames = frames.filter(|&n| n > 0).unwrap_or(self.last_frames);
@@ -203,7 +206,8 @@ impl TrackDecoder {
}
/// The frames every packet decodes to, for codecs whose packets all have one length: AAC
/// (1024, 2048 for HE-AAC); `None` for PCM.
/// (1024, 2048 for HE-AAC; a packet of several ADTS frames decodes to that many times this);
/// `None` for PCM.
pub fn frames_per_packet(&self) -> Option<usize> {
match &self.inner {
#[cfg(feature = "aac-decode")]
@@ -278,17 +282,47 @@ fn append(out: &mut Vec<Vec<f32>>, more: Vec<Vec<f32>>) {
}
}
/// The raw access units of an ADTS-framed packet (one or more whole frames, nothing else), or
/// `None` when the packet is not ADTS. Frames with several raw data blocks are unsupported.
/// The most ADTS frames a packet may hold: its audio is decoded and held in memory before any
/// decode limit applies. 1024 frames is the 2^20 frames the demuxers allow a packet.
#[cfg(feature = "aac-decode")]
fn adts_payloads(packet: &[u8]) -> Option<Result<Vec<&[u8]>>> {
const MAX_ADTS_FRAMES_PER_PACKET: usize = 1024;
/// The configuration of an AAC stream from its first ADTS header. Channel configuration 0 leaves
/// the layout to a program config element in the stream, looked for in `first_units`.
#[cfg(feature = "aac-decode")]
fn adts_config(header: &aac::AdtsHeader, first_units: &[&[u8]]) -> Result<Vec<u8>> {
if header.channel_config != 0 {
return Ok(header.to_asc().to_bytes());
}
let mut dec = aac::Decoder::from_adts(header)?;
for unit in first_units {
let _ = dec.decode(unit);
if dec.config().pce.is_some() {
return Ok(dec.config().to_bytes());
}
}
Err(AudioError::Malformed("ADTS stream with channel configuration 0, and no program config element in its first packets".into()))
}
/// The frames of an ADTS-framed packet (one or more whole frames, nothing else), or `None` when
/// the packet is not ADTS.
#[cfg(feature = "aac-decode")]
fn adts_frames(packet: &[u8]) -> Option<Vec<(aac::AdtsHeader, &[u8])>> {
if packet.len() < 7 || packet[0] != 0xFF || packet[1] & 0xF6 != 0xF0 {
return None;
}
let frames = aac::split_adts(packet).ok()?;
let whole: usize = frames.iter().map(|(h, _)| h.frame_length).sum();
if whole != packet.len() {
return None;
(whole == packet.len()).then_some(frames)
}
/// The raw access units of an ADTS-framed packet, or `None` when the packet is not ADTS. Frames
/// with several raw data blocks are unsupported.
#[cfg(feature = "aac-decode")]
fn adts_payloads(packet: &[u8]) -> Option<Result<Vec<&[u8]>>> {
let frames = adts_frames(packet)?;
if frames.len() > MAX_ADTS_FRAMES_PER_PACKET {
return Some(Err(AudioError::Malformed(format!("a packet of {} ADTS frames", frames.len()))));
}
if frames.iter().any(|(h, _)| h.raw_data_blocks > 1) {
return Some(Err(AudioError::Unsupported("ADTS frames with several raw data blocks".into())));
@@ -353,6 +387,71 @@ mod tests {
assert!(matches!(dec.conceal(None), Err(AudioError::TooLarge(_))));
}
/// One ADTS frame (stereo, 48 kHz, or channel configuration 0 with a PCE) of silence.
#[cfg(feature = "aac-decode")]
fn adts_frame(channel_config: u8) -> Vec<u8> {
use craft_media::bitstream::BitWriter;
let mut bw = BitWriter::new();
if channel_config == 0 {
bw.write_bits(5, 3); // ID_PCE
aac::ProgramConfig { front: vec![(false, 0), (true, 0)], ..Default::default() }.write(&mut bw);
bw.write_bits(0, 3); // ID_SCE
bw.write_bits(0, 4);
bw.write_bits(0, 22); // global_gain, ics_info, no pulse, TNS or gain control data
}
bw.write_bits(1, 3); // ID_CPE
bw.write_bits(0, 5); // tag, common_window
bw.write_bits(0, 44); // two such channel streams
bw.write_bits(7, 3); // ID_END
let au = bw.finish();
[&aac::AdtsHeader::write(2, 3, channel_config, au.len(), 0x7FF)[..], &au].concat()
}
#[cfg(feature = "aac-decode")]
fn aac_params() -> CodecParams {
CodecParams::new(CodecId::Aac)
}
#[cfg(feature = "aac-decode")]
#[test]
fn adts_packets_conceal_to_the_length_of_all_their_frames() {
let frame = adts_frame(2);
let mut dec = TrackDecoder::new(&aac_params(), &[&frame]).unwrap();
let packet = frame.repeat(3);
assert_eq!(dec.decode(&packet).unwrap().frames(), 3072);
// the middle frame's payload is damaged: its element id is ID_CCE, which is unsupported
let mut damaged = packet.clone();
damaged[frame.len() + 7] = 0b0100_0000;
assert!(dec.decode(&damaged).is_err());
assert_eq!(dec.conceal(Some(&damaged)).unwrap().frames(), 3072);
assert_eq!(dec.conceal(Some(&frame)).unwrap().frames(), 1024);
assert_eq!(dec.conceal(None).unwrap().frames(), 1024);
}
#[cfg(feature = "aac-decode")]
#[test]
fn a_packet_of_too_many_adts_frames_is_refused() {
let frame = adts_frame(2);
let mut dec = TrackDecoder::new(&aac_params(), &[&frame]).unwrap();
assert_eq!(dec.decode(&frame.repeat(MAX_ADTS_FRAMES_PER_PACKET)).unwrap().frames(), MAX_ADTS_FRAMES_PER_PACKET * 1024);
let packet = frame.repeat(MAX_ADTS_FRAMES_PER_PACKET + 1);
assert!(matches!(dec.decode(&packet), Err(AudioError::Malformed(_))));
assert_eq!(dec.conceal(Some(&packet)).unwrap().frames(), MAX_ADTS_FRAMES_PER_PACKET * 1024);
}
#[cfg(feature = "aac-decode")]
#[test]
fn adts_channel_configuration_0_takes_its_layout_from_the_stream() {
let frame = adts_frame(0);
let mut dec = TrackDecoder::new(&aac_params(), &[&frame]).unwrap();
assert_eq!(dec.channels(), 3);
assert_eq!(dec.decode(&frame).unwrap().channels.len(), 3);
// no program config element where one is needed
let bare = aac::AdtsHeader::write(2, 3, 0, 1, 0x7FF);
let bare = [&bare[..], &[0xE0]].concat();
assert!(matches!(TrackDecoder::new(&aac_params(), &[&bare]), Err(AudioError::Malformed(_))));
}
#[test]
fn codecs_without_a_packet_decoder_are_unsupported() {
for codec in [CodecId::Flac, CodecId::Vorbis, CodecId::Alac, CodecId::Mp3, CodecId::Opus, CodecId::H264] {
+61 -3
View File
@@ -129,7 +129,8 @@ fn run(ff: &Path, args: &[&str]) -> bool {
Command::new(ff).args(["-v", "error", "-y"]).args(args).output().is_ok_and(|o| o.status.success())
}
/// The fixture for `c`, written by ffmpeg (cached), or `None` when this ffmpeg lacks an encoder.
/// The fixture for `c`, written by ffmpeg (cached), or `None` when this ffmpeg lacks an encoder
/// (a failure with `CRAFT_MEDIA_REQUIRE_ORACLES=1`).
fn fixture(ff: &Path, c: &Case) -> Option<PathBuf> {
let out = crate::testkit::fixtures_dir("routing").join(c.name);
let source = match (c.mono, c.surround) {
@@ -138,13 +139,15 @@ fn fixture(ff: &Path, c: &Case) -> Option<PathBuf> {
_ => STEREO.to_string(),
};
let input = format!("{source}:s={}", c.rate);
crate::testkit::fixtures::generate(&out, |tmp| {
let made = crate::testkit::fixtures::generate(&out, |tmp| {
let tmp = tmp.to_string_lossy();
let mut args = vec!["-f", "lavfi", "-i", &input, "-t", "5"];
args.extend_from_slice(c.args);
args.push(&tmp);
run(ff, &args)
})
});
assert!(made.is_some() || !craft_media_testkit::oracles_required(), "{}: ffmpeg could not write this fixture", c.name);
made
}
/// ffmpeg's decode of `file` as interleaved f32.
@@ -441,3 +444,58 @@ fn matroska_aac_without_block_durations_ends_at_its_duration_tag() {
assert_eq!(frames(Some("00:00:09.000000000")), all, "ends after the last packet");
assert_eq!(frames(Some("not a duration")), all);
}
/// ADTS with channel configuration 0 takes its layout from the program config element in the stream.
#[test]
fn adts_with_a_program_config_element_decodes() {
let ff = require_ffmpeg!();
let c = Case {
name: "aac-pce.aac",
surround: true,
mono: false,
rate: 48_000,
args: &["-c:a", "aac", "-b:a", "384k", "-aac_pns", "0", "-aac_pce", "1"],
container: Container::Adts,
codec: CodecId::Aac,
layout: ChannelLayout::unordered(6),
channels: 6,
frames: None,
min_snr: 100.0,
};
let Some(path) = fixture(&ff, &c) else { return };
let bytes = std::fs::read(&path).unwrap();
assert_eq!(bytes[3] >> 6 | (bytes[2] & 1) << 2, 0, "the fixture signals its layout in a PCE");
let (info, buf) = decode(&bytes, None).unwrap();
assert_eq!((info.container, info.codec, buf.channels.len()), (c.container, c.codec, c.channels));
assert_eq!(info.layout, c.layout);
// Decoded order (C, L, R, surrounds, LFE) against ffmpeg's (L, R, C, LFE, surrounds).
let wav: Vec<Vec<f32>> = [1, 2, 0, 5, 3, 4].iter().map(|&i| buf.channels[i].clone()).collect();
let s = snr(&wav, &reference(&ff, &path));
assert!(s >= c.min_snr, "SNR {s:.1} dB");
}
/// `probe` and `decode` agree on an MP4's length when no single edit gives it: the samples'
/// durations end the track, as they do for ffmpeg.
#[test]
fn mp4_without_an_edit_list_ends_with_its_sample_durations() {
let ff = require_ffmpeg!();
let c = Case {
name: "aac-no-edits.m4a",
surround: false,
mono: false,
rate: 44_100,
args: &["-c:a", "aac", "-b:a", "128k", "-aac_pns", "0", "-use_editlist", "0"],
container: Container::Mp4,
codec: CodecId::Aac,
layout: ChannelLayout::STEREO,
channels: 2,
frames: None,
min_snr: 100.0,
};
let Some(path) = fixture(&ff, &c) else { return };
let bytes = std::fs::read(&path).unwrap();
let (info, buf) = decode(&bytes, None).unwrap();
assert_eq!(probe(&bytes, None).unwrap().frames, buf.frames() as u64);
assert_eq!(info.frames, buf.frames() as u64);
assert_eq!(buf.frames(), reference(&ff, &path).len() / c.channels);
}