From 650d1996becbc56c60c1f1f1e727d29a957f7480 Mon Sep 17 00:00:00 2001 From: alehel Date: Sun, 8 Nov 2026 11:10:55 +0100 Subject: [PATCH] craft-audio: say why the ADTS-in-MP4 seek limit is kept (#443) The Limits entry from 79f5839 said what goes wrong but not why it is left. It now says that such a file is outside the MP4 specification: an MP4 AAC sample is one raw access unit with the configuration in the sample description (ISO/IEC 14496-14, 14496-3), so ADTS headers do not belong in it, and a sample's duration is the time it decodes to (ISO/IEC 14496-12). It says why files within the specification are not affected, what still works, and why seeks and edits in such files are not corrected: finding where each sample's audio starts means counting the ADTS frames in every sample before it, one read per sample, which would slow every ADTS-framed MP4 at open to work around files outside the specification. Co-Authored-By: Claude Opus 5.5 --- crates/audio/README.md | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/crates/audio/README.md b/crates/audio/README.md index dca88a7a..6f61aa5a 100644 --- a/crates/audio/README.md +++ b/crates/audio/README.md @@ -314,11 +314,26 @@ match craft_audio::decode(&bytes, None) { - MP4 edit lists: a single media edit (encoder priming and the end) is applied, alone or after one empty edit; the empty edit's delay is not added as silence. Other edit lists are not applied, and the media plays as it is stored. -- AAC in MP4 stored as ADTS frames (not standard), when the sample durations count fewer frames - than the samples hold (HE-AAC's core frames, or several frames to a sample): the track decodes in - full when read from the start, but `Decoder::seek` places samples by those durations, so it lands - later than asked, and an edit list measured in them can end the track early. Telling would mean - reading every sample. +- AAC in MP4 stored as ADTS frames, when the sample durations count fewer frames than the samples + hold: HE-AAC durations that count only the core frames, or several frames in a sample timed as + one. Such a file is broken: it is outside the MP4 specification in two ways. + - An MP4 AAC sample is one raw access unit, with the stream's configuration in the sample + description (ISO/IEC 14496-14 and 14496-3). ADTS is a transport framing for streams such as + MPEG-TS and raw `.aac`, and its headers do not belong in an MP4 sample. + - A sample's duration in the time-to-sample table is the time it decodes to (ISO/IEC 14496-12). + Durations that count fewer frames than the samples hold do not describe the track. + + ffmpeg, Apple's encoders, phones and cameras write raw AAC with matching durations, and a + correct remux from MPEG-TS or raw `.aac` converts the framing, so files within the specification + are not affected. In a file outside it: + - Read from the start, the track decodes in full: durations that fall short of the frames in + the samples read at open do not end it. + - `Decoder::seek` places samples by those durations, so it lands later than asked, and an edit + list measured in them can end the track early. + - These are not corrected because where each sample's audio starts is only known by counting + the ADTS frames in every sample before it: one read per sample, hundreds of thousands on a + two-hour track. Done at open, that would slow every ADTS-framed MP4 to work around files + that are outside the specification. ## Codecs