Commit Graph
102 Commits
Author SHA1 Message Date
FredApps 1d413b992d Merge pull request #1688 from storytold/codex/flac-main-reconcile
Reconcile native FLAC decoder with merged QOA support
2026-11-16 00:17:47 +01:00
FredApps c26b32c8ac Merge pull request #1636 from storytold/codex/shared-pcm-packet
audio: share interleaved PCM packet decoding
2026-11-16 00:05:15 +01:00
FredApps d1a431c4b5 audio: pure-Rust Vorbis encoder (1-8 channels, quality, tags, wasm) and an Ogg page writer (#1642)
* audio: pure-Rust Vorbis encoder, and an Ogg page writer in craft-media

Replace the libvorbis wrapper (vorbis_rs, C, desktop only) behind
`vorbis-encode` with FilmCraft's clean-room Vorbis I encoder by
@JordyUSA (storytold/filmcraft#4481), and move its Ogg page writer into
craft-media's `ogg` module.

craft-audio:
- `vorbis::encoder::{Encoder, EncoderConfig, Error}`: 1-8 channels in
  Vorbis order, any sample rate, quality -1 to 10 (VBR), streaming
  packets with granule positions.
- `encode_ogg_vorbis` with `OggVorbisOptions` (quality, vendor, tags,
  serial) writes a whole Ogg Vorbis file with a comment header.
- `encode_ogg(buf, kbps)` and `OGG_BITRATES` keep working: the bit rate
  picks the quality whose nominal rate it is (`quality_for_bitrate`).
- `encode` with `EncodeFormat::Ogg` takes 1-8 channel WAV-order layouts
  and writes them in Vorbis order, and `RateControl::Quality`.
- Ogg is built on wasm32 too.
- Residue partitions hold whole sample frames (48, 40, 48 or 56 values
  for 3, 5, 6 and 7 channels): libvorbis decodes each partition from
  channel 0, so 32-value partitions came out scrambled for those counts.
- The first audio packet gets a page of its own, so a short file's
  first audio page is never also its last (ffmpeg then kept the padding).

craft-media: `ogg::PageWriter`, `opus_tags`, `vorbis_comment`.

Part of storytold/craft-libs#778.

* audio, media: address review on the Vorbis encoder and Ogg writer

- Encoder: read cursor into the pending input, compacted once per call,
  so a large encode() call no longer moves the tail on every block.
- Tests: check ffmpeg's libvorbis decoder only when ffmpeg has it.
- PageWriter::finish: the end granule replaces the last page's only when
  a packet completes on it (RFC 7845 4.4); documented and tested.
- README: Ogg Opus packet granules are the samples decoded so far.

* audio: pick Ogg by extension on wasm32 too

The Vorbis encoder is pure Rust now, so encode_format_for("ogg") follows
the vorbis-encode feature alone, as EncodeFormat::Ogg.is_built() and
ENCODE_EXTENSIONS already do.
2026-11-15 22:59:29 +01:00
FredApps 5a7a63a21d Merge pull request #1593 from storytold/i778/opus-encoder-effectcraft-speedups
craft-audio: port EffectCraft's later Opus encoder speed-ups
2026-11-15 22:26:13 +01:00
Dylan M. Taylor 30596f73c5 Merge pull request #1707 from storytold/extract/wav-marker-metadata
audio: extract WAV cue, adtl, RF64 and XMP marker metadata
2026-11-15 15:43:10 -05:00
Dylan M. Taylor 27b1134e23 Share Opus FFT and CELT allocation primitives 2026-11-15 18:57:27 +00:00
Dylan M. Taylor 5dcddf5bf0 Merge branch 'main' of https://github.com/storytold/craft-libs into codex/shared-pcm-packet
# Conflicts:
#	crates/audio/src/lib.rs
2026-11-15 18:50:21 +00:00
Dylan M. Taylor 3f4de9ca11 Merge pull request #1637 from storytold/codex/adts-asc-config
audio: expose ASC prefix to ADTS configuration
2026-11-15 13:36:24 -05:00
Dylan M. Taylor a28cb61765 Cover interleaved PCM sample representations 2026-11-15 18:33:29 +00:00
Dylan M. Taylor affd53bf5f Merge remote-tracking branch 'origin/main' into codex/flac-main-reconcile
# Conflicts:
#	crates/audio/Cargo.toml
#	crates/audio/src/lib.rs
2026-11-15 17:52:06 +00:00
Dylan M. Taylor 7b7568d95e Merge pull request #1608 from storytold/codex/wav-range-cutover
audio: share bounded range-backed WAV PCM access
2026-11-15 12:49:32 -05:00
Dylan M. Taylor 25b1cf2b6f audio: expose validated ADTS configuration fields 2026-11-15 16:30:22 +00:00
Dylan M. Taylor b958ba705f audio: expose ASC prefix to ADTS configuration 2026-11-15 16:29:38 +00:00
Dylan M. Taylor 64b7a7f21e audio: share interleaved PCM packet decoding 2026-11-15 16:26:40 +00:00
Dylan M. Taylor ad9850f18f fix(audio): validate FLAC stream precision and complete consumption 2026-11-15 11:05:37 -05:00
Dylan M. Taylor aa398af9dd audio: fix range test lint on workspace gates 2026-11-15 15:48:01 +00:00
Dylan M. Taylor badb790411 audio: expose WAV signature detection 2026-11-15 15:40:02 +00:00
Dylan M. Taylor 8e394d43fa audio: share bounded range-backed WAV PCM access 2026-11-15 15:39:42 +00:00
FredApps b850981f83 craft-audio: port EffectCraft's later Opus encoder speed-ups
The shared encoder was imported from storytold/effectcraft@c618429. EffectCraft's
copy has since gained @vangel76's speed-ups (effectcraft #2394, #2398, #2399,
#2402, #2403, #2404) and the strict-Clippy CELT stage refactor. Port them so
EffectCraft can drop crates/opusenc without slowing WebM export.

Packets are byte-identical to effectcraft-opusenc at d6c9ad66 (2100 packets
across SILK, hybrid and CELT, mono and stereo, 10-320 kb/s). The shared Error
API is unchanged.

Refs #778
2026-11-15 15:29:08 +01:00
el-j 2d5f2e4a73 craft-audio: flac_decode tests without index loops (CI clippy) 2026-11-15 15:25:56 +01:00
el-j ee20ae63fd craft-audio: qoa_oracle test without index loops (CI clippy) 2026-11-15 15:24:46 +01:00
el-j cdf4a2fa6e craft-audio: native FLAC decoder (flac-decode)
Adds flac::decode for whole files and flac::Decoder for single frames:
RFC 9639 subframes (constant, verbatim, fixed, LPC), Rice and escaped
partitions, wasted bits, all stereo modes, fixed and variable block
sizes, 4 to 32 bits, CRC-8 and CRC-16 checked. Not yet wired into
craft_audio::decode, which still uses symphonia for FLAC.

Part of #1546
2026-11-15 15:11:43 +01:00
el-j ff61266d63 craft-audio: QOA (Quite OK Audio) decoder and encoder
Adds the qoa module behind the qoa, qoa-decode and qoa-encode features:
whole .qoa files, 16-bit, 1 to 8 channels. Streaming files are
unsupported. An ffmpeg oracle test checks that ffmpeg decodes our files
to exactly the samples our decoder returns.

Closes #1541
2026-11-15 12:38:57 +01:00
Dylan M. Taylor c4fb292825 Use canonical parity check in WAV iXML fixture 2026-11-14 23:02:09 -05:00
Dylan M. Taylor 44802bd54d craft-audio: preserve iXML WAV time-reference fallback 2026-11-14 22:57:45 -05:00
Dylan M. Taylor 92e665a5e8 Exercise Vorbis block boundaries with distinct channel signals 2026-11-14 18:33:24 -05:00
Dylan M. Taylor 354acd6a54 Cover Vorbis in format contract tests and assert streaming refusal 2026-11-14 18:15:14 -05:00
Dylan M. Taylor dc2b83d7f4 Document Vorbis rate semantics and validate standalone options 2026-11-14 18:13:31 -05:00
Dylan M. Taylor 9efb0a1575 Complete standalone Vorbis feature and option validation 2026-11-14 18:12:40 -05:00
Dylan M. Taylor 8ae42d1aeb Merge main and reconcile Vorbis encoding with current audio APIs 2026-11-14 18:12:03 -05:00
Dylan Taylor 630c570789 fix: update tests for vorbis-encode feature (ogg support) 2026-11-14 10:16:33 -05:00
Dylan M. Taylor b9c1dfa18f feat(audio): migrate Ogg Vorbis encoding to craft-audio (PR #26)
- Add vorbis-encode feature with dep:vorbis_rs = "0.5"
- Implement encode_ogg() via VorbisEncoderBuilder matching original behavior:
  - Fixed stream serial 0x4C43_5246, VBR target bitrate (64-500 kbit/s)
  - 4096-frame blocks, panic catch with QualityVbr fallback (target_quality: 0.6)
- Add EncodeFormat::Ogg variant (cfg-gated on vorbis-encode)
- Update ENCODE_EXTENSIONS and encode_format_for() to include .ogg
2026-11-14 10:02:09 -05:00
alehel 4732a89b02 craft-audio: review of the format queries
- EncodeFormat::max_channels gave 65535 for WAV and AIFF, whose headers
  hold 32767 (a WAV fewer at wider samples, which the doc now says).
- Tests pin the AAC and MP3 layouts, each format's channel limit with the
  count above it, and every listed sample rate with its neighbours: the
  agreement test alone compared the tables with a writer that reads them.
- Docs: FileEncoder::bitrate points at the rule RateControl::Bitrate
  states; the README says when check exists.

Refs #1230
2026-11-14 09:37:54 +01:00
alehel 1570d6a29a craft-audio: ask what a format takes before writing a file
- EncodeOptions::check says whether a file can be begun, with the error
  FileEncoder::new would return and nothing written.
- EncodeOptions::bitrate and FileEncoder::bitrate give the constant bit
  rate actually used, after it is moved to one the stream can carry.
- EncodeFormat::sample_rates, max_channels, fixed_layouts, supports_layout
  and supports_quality say what a format takes, also when its feature is
  off. The writer reads its layouts from the same table, and tests hold
  the tables to the encoders and to what the writer accepts.

Refs #1230
2026-11-14 08:48:19 +01:00
alehel 6f7c041a09 craft-audio: tests of the coded formats hold on ffmpeg 8 and without AAC
- ffmpeg 8 skips an AAC M4A's priming but plays the last packet's padding
  (ffmpeg 9 cuts it): the ffmpeg oracle test takes either length, and the
  README says which readers end the audio where.
- With alac-encode and no aac-encode, `m4a` names ALAC: the extension test
  compared it with AAC.

Refs #1230
2026-11-14 08:31:05 +01:00
alehel e5fa1ac2f9 craft-audio: review round 4 for the coded formats
- craft-media: Mp4Writer counts written bytes with a checked sum, so a
  writer that takes bytes past the last position gets an error, not an
  overflow (correction to round 3, which checked only the header).
- Docs: the AAC bit-rate floor keeps a margin above where the encoder
  codes silence, about half of it.

Refs #1230
2026-11-14 03:22:06 +01:00
alehel 098eb2e43a craft-audio: review round 3 for the coded formats
- craft-media: Mp4Writer::new refuses a writer positioned where the header
  would end past u64::MAX instead of overflowing the sum (correction to
  round 2, which let the M4A writer reach that addition).
- AAC is written at 6 kbit/s or a quarter of the sample rate per channel at
  least: the encoder accepts lower rates but codes only silence at them, so
  with_bitrate(128) gave a silent file.
- README: what to do with a layout a coded format refuses, and the names
  encode_format_for takes for M4A.

Refs #1230
2026-11-14 03:16:51 +01:00
alehel 63712c33e6 craft-audio: review round 2 for the coded formats
- craft-media: Mp4Writer counts chunk offsets from the first byte of the
  file, not from position 0 of the writer, so a file begun mid-writer is
  valid. This replaces the seek adapter the M4A writer wrapped its sink in.
- Tests pin the bytes of the roll sample group and each format's feature.
- Docs: ALAC's length comes from the sample durations, not an edit list;
  which readers take an empty AAC file; speaker names as the errors print
  them; craft-media's README lists roll_distance.
- Error messages for a layout a format has not no longer suggest setting
  a layout that was set.

Refs #1230
2026-11-14 03:16:51 +01:00
alehel 3dadce4d7c craft-audio: review round 1 for the coded formats
- craft-media: TrackConfig::roll_distance writes a `roll` sample group
  (sgpd + sbgp) over every sample. The AAC M4A track sets -1: Apple's
  decoders took 2112 frames of priming and dropped the start of the audio
  without it (afinfo now reports the frames written and 1024 priming).
- EncodeFormat::is_built and feature: which formats this build writes.
- Clippy: the writer enum's variants differ in size with flac-encode and
  a codec feature but no pcm-encode.
- Tests for the finished sample entry (AAC bit rate, ALAC cookie), ALAC
  dither and afinfo's reading of the priming.
- README: ffmpeg's reading of 8-channel AAC, and files with no audio.

Refs #1230
2026-11-14 03:16:51 +01:00
alehel eee76c3fc2 craft-audio: AAC, ALAC, MP3, AC-3 and E-AC-3 as formats of encode and FileEncoder
EncodeFormat gains Aac and Alac (M4A through craft-media's MP4 muxer) and
Mp3, Ac3 and Eac3 (raw streams), so one call writes any of them from
WAV-order f32 audio and EncodeOptions.

- EncodeOptions::rate_control (with_bitrate, with_quality): a bit rate is
  moved to the nearest one the stream can carry; quality is AAC's VBR.
- Channels are mapped onto the codec's order by speaker, from
  EncodeOptions::layout. A layout the format has not is Unsupported
  instead of reaching the wrong speakers.
- M4A: the edit list skips the AAC priming, and the last sample's duration
  is cut so the track ends at the last frame written.
- encode_format_for, ENCODE_EXTENSIONS and EncodeFormat::ALL list the new
  formats; aac-encode and alac-encode now turn on craft-media's isobmff.

Refs #1230
2026-11-14 03:16:51 +01:00
alehelandClaude Opus 5.5 7c652beec4 craft-audio: review fixes, round 1 (#446)
- `Encoder::with_application` checks the channel count before clamping
  the bitrate, so 86 channels or more return `Error::Channels` instead
  of panicking in `clamp`. Tested with 86 and 255.
- The ffmpeg oracle test skips a missing `opus`/`libopus` decoder
  through the testkit's `skip`, so `CRAFT_MEDIA_REQUIRE_ORACLES=1` fails
  it.
- README: the encode example pads at least the pre-skip after the audio
  before rounding up to a frame, so the encoder delay is flushed; the
  MP4 note says an explicit edit replaces the `media_start` edit and
  gives its fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:09:46 -05:00
alehelandClaude Opus 5.5 bdf8cb2a19 craft-audio: Opus encoder
EffectCraft's `effectcraft-opusenc` as `opus::encoder` (and
`opus::Encoder`) behind `opus-encode`: SILK, hybrid and CELT, 48 kHz,
20 ms frames, mono or stereo. The constructors and `encode_float`
return `Result` instead of panicking. `Bandwidth` is one type for the
decoder and the encoder, and the two `Mode` enums convert into each
other. `encode` now turns on `opus-encode`.

The encoder tests (modes, ffmpeg's two Opus decoders as oracles) and
`TrackDecoder` on Opus without an `OpusHead` come with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:09:46 -05:00
Dylan M. Taylor 68e53e2fee Merge pull request #1130 from alehel/claude/he-aac-codec-support-53a592
craft-audio: decode the HE-AAC SBR high band
2026-11-13 05:46:15 -05:00
alehelandClaude Opus 5.5 1f3be111b8 craft-audio: drop a short last SBR patch before the patch limit
A header whose sixth patch is short (96 kHz, start 3, stop 9) was
rejected, though that patch is dropped and five remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 09:56:22 +01:00
Dylan M. Taylor 47107f6601 craft-mesh: retain glTF texture and sampler inventory (#1151)
* craft-mesh: expose unreferenced glTF texture variants

* mesh: validate texture inventory image indices

* Fix PR 1151 validation and workspace checks

* Fix remaining Rust 1.95 workspace lint findings

* Fix proof workspace Clippy warning

* Fix remaining workspace Clippy findings

* Fix video all-features Clippy warning

* KRA: parse nested layers without recursion

* Preserve KRA layer limit traversal order

* Avoid expect in iterative KRA parser

* OpenRaster: parse nested stacks without recursion

* Plate tests honor optional font fixtures

* Keep bundled font index bytes unchanged

* Preserve font build license fixture bytes

* ci: allow longer workspace test matrix

* test: respect Windows Hunspell path behavior

* ci: provision keymap fonts for all-feature tests

* ci: provision PSD corpus for full workspace tests
2026-11-13 02:17:30 -05:00
alehelandClaude Opus 5.5 fe8c1f697a craft-audio: keep the SBR history when a new program config fails
The SBR decoders now record the layout they were built for and are rebuilt when the
decoder's layout differs, instead of being cleared in set_layout. An access unit whose new
program config fails puts the layout back, and with it the SBR decoders still match.

Refs #1121

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 07:34:10 +01:00
alehelandClaude Opus 5.5 48f5283ec8 craft-audio: cite no section number for l_A
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 07:34:10 +01:00
alehelandClaude Opus 5.5 fa9ca36697 craft-audio: SBR review fixes
- A VARFIX pointer of 1 marks no transient envelope, like 0.
- A start at QMF channel 32 is valid; kx is checked once the crossover is known.
- The noise and harmonic gains take q / (1 + q) first, so two clamped levels no longer
  overflow to infinity and NaN.
- A header that runs past its fill element is refused before it replaces the one in use.
- A new program config layout drops the SBR decoders, whose history was the old order's.
- The ffmpeg oracle requires a high band to compare on every channel but the 5.1 LFE.

Refs #1121

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 07:34:10 +01:00
alehelandClaude Opus 5.5 f68e4369eb craft-audio: decode the HE-AAC SBR high band
aac::Decoder::set_sbr(true) decodes HE-AAC at twice the core rate with the
SBR tool of ISO/IEC 14496-3 §4.6.18: QMF analysis and synthesis, frequency
band tables, patching with LPC inverse filtering, envelope adjustment with
limiter, boost, noise and sinusoids. TrackDecoder uses it in place of the
2x upsampler. Tables are transcribed from ETSI ES 201 980 Annex I.

Matches ffmpeg's decoder on AudioToolbox HE-AAC streams of 1-6 channels:
the same 962-sample delay, ~54 dB SNR, high band level within 0.01 dB.
Parametric stereo is still not decoded.

Refs #1121

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 07:34:10 +01:00
Aleksander Helgaker 493476eb8d craft-audio: native MP3 (Layer III) decoder (feature mp3-decode)
* craft-audio: native MP3 (Layer III) decoder (feature `mp3-decode`)

A clean-room Layer III decoder written from ISO/IEC 11172-3 and 13818-3:
MPEG-1, MPEG-2 and MPEG-2.5 (8-48 kHz), CBR and VBR, mono, stereo, dual
channel and joint stereo (mid/side and intensity), long, short and mixed
blocks, and the bit reservoir. It shares the encoder's Huffman and
scalefactor band tables and the MP1/MP2 synthesis filter.

`decode`, `Decoder` and `TrackDecoder` now read raw .mp3 files and MP3 in
MP4 and Matroska natively instead of through symphonia. A Xing / Info / VBRI
frame is not a packet, and the LAME tag's encoder delay and padding are
applied, so a file decodes to its exact presented length. The seek
pre-roll covers how far back the bit reservoir can reach, so a seek is
bit-exact. symphonia stays the fallback for free-format MP3.

The encoder moves to mp3/encoder.rs so that each half builds without the
other.

* craft-audio: MP3 review and CI fixes

- Huffman: big values stop at the granule's part2_3_length. A damaged
  granule no longer reads the next granule's bits, as ffmpeg does.
- Joint stereo runs on the lines in coded order, before short blocks are
  reordered, as ffmpeg and libmad do. Where the two channels' block layouts
  differ, against the specification, each channel is then reordered by its
  own layout.
- Seek pre-roll: sized from the smallest frame the stream can hold (the
  lowest bit rate for its version, rate, mode and CRC), not the last frame,
  so a seek into a VBR stream's small frames still has the reservoir. The
  decoder learns the stream's format from its first frame.
- Mixed-block oracle test: edit only granules whose scalefactors take no
  bits, so the edited streams stay valid, and not before a pure short
  block. ffmpeg's short-block IMDCT drops overlap that a mixed block's long
  subbands carry, where the specification adds it. Use 16 kHz instead of
  12 kHz.
- Routing oracle: ffmpeg before 9 decodes the 1105 samples of the MP3
  Matroska CodecDelay.

* craft-audio: MPEG audio packets of several frames keep their length

An MP4 or Matroska packet may hold several MP1/MP2/MP3 frames.
TrackDecoder now counts the frames each packet holds from their headers:

- frames_per_packet is None once a packet in the lookahead or a decoded
  packet has held several frames, so an MP4 track's length is no longer
  capped at one frame per sample and the container's timing decides.
- Concealment is the length of the frames a damaged packet holds. For MP3
  a packet lost whole is as long as the last packet, and one frame before
  any is decoded.

Also corrects the Huffman doc comment: a big-value pair that starts
before the granule's end is kept whole, as ffmpeg does.

* craft-audio: MPEG audio concealment after partial packets and before the first

- A packet whose later frame fails decodes only in part: like AC-3, it no
  longer replaces the length of the last whole packet that concealment
  uses.
- MP1/MP2: a packet lost before any has decoded, or right after a seek, is
  one frame (384 or 1152 samples) of silence rather than none, as MP3's
  already is.

---------

Co-authored-by: alehel <git@ornumhelgaker.no>
2026-11-12 09:12:40 -05:00