daniel543x/craft-libs
1
0 Fork 0
mirror of https://github.com/storytold/craft-libs.git synced 2026-11-13 22:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

craft-audio: Opus decoder (7/10) (#445)

Browse Source 
* craft-audio: Opus decoder

FilmCraft's `filmcraft-opus` decoder as `craft_audio::opus` behind
`opus-decode`: SILK, hybrid and CELT, every frame size, PLC and FEC,
multistream (`Decoder`) and single stream (`StreamDecoder`), 8-48 kHz
output.

- `TrackDecoder` decodes Opus; mono and stereo without an `OpusHead`
  (RTP, MPEG-TS) get a default one. Lost packets use Opus's own
  concealment.
- The native read path takes Ogg Opus (pre-skip and end trimming per
  RFC 7845), and Opus in Matroska/WebM and MP4, where packet starts
  after a seek come from the packets' own lengths.

The RFC 8251 conformance test runs all 36 vectors.
`scripts/fetch-opus-vectors.sh` downloads them (SHA-256 checked) and
the CI test job caches them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#445)

- CELT concealment keeps its pitch in its own field. It reused postfilter_period_old, which the
  next good frame's post-filter cross-fades from.
- Matroska Opus seeks: a failed packet read no longer caches a wrong start; a one-byte packet
  is read as one byte.
- A malformed OpusHead in Ogg is reported as malformed instead of falling back.
- The libopus oracle tests skip through the testkit, so CRAFT_MEDIA_REQUIRE_ORACLES fails them.
- Decoder::reset documents that it re-arms pre-skip trimming.
- fetch-opus-vectors.sh retries its size probe.
- CI: the vectors are fetched after ffmpeg is installed, and apt's timeout runs as root so a
  stalled apt-get is killed and frees the dpkg lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#445)

- CELT: the noise-based concealment sets skip_plc, as libopus does, so
  pitch-based concealment waits for two good frames in a row instead of
  extending history that holds noise.
- SILK: decode_only_middle is per call, as in libopus. A lost or mono
  frame no longer reuses the mid-only flag from an earlier frame.
- A damaged Opus packet is concealed at its own length (from its TOC and
  framing) rather than the previous packet's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
This commit is contained in:
Aleksander HelgakerClaude Opus 5.5Dylan M. Taylor
 authored and GitHub committed 2026-11-11 07:36:57 -05:00
1 parent 5b58973bc6
commit 861c5e172d
34 files changed
+7863 -83
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/audio.yml
+23 -10
View File
@@ -3,9 +3,9 @@ name: Shared audio
on:
push:
branches: [main]
paths: ["crates/audio/**", "crates/media/**", "crates/media-testkit/**", "Cargo.toml", "Cargo.lock", ".github/workflows/audio.yml"]
paths: ["crates/audio/**", "crates/media/**", "crates/media-testkit/**", "scripts/fetch-opus-vectors.sh", "Cargo.toml", "Cargo.lock", ".github/workflows/audio.yml"]
pull_request:
paths: ["crates/audio/**", "crates/media/**", "crates/media-testkit/**", "Cargo.toml", "Cargo.lock", ".github/workflows/audio.yml"]
paths: ["crates/audio/**", "crates/media/**", "crates/media-testkit/**", "scripts/fetch-opus-vectors.sh", "Cargo.toml", "Cargo.lock", ".github/workflows/audio.yml"]
workflow_dispatch:
permissions:
@@ -39,7 +39,7 @@ jobs:
# Each feature on its own, so code that needs another feature fails here.
- name: Clippy, each feature alone
run: |
for f in "" decode decode-native symphonia encode lossless-encode pcm-encode flac-encode peaks aac-decode aac-encode "aac-encode,rayon" serde; do
for f in "" decode decode-native symphonia encode lossless-encode pcm-encode flac-encode peaks aac-decode aac-encode "aac-encode,rayon" opus-decode serde; do
echo "::group::features: ${f:-none}"
cargo clippy -p craft-audio --all-targets --locked --no-default-features ${f:+--features "$f"} -- -D warnings
echo "::endgroup::"
@@ -92,7 +92,7 @@ jobs:
- run: cargo test -p craft-audio --no-default-features --features decode-native --locked
test:
timeout-minutes: 20
name: Audio tests with ffmpeg oracles
name: Audio tests with ffmpeg oracles and the Opus conformance vectors
# Pinned for the ffmpeg its apt ships (8.0), so the files the oracle tests write do not change
# with the runner image.
runs-on: ubuntu-26.04
@@ -105,23 +105,36 @@ jobs:
with:
key: craft-audio-test
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Build tests while installing ffmpeg
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ runner.temp }}/media-fixtures/opus/vectors
# The archive's SHA-256 (scripts/fetch-opus-vectors.sh checks it).
key: opus-testvectors-rfc8251-6b26a22f
- name: Build tests while installing ffmpeg and fetching the Opus vectors
run: |
# The man-db index rebuild after dpkg is slow on hosted runners.
sudo rm -f /var/lib/man-db/auto-update
# A mirror can stall for most of the job's time limit, and the next try usually gets a good one.
install_ffmpeg() {
for attempt in 1 2 3; do
timeout -k 10 180 sudo sh -c 'apt-get update -qq && apt-get install -y -qq --no-install-recommends ffmpeg' && return 0
# timeout runs as root, so a stalled apt-get is killed too and frees the dpkg lock.
sudo timeout -k 10 180 sh -c 'apt-get update -qq && apt-get install -y -qq --no-install-recommends ffmpeg' && return 0
echo "apt: attempt $attempt failed with exit code $?"
timeout -k 10 60 sudo dpkg --configure -a || true
sudo timeout -k 10 60 dpkg --configure -a || true
done
return 1
}
install_ffmpeg > "$RUNNER_TEMP/apt.log" 2>&1 &
apt=$!
# The vectors are fetched after ffmpeg, so the two downloads do not slow each other.
{
install_ffmpeg > "$RUNNER_TEMP/apt.log" 2>&1 || exit 1
scripts/fetch-opus-vectors.sh "$RUNNER_TEMP/media-fixtures/opus/vectors" > "$RUNNER_TEMP/vectors.log" 2>&1 || exit 2
} &
fetch=$!
cargo test -p craft-audio --all-features --locked --no-run
wait "$apt" || { cat "$RUNNER_TEMP/apt.log"; exit 1; }
wait "$fetch" || case $? in
1) cat "$RUNNER_TEMP/apt.log"; exit 1 ;;
*) cat "$RUNNER_TEMP/vectors.log"; exit 1 ;;
esac
ffmpeg -hide_banner -version | sed -n 1p
- run: cargo test -p craft-audio --all-features --locked
env:
README.md
+1 -1
View File
@@ -92,7 +92,7 @@ craft-<name> = { git = "https://github.com/storytold/craft-libs", rev = "<commit
scopes, conflicts, user overrides stored through craft-settings and import of other
applications' shortcut sets. Command catalogs, file readers and the shortcut editor UI stay
in each app.
- [`craft-audio`](crates/audio/README.md): audio file I/O and codecs (WAV, AIFF, FLAC, AAC),
- [`craft-audio`](crates/audio/README.md): audio file I/O and codecs (WAV, AIFF, FLAC, AAC, Opus),
streaming and seeking, and packet decoding for any demuxer. See its README for formats and
features.
- [`craft-media`](crates/media/README.md): bit-level I/O (`bitstream`), clean-room MP4/MOV
crates/audio/Cargo.toml
+6 -5
View File
@@ -1,6 +1,6 @@
[package]
name = "craft-audio"
description = "Audio file I/O and codecs: WAV/AIFF read and write, FLAC encoder, AAC decoder and encoder."
description = "Audio file I/O and codecs: WAV/AIFF read and write, FLAC encoder, AAC decoder and encoder, Opus decoder."
version.workspace = true
edition.workspace = true
license.workspace = true
@@ -14,12 +14,12 @@ all-features = true
workspace = true
[features]
# Probe and decode files: WAV and AIFF natively; AAC and PCM through craft-media's demuxers and
# this crate's decoders; MP1/MP2/MP3, FLAC, ALAC, Vorbis, CAF, ADPCM, A-law and µ-law through
# symphonia.
# Probe and decode files: WAV and AIFF natively; AAC, Opus and PCM through craft-media's
# demuxers and this crate's decoders; MP1/MP2/MP3, FLAC, ALAC, Vorbis, CAF, ADPCM,
# A-law and µ-law through symphonia.
decode = ["decode-native", "symphonia"]
# `decode` without symphonia: only the formats with a native decoder (others are Unsupported).
decode-native = ["aac-decode", "craft-media/isobmff", "craft-media/matroska", "craft-media/ogg", "dep:log"]
decode-native = ["aac-decode", "opus-decode", "craft-media/isobmff", "craft-media/matroska", "craft-media/ogg", "dep:log"]
# The symphonia fallback for the formats with no native decoder yet.
symphonia = ["decode-native", "dep:symphonia"]
# Every encoder: the lossless file writers and the AAC encoder.
@@ -33,6 +33,7 @@ flac-encode = ["craft-media/bitstream"]
# Codec modules on their own, for packet-level use.
aac-decode = ["craft-media/bitstream"]
aac-encode = ["craft-media/bitstream"]
opus-decode = []
# Multi-resolution waveform overviews (`peaks`).
peaks = []
# Encode AAC channels in parallel (with `aac-encode`).
crates/audio/README.md
+45 -23
View File
@@ -1,7 +1,7 @@
# craft-audio
Audio file I/O and codecs for the Craft apps. Native implementations: WAV and AIFF read/write,
FLAC encoder, AAC decoder/encoder.
FLAC encoder, AAC decoder/encoder, Opus decoder.
Formats without a native decoder are decoded by [symphonia](#symphonia), which is to be replaced
by native decoders.
@@ -11,7 +11,7 @@ by native decoders.
per channel), in WAV channel order (L R C LFE, back, side) whenever the channel layout is known.
- Malformed input returns an error, never a panic (see [symphonia](#symphonia) for the one
exception). No `unsafe`. Readers and decoders are tested with corrupted and truncated input.
- Implemented from the published specifications; tested against ffmpeg.
- Implemented from the published specifications; tested against ffmpeg and conformance vectors.
Builds for `wasm32-unknown-unknown`.
- Containers (MP4/MOV, Matroska/WebM, Ogg), bit I/O and the `Container`/`CodecId`/`CodecParams`
types come from [`craft-media`](../media/README.md). This crate never depends on video codecs.
@@ -24,18 +24,20 @@ No default features. The shared types (`AudioBuffer`, `AudioInfo`, `ChannelLayou
| Feature | Provides | Pulls in |
|---|---|---|
| `decode` | `decode`, `probe`, `Decoder`, `tracks`, `TrackDecoder`; every format in [Format support](#format-support) | `decode-native`, `symphonia` |
| `decode-native` | the same, native formats only (others are `Unsupported`) | `aac-decode`, craft-media `isobmff`/`matroska`/`ogg`, `log` |
| `decode-native` | the same, native formats only (others are `Unsupported`) | the two decoders below, craft-media `isobmff`/`matroska`/`ogg`, `log` |
| `symphonia` | the fallback for formats with no native decoder; turning it on is the same as `decode` | `decode-native`, symphonia |
| `encode` | every encoder: the WAV, AIFF and FLAC file writers, and the AAC packet encoder | `lossless-encode`, `aac-encode` |
| `lossless-encode` | WAV, AIFF and FLAC writing | `pcm-encode`, `flac-encode` |
| `pcm-encode` | `encode`, `FileEncoder` for WAV and AIFF; `wav_float_header` | none |
| `flac-encode` | `encode`, `FileEncoder` for FLAC | craft-media `bitstream` |
| `aac-decode`, `aac-encode` | `aac::Decoder`, `aac::Encoder` | craft-media `bitstream` |
| `opus-decode` | `opus::Decoder`/`StreamDecoder` | none |
| `peaks` | `peaks::Peaks` | none |
| `rayon` | parallel AAC encoding (with `aac-encode`) | `rayon` |
| `serde` | `Serialize`/`Deserialize` on the shared types | `serde` |
`aac-decode` also gives `TrackDecoder`, for AAC and PCM.
Any one decoder feature (`aac-decode`, `opus-decode`) also gives
`TrackDecoder`, for that codec and PCM.
An app that decodes 32-bit FLAC with `decode` needs a profile override in its workspace; see
[symphonia](#symphonia).
@@ -150,8 +152,8 @@ known stops at the end, and the next `next_block` returns `None`. When `length_k
of the file to count them. `Decoder::probe` is also how to probe a file on disk.
Native formats read their samples from the source packet by packet. The container's packet index
stays in memory (tens of bytes per packet), and opening a Matroska file or a raw ADTS stream reads
through the whole file to build it. Formats symphonia decodes are read whole first.
stays in memory (tens of bytes per packet), and opening an Ogg or Matroska file or a raw ADTS
stream reads through the whole file to build it. Formats symphonia decodes are read whole first.
A damaged packet is concealed and counted in `info().damaged`. A source that fails a read (a
network drop) returns `AudioError::Io` from `next_block` instead, and that packet is skipped. To
@@ -176,18 +178,20 @@ for p in &packets {
}
```
It decodes AAC (finding HE-AAC in the lookahead) and PCM, and returns `AudioBuffer`s in WAV
order when `layout()` is ordered. `codec()`, `sample_rate()` and `layout()` describe the output. Container timing is the caller's.
It decodes AAC (finding HE-AAC in the lookahead), Opus and PCM, and returns `AudioBuffer`s in WAV
order when `layout()` is ordered. `codec()`, `sample_rate()` and `layout()`
describe the output. Container timing is the caller's.
Streams without codec configuration work where the packets carry it. AAC with empty `extradata`
takes its configuration from the ADTS headers (MPEG-TS, raw `.aac`), which are stripped from each
packet.
packet. Opus with empty `extradata` and 1 or 2 channels (RTP, MPEG-TS) needs no `OpusHead`.
To seek: position your demuxer at the packet that starts `preroll()` frames or more before the
target, call `reset()`, decode from there, and drop the output before the target.
The codec modules are also usable on their own. `aac::Decoder` returns the codec's channel order;
`aac::wav_order(channel_config)` maps it to WAV order.
The codec modules are also usable on their own. `aac::Decoder` and `opus::Decoder` return the
codec's channel order; `aac::wav_order(channel_config)` and `opus::wav_order(channels)` map it to
WAV order.
### Encode a file
@@ -289,14 +293,14 @@ match craft_audio::decode(&bytes, None) {
| AAC-LC | native | native |
| HE-AAC v1/v2 | native, core only (see [Limits](#limits)) | — |
| MP1, MP2, MP3 (MPEG audio Layer I/II/III) | symphonia | — |
| Opus | native | — |
| ALAC, Vorbis, ADPCM, A-law, µ-law | symphonia | — |
| Opus | — | — |
| Container | Demuxer | Timing |
|---|---|---|
| MP4, M4A, MOV | `craft_media::isobmff` for native codecs; symphonia for the others (ALAC, MP3, FLAC) | edit lists (AAC priming) |
| MP4, M4A, MOV | `craft_media::isobmff` for native codecs; symphonia for the others (ALAC, MP3, FLAC) | edit lists (AAC priming, Opus pre-skip) |
| Matroska, WebM | `craft_media::matroska` for native codecs; symphonia for the others (Vorbis, FLAC, MP3) | `CodecDelay`, `DiscardPadding` |
| Ogg | symphonia (Vorbis, FLAC) | — |
| Ogg | `craft_media::ogg` (Opus); symphonia (Vorbis, FLAC) | pre-skip, end trimming |
| Raw ADTS AAC | native frame sync | — |
| Raw MP1/MP2/MP3 | symphonia | — |
| FLAC | symphonia | — |
@@ -316,14 +320,17 @@ match craft_audio::decode(&bytes, None) {
decode's, at the same band energy.
- MP4 gapless information outside an edit list is not read: a file from Apple's encoders with
`iTunSMPB` and no `elst` decodes with its priming and padding.
- AAC channel configuration 7 and program-config layouts, and AIFF with more than four channels,
decode with an unordered layout, in the codec's order.
- A Matroska track with no block durations (AAC from ffmpeg 6) ends at its `DURATION` tag, when
that ends inside the last packet. Otherwise all its audio is presented, including any padding.
- ffmpeg before 9 ends some codecs' tracks in MP4 and Matroska up to 11 ms early. The audio past
the end a file declares is not presented.
- A Matroska track's end comes from its timestamps, so it is known to one timestamp tick (usually
1 ms) unless the last block has `DiscardPadding`.
- WebM Opus: `probe` can over-report by up to one block (the last block's `DiscardPadding` is
known only when it is read); `decode` and `Decoder` are exact.
- AAC channel configuration 7 and program-config layouts, Opus mapping families other than 0 and
1, and AIFF with more than four channels decode with an unordered layout, in the codec's order.
- Not decoded at all: AC-3, E-AC-3 and Speex.
- A FLAC layout that decoders would not assume for the channel count depends on its
`WAVEFORMATEXTENSIBLE_CHANNEL_MASK` comment. This includes the usual layouts for 5 and 6 channels:
without the comment, back surrounds are read as side surrounds. It includes the usual 7-channel
@@ -334,7 +341,6 @@ match craft_audio::decode(&bytes, None) {
sample rate the frame header cannot name, or with a layout that needs the mask comment.
- `ChannelLayout::unordered` is written to FLAC as mask `0x0000`, as RFC 9639 §8.6.2 allows, but
ffmpeg and `flac -d` ignore that mask and use their default layout for the channel count.
- Not decoded at all: AC-3, E-AC-3, Opus and Speex.
- `TrackDecoder` returns `Unsupported` for MP3, FLAC, ALAC and Vorbis; `decode` and `Decoder` read
those through symphonia.
- MP4 edit lists: a single media edit (encoder priming and the end) is applied, alone or after one
@@ -370,6 +376,7 @@ Each codec module's rustdoc lists the coding tools it implements.
| WAV, AIFF | Microsoft RIFF, Apple AIFF/AIFF-C | read: integer 8–32-bit, float 32/64-bit, `WAVE_FORMAT_EXTENSIBLE` masks, RF64/BW64, BWF `bext`, truncated files; write: 16/24/32-bit integer and 32-bit float, RF64 above 4 GiB, any layout, TPDF dither | round trips; hostile-input tests |
| FLAC encoder | RFC 9639 | 16, 24, 32 bits; 1–8 channels, layouts decoders would not assume for the count as a `WAVEFORMATEXTENSIBLE_CHANNEL_MASK` comment; up to 655,350 Hz; LPC, stereo decorrelation, wasted bits, STREAMINFO MD5 | bit-exact through ffmpeg, STREAMINFO MD5 checked against ffmpeg |
| AAC (`aac`) | ISO/IEC 14496-3 | AAC-LC decode and encode (CBR, VBR, ADTS), 1–8 channels; HE-AAC: see [Limits](#limits) | ffmpeg: ~135 dB SNR; PNS noise differs per sample at equal band energy |
| Opus (`opus`) | RFC 6716, 8251, 7845 | decoder: SILK, hybrid, CELT, all frame sizes, PLC and FEC, multistream (`Decoder`) and single stream (`StreamDecoder`), 8–48 kHz output | all 36 RFC 8251 conformance cases |
## symphonia
@@ -408,22 +415,34 @@ the unit tests. Run everything with:
cargo test -p craft-audio --all-features
```
Tests that need ffmpeg are skipped when it is missing. To fail instead, set
`CRAFT_MEDIA_REQUIRE_ORACLES=1`. The helpers and their variables are shared with craft-media (see
[its tests](../media/README.md#tests)). The test files describe what they check;
Tests that need ffmpeg or the Opus vectors are skipped when those are missing. To fail instead,
set `CRAFT_MEDIA_REQUIRE_ORACLES=1`. The helpers and their variables are shared with craft-media
(see [its tests](../media/README.md#tests)). The test files describe what they check;
`tests/codecs/routing_oracle.rs` is the end-to-end check of containers, codecs, timing, seeking
and track choice against ffmpeg.
The Opus conformance test needs the RFC 8251 vectors (75 MB, not committed). Run the script from
the repository root; it fetches them into `target/fixtures/opus/vectors`. With
`CRAFT_MEDIA_FIXTURES_DIR` set, pass the directory:
```bash
scripts/fetch-opus-vectors.sh
```
```bash
scripts/fetch-opus-vectors.sh "$CRAFT_MEDIA_FIXTURES_DIR/opus/vectors"
```
CI (`.github/workflows/audio.yml`) runs clippy for each feature on its own, rustdoc with warnings
as errors, the WASM build, encode-only and native-only test runs, and an MSRV check. It also runs
the full tests with ffmpeg required.
the full tests with ffmpeg and the vectors required.
## Origin
| Part | Source |
|---|---|
| WAV, AIFF, `decode`/`probe`, peaks | SoundCraft `soundcraft-audio-io` [7dc56a4](https://github.com/storytold/soundcraft/commit/7dc56a4dbeebf57eacece09536726c57844fc59d) |
| FLAC encoder, AAC | FilmCraft [8fda2f2](https://github.com/storytold/filmcraft/commit/8fda2f232198c4c79ff9cfe5b6a347bb1375fa9e) |
| FLAC encoder, AAC, Opus decoder, Ogg | FilmCraft [8fda2f2](https://github.com/storytold/filmcraft/commit/8fda2f232198c4c79ff9cfe5b6a347bb1375fa9e) |
Where apps had overlapping implementations, the one that measured better was kept, with the
tests of both:
@@ -463,10 +482,13 @@ The codec modules' public items keep their names, so apps can re-export them
of `pcm-encode` (WAV, AIFF) and `flac-encode`. `encode` turns on every encoder, AAC
included. Writing a format whose feature is off returns `AudioError::Unsupported`.
- AAC encoder settings it cannot use are `aac::Error::InvalidSettings` (`AudioError::Encode`).
- `TrackDecoder::decode` returns an error for a corrupt Opus packet, as for the other codecs;
call `conceal`.
- `FileEncoder::write` takes any per-channel slices, but refuses a block with missing channels or
short slices instead of filling them with silence.
- `DecodeOptions` is not `Copy`.
- The FLAC encoder's vendor string is `craft-audio <version>`.
- `decode` tries native decoders first. AAC no longer goes through symphonia, except in CAF.
- `decode` tries native decoders first. AAC and Opus no longer go through symphonia, except AAC in
CAF.
MP4 and Matroska files decode to presentation length. A packet that fails is concealed, not
dropped. A failed source read is `AudioError::Io`.
crates/audio/src/layout.rs
+1 -1
View File
@@ -145,7 +145,7 @@ impl TryFrom<RawLayout> for ChannelLayout {
}
/// A channel count as the API holds it, saturating at `u16::MAX`.
#[cfg(any(feature = "aac-decode", feature = "pcm-encode"))]
#[cfg(any(feature = "aac-decode", feature = "opus-decode", feature = "pcm-encode"))]
pub(crate) fn count(n: usize) -> u16 {
u16::try_from(n).unwrap_or(u16::MAX)
}
crates/audio/src/lib.rs
+18 -6
View File
@@ -8,8 +8,8 @@
//! `wasm32-unknown-unknown`.
//!
//! - [`decode`], [`probe`], [`Decoder`] (streaming, seekable), [`tracks`], [`detect_container`]
//! (feature `decode`): WAV and AIFF are read natively; AAC and PCM in MP4/MOV,
//! Matroska/WebM or raw ADTS streams go through craft-media's demuxers and
//! (feature `decode`): WAV and AIFF are read natively; AAC, Opus and PCM in MP4/MOV,
//! Matroska/WebM, Ogg or raw streams go through craft-media's demuxers and
//! [`TrackDecoder`]; everything else (MP1/MP2/MP3, FLAC, ALAC, Vorbis, CAF, ADPCM, A-law,
//! µ-law) through `symphonia`, which is to be replaced by native decoders. Feature
//! `decode-native` is the same without symphonia.
@@ -21,7 +21,7 @@
//! `lossless-encode` and `aac-encode`.
//! - [`peaks`] (feature `peaks`): min/max/RMS pyramid for waveform drawing.
//! - Codec modules for packet-level use, each with its own features: [`aac`] (`aac-decode`,
//! `aac-encode`).
//! `aac-encode`), [`opus`] (`opus-decode`).
//!
//! The shared types below build with no features.
@@ -29,13 +29,15 @@
pub mod aac;
mod buffer;
mod layout;
#[cfg(feature = "aac-decode")]
#[cfg(feature = "opus-decode")]
pub mod opus;
#[cfg(any(feature = "aac-decode", feature = "opus-decode"))]
mod pcm;
#[cfg(feature = "peaks")]
pub mod peaks;
#[cfg(feature = "decode-native")]
mod read;
#[cfg(feature = "aac-decode")]
#[cfg(any(feature = "aac-decode", feature = "opus-decode"))]
mod track;
#[cfg(any(feature = "decode-native", feature = "pcm-encode"))]
mod wav;
@@ -52,7 +54,7 @@ pub use layout::{ChannelLayout, Speaker};
pub use read::SharedSource;
#[cfg(feature = "decode-native")]
pub use read::{DEFAULT_MAX_SAMPLES, DecodeOptions, Decoder, TrackInfo, decode, decode_with, detect_container, probe, probe_with, tracks, tracks_of};
#[cfg(feature = "aac-decode")]
#[cfg(any(feature = "aac-decode", feature = "opus-decode"))]
pub use track::{LOOKAHEAD_PACKETS, TrackDecoder};
#[cfg(feature = "pcm-encode")]
pub use write::wav_float_header;
@@ -99,6 +101,16 @@ impl From<aac::Error> for AudioError {
}
}
#[cfg(feature = "opus-decode")]
impl From<opus::Error> for AudioError {
fn from(e: opus::Error) -> Self {
match e {
opus::Error::Unsupported(_) => AudioError::Unsupported(e.to_string()),
_ => AudioError::Malformed(e.to_string()),
}
}
}
/// The file formats [`encode`] writes. Each needs its feature: WAV and AIFF `pcm-encode`, FLAC
/// `flac-encode`; asking for one that is not built returns [`AudioError::Unsupported`].
#[non_exhaustive]
crates/audio/src/opus/decode/celt/bands.rs
+988
View File
@@ -0,0 +1,988 @@
//! Band shape decoding: PVQ, spreading, TF changes, band splitting, stereo, folding
//! (RFC 6716 §4.3.4 – §4.3.6).
use super::rate::{Mode, QTHETA_OFFSET, QTHETA_OFFSET_TWOPHASE, get_pulses};
use super::tables::{E_MEANS, EBANDS, NB_EBANDS, ORDERY, SHORT_MDCT};
use crate::opus::decode::range::{BITRES, RangeDecoder, ilog};
pub const SPREAD_NONE: usize = 0;
pub const SPREAD_AGGRESSIVE: usize = 3;
#[inline]
pub fn lcg_rand(seed: u32) -> u32 {
seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223)
}
/// Decodes a PVQ codeword of `k` pulses in `n` dimensions; returns the sum of squares.
fn decode_pulses(y: &mut [i32], n: usize, k: usize, dec: &mut RangeDecoder, v: &mut Vec<u64>) -> f32 {
// Table of V(nn, kk) for nn in 0..=n, kk in 0..=k (saturating).
let w = k + 1;
v.clear();
v.resize((n + 1) * w, 0);
v[0] = 1;
for nn in 1..=n {
v[nn * w] = 1;
for kk in 1..=k {
let a = v[(nn - 1) * w + kk];
let b = v[nn * w + kk - 1];
let c = v[(nn - 1) * w + kk - 1];
v[nn * w + kk] = a.saturating_add(b).saturating_add(c);
}
}
let total = v[n * w + k];
let ft = total.min(u32::MAX as u64) as u32;
let mut i = if ft >= 2 { dec.uint(ft) as u64 } else { 0 };
let mut kk = k;
let mut ryy = 0f32;
for j in 0..n {
let rem = n - j;
if kk == 0 {
y[j] = 0;
continue;
}
let vv = |nn: usize, kx: usize| v[nn * w + kx];
let mut p = (vv(rem - 1, kk) + vv(rem, kk)) / 2;
let neg = i >= p;
if neg {
i -= p;
}
let k0 = kk;
p -= vv(rem - 1, kk);
while p > i && kk > 0 {
kk -= 1;
p -= vv(rem - 1, kk);
}
let mag = (k0 - kk) as i32;
y[j] = if neg { -mag } else { mag };
i -= p.min(i);
ryy += (mag * mag) as f32;
}
ryy
}
fn exp_rotation1(x: &mut [f32], len: usize, stride: usize, c: f32, s: f32) {
let ms = -s;
if len <= stride {
return;
}
for i in 0..len - stride {
let x1 = x[i];
let x2 = x[i + stride];
x[i + stride] = c * x2 + s * x1;
x[i] = c * x1 + ms * x2;
}
if len >= 2 * stride + 1 {
for i in (0..=len - 2 * stride - 1).rev() {
let x1 = x[i];
let x2 = x[i + stride];
x[i + stride] = c * x2 + s * x1;
x[i] = c * x1 + ms * x2;
}
}
}
/// Inverse spreading rotation (decoder direction).
fn exp_rotation(x: &mut [f32], len: usize, stride: usize, k: usize, spread: usize) {
const SPREAD_FACTOR: [usize; 3] = [15, 10, 5];
if 2 * k >= len || spread == SPREAD_NONE {
return;
}
let factor = SPREAD_FACTOR[spread - 1];
let gain = len as f32 / (len + factor * k) as f32;
let theta = 0.5 * gain * gain;
let c = (0.5 * std::f32::consts::PI * theta).cos();
let s = (0.5 * std::f32::consts::PI * (1.0 - theta)).cos();
let mut stride2 = 0;
if len >= 8 * stride {
stride2 = 1;
while (stride2 * stride2 + stride2) * stride + (stride >> 2) < len {
stride2 += 1;
}
}
let sub = len / stride;
for i in 0..stride {
let xs = &mut x[i * sub..(i + 1) * sub];
if stride2 != 0 {
exp_rotation1(xs, sub, stride2, s, c);
}
exp_rotation1(xs, sub, 1, c, s);
}
}
fn extract_collapse_mask(iy: &[i32], n: usize, b: usize) -> u32 {
if b <= 1 {
return 1;
}
let n0 = n / b;
let mut mask = 0u32;
for i in 0..b {
if iy[i * n0..(i + 1) * n0].iter().any(|&v| v != 0) {
mask |= 1 << i;
}
}
mask
}
pub fn renormalise_vector(x: &mut [f32], gain: f32) {
let e: f32 = 1e-15 + x.iter().map(|v| v * v).sum::<f32>();
let g = gain / e.sqrt();
for v in x.iter_mut() {
*v *= g;
}
}
fn haar1(x: &mut [f32], n0: usize, stride: usize) {
let n0 = n0 >> 1;
const S: f32 = std::f32::consts::FRAC_1_SQRT_2;
for i in 0..stride {
for j in 0..n0 {
let a = x[stride * 2 * j + i] * S;
let b = x[stride * (2 * j + 1) + i] * S;
x[stride * 2 * j + i] = a + b;
x[stride * (2 * j + 1) + i] = a - b;
}
}
}
fn ordery(stride: usize) -> &'static [usize] {
&ORDERY[stride - 2..2 * stride - 2]
}
fn deinterleave_hadamard(x: &mut [f32], n0: usize, stride: usize, hadamard: bool, tmp: &mut Vec<f32>) {
let n = n0 * stride;
tmp.clear();
tmp.resize(n, 0.0);
if hadamard {
let ord = ordery(stride);
for i in 0..stride {
for j in 0..n0 {
tmp[ord[i] * n0 + j] = x[j * stride + i];
}
}
} else {
for i in 0..stride {
for j in 0..n0 {
tmp[i * n0 + j] = x[j * stride + i];
}
}
}
x[..n].copy_from_slice(tmp);
}
fn interleave_hadamard(x: &mut [f32], n0: usize, stride: usize, hadamard: bool, tmp: &mut Vec<f32>) {
let n = n0 * stride;
tmp.clear();
tmp.resize(n, 0.0);
if hadamard {
let ord = ordery(stride);
for i in 0..stride {
for j in 0..n0 {
tmp[j * stride + i] = x[ord[i] * n0 + j];
}
}
} else {
for i in 0..stride {
for j in 0..n0 {
tmp[j * stride + i] = x[i * n0 + j];
}
}
}
x[..n].copy_from_slice(tmp);
}
#[inline]
fn frac_mul16(a: i32, b: i32) -> i32 {
(16384 + (a as i16 as i32) * (b as i16 as i32)) >> 15
}
fn bitexact_cos(x: i32) -> i32 {
let tmp = (4096 + x * x) >> 13;
let x2 = tmp;
let x2 = (32767 - x2) + frac_mul16(x2, -7651 + frac_mul16(x2, 8277 + frac_mul16(-626, x2)));
1 + x2
}
fn bitexact_log2tan(isin: i32, icos: i32) -> i32 {
let lc = ilog(icos as u32);
let ls = ilog(isin as u32);
let icos = icos << (15 - lc);
let isin = isin << (15 - ls);
(ls - lc) * (1 << 11) + frac_mul16(isin, frac_mul16(isin, -2597) + 7932) - frac_mul16(icos, frac_mul16(icos, -2597) + 7932)
}
fn isqrt32(v: u32) -> u32 {
let mut r = (v as f64).sqrt() as u32;
while (r as u64) * (r as u64) > v as u64 {
r -= 1;
}
while ((r + 1) as u64) * ((r + 1) as u64) <= v as u64 {
r += 1;
}
r
}
fn compute_qn(n: i32, b: i32, offset: i32, pulse_cap: i32, stereo: bool) -> i32 {
const EXP2_TABLE8: [i32; 8] = [16384, 17866, 19483, 21247, 23170, 25267, 27554, 30048];
let mut n2 = 2 * n - 1;
if stereo && n == 2 {
n2 -= 1;
}
let mut qb = sdiv(b + n2 * offset, n2);
qb = qb.min(b - pulse_cap - (4 << BITRES));
qb = qb.min(8 << BITRES);
if qb < (1 << BITRES >> 1) {
1
} else {
let qn = EXP2_TABLE8[(qb & 0x7) as usize] >> (14 - (qb >> BITRES));
(qn + 1) >> 1 << 1
}
}
/// C-style truncating signed division.
#[inline]
fn sdiv(a: i32, b: i32) -> i32 {
a / b
}
pub struct BandCtx<'m, 'd, 'b> {
pub m: &'m Mode,
pub dec: &'d mut RangeDecoder<'b>,
pub i: usize,
pub intensity: usize,
pub spread: usize,
pub tf_change: i32,
pub remaining_bits: i32,
pub seed: u32,
pub disable_inv: bool,
pub avoid_split_noise: bool,
pub pvq_v: Vec<u64>,
pub iy: Vec<i32>,
pub tmp: Vec<f32>,
}
struct Split {
inv: bool,
imid: i32,
iside: i32,
delta: i32,
itheta: i32,
qalloc: i32,
}
impl BandCtx<'_, '_, '_> {
fn compute_theta(&mut self, n: i32, b: &mut i32, bb: usize, b0: usize, lm: i32, stereo: bool, fill: &mut u32) -> Split {
let i = self.i;
let pulse_cap = self.m.log_n[i] + lm * (1 << BITRES);
let offset = (pulse_cap >> 1) - if stereo && n == 2 { QTHETA_OFFSET_TWOPHASE } else { QTHETA_OFFSET };
let mut qn = compute_qn(n, *b, offset, pulse_cap, stereo);
if stereo && i >= self.intensity {
qn = 1;
}
let tell = self.dec.tell_frac() as i32;
let mut itheta = 0i32;
let mut inv = false;
if qn != 1 {
if stereo && n > 2 {
let p0 = 3u32;
let x0 = (qn / 2) as u32;
let ft = p0 * (x0 + 1) + x0;
let fs = self.dec.decode(ft);
let x = if fs < (x0 + 1) * p0 { fs / p0 } else { x0 + 1 + (fs - (x0 + 1) * p0) };
let (fl, fh) = if x <= x0 { (p0 * x, p0 * (x + 1)) } else { ((x - 1 - x0) + (x0 + 1) * p0, (x - x0) + (x0 + 1) * p0) };
self.dec.update(fl, fh, ft);
itheta = x as i32;
} else if b0 > 1 || stereo {
itheta = self.dec.uint(qn as u32 + 1) as i32;
} else {
let half = (qn >> 1) as u32;
let ft = (half + 1) * (half + 1);
let fm = self.dec.decode(ft);
let (fl, fs);
if fm < (half * (half + 1)) >> 1 {
let it = (isqrt32(8 * fm + 1) - 1) >> 1;
fs = it + 1;
fl = it * (it + 1) >> 1;
itheta = it as i32;
} else {
let q = qn as u32;
let it = (2 * (q + 1) - isqrt32(8 * (ft - fm - 1) + 1)) >> 1;
fs = q + 1 - it;
fl = ft - ((q + 1 - it) * (q + 2 - it) >> 1);
itheta = it as i32;
}
self.dec.update(fl, fl + fs, ft);
}
itheta = ((itheta as u32 * 16384) / qn as u32) as i32;
} else if stereo {
if *b > 2 << BITRES && self.remaining_bits > 2 << BITRES {
inv = self.dec.bit_logp(2);
}
if self.disable_inv {
inv = false;
}
itheta = 0;
}
let qalloc = self.dec.tell_frac() as i32 - tell;
*b -= qalloc;
let (imid, iside, delta);
if itheta == 0 {
imid = 32767;
iside = 0;
*fill &= (1 << bb) - 1;
delta = -16384;
} else if itheta == 16384 {
imid = 0;
iside = 32767;
*fill &= ((1 << bb) - 1) << bb;
delta = 16384;
} else {
imid = bitexact_cos(itheta);
iside = bitexact_cos(16384 - itheta);
delta = frac_mul16((n - 1) << 7, bitexact_log2tan(iside, imid));
}
Split { inv, imid, iside, delta, itheta, qalloc }
}
fn quant_band_n1(&mut self, x: &mut [f32], y: Option<&mut [f32]>, lowband_out: Option<&mut [f32]>) -> u32 {
let dec_sign = |ctx: &mut Self| -> f32 {
let mut sign = 0;
if ctx.remaining_bits >= 1 << BITRES {
sign = ctx.dec.bits(1);
ctx.remaining_bits -= 1 << BITRES;
}
if sign != 0 { -1.0 } else { 1.0 }
};
x[0] = dec_sign(self);
if let Some(y) = y {
y[0] = dec_sign(self);
}
if let Some(lo) = lowband_out {
lo[0] = x[0];
}
1
}
#[allow(clippy::too_many_arguments)]
fn quant_partition(
&mut self,
x: &mut [f32],
n: usize,
mut b: i32,
mut bb: usize,
lowband: Option<&[f32]>,
mut lm: i32,
gain: f32,
mut fill: u32,
) -> u32 {
let b0 = bb;
let i = self.i;
let cache = self.m.cache(lm, i);
let cache_max = cache[cache[0] as usize] as i32;
if lm != -1 && b > cache_max + 12 && n > 2 {
let n = n >> 1;
let (xa, ya) = x.split_at_mut(n);
lm -= 1;
if bb == 1 {
fill = (fill & 1) | (fill << 1);
}
bb = (bb + 1) >> 1;
let sp = self.compute_theta(n as i32, &mut b, bb, b0, lm, false, &mut fill);
let mid = sp.imid as f32 / 32768.0;
let side = sp.iside as f32 / 32768.0;
let mut delta = sp.delta;
if b0 > 1 && (sp.itheta & 0x3fff) != 0 {
if sp.itheta > 8192 {
delta -= delta >> (4 - lm);
} else {
delta = 0.min(delta + ((n as i32) << BITRES >> (5 - lm)));
}
}
let mut mbits = 0.max(b.min((b - delta) / 2));
let mut sbits = b - mbits;
self.remaining_bits -= sp.qalloc;
let next_lowband2 = lowband.map(|l| &l[n..]);
let mut rebalance = self.remaining_bits;
let mut cm;
if mbits >= sbits {
cm = self.quant_partition(xa, n, mbits, bb, lowband, lm, gain * mid, fill);
rebalance = mbits - (rebalance - self.remaining_bits);
if rebalance > 3 << BITRES && sp.itheta != 0 {
sbits += rebalance - (3 << BITRES);
}
cm |= self.quant_partition(ya, n, sbits, bb, next_lowband2, lm, gain * side, fill >> bb) << (b0 >> 1);
} else {
cm = self.quant_partition(ya, n, sbits, bb, next_lowband2, lm, gain * side, fill >> bb) << (b0 >> 1);
rebalance = sbits - (rebalance - self.remaining_bits);
if rebalance > 3 << BITRES && sp.itheta != 16384 {
mbits += rebalance - (3 << BITRES);
}
cm |= self.quant_partition(xa, n, mbits, bb, lowband, lm, gain * mid, fill);
}
return cm;
}
// No split.
let m = self.m;
let mut q = m.bits2pulses(i, lm, b);
let mut curr_bits = m.pulses2bits(i, lm, q);
self.remaining_bits -= curr_bits;
while self.remaining_bits < 0 && q > 0 {
self.remaining_bits += curr_bits;
q -= 1;
curr_bits = m.pulses2bits(i, lm, q);
self.remaining_bits -= curr_bits;
}
if q != 0 {
let k = get_pulses(q) as usize;
let mut iy = std::mem::take(&mut self.iy);
iy.clear();
iy.resize(n, 0);
let ryy = decode_pulses(&mut iy, n, k, self.dec, &mut self.pvq_v);
let g = gain / ryy.sqrt();
for (xv, &yv) in x[..n].iter_mut().zip(iy.iter()) {
*xv = g * yv as f32;
}
exp_rotation(&mut x[..n], n, bb, k, self.spread);
let cm = extract_collapse_mask(&iy, n, bb);
self.iy = iy;
cm
} else {
let cm_mask = (1u32 << bb) - 1;
fill &= cm_mask;
if fill == 0 {
x[..n].fill(0.0);
0
} else {
let cm = match lowband {
None => {
for v in x[..n].iter_mut() {
self.seed = lcg_rand(self.seed);
*v = ((self.seed as i32) >> 20) as f32;
}
cm_mask
}
Some(lb) => {
for j in 0..n {
self.seed = lcg_rand(self.seed);
let tmp = if self.seed & 0x8000 != 0 { 1.0 / 256.0 } else { -1.0 / 256.0 };
x[j] = lb[j] + tmp;
}
fill
}
};
renormalise_vector(&mut x[..n], gain);
cm
}
}
}
#[allow(clippy::too_many_arguments)]
pub fn quant_band(
&mut self,
x: &mut [f32],
n: usize,
b: i32,
mut bb: usize,
mut lowband: Option<&mut [f32]>,
lm: i32,
lowband_out: Option<&mut [f32]>,
gain: f32,
mut fill: u32,
) -> u32 {
let n0 = n;
let mut n_b = n;
let b0_orig = bb;
let mut time_divide = 0;
let mut recombine = 0;
let long_blocks = b0_orig == 1;
let mut tf_change = self.tf_change;
n_b /= bb;
if n == 1 {
return self.quant_band_n1(x, None, lowband_out);
}
if tf_change > 0 {
recombine = tf_change;
}
const BIT_INTERLEAVE: [u32; 16] = [0, 1, 1, 1, 2, 3, 3, 3, 2, 3, 3, 3, 2, 3, 3, 3];
for k in 0..recombine {
if let Some(lb) = lowband.as_deref_mut() {
haar1(lb, n >> k, 1 << k);
}
fill = BIT_INTERLEAVE[(fill & 0xF) as usize] | (BIT_INTERLEAVE[(fill >> 4) as usize] << 2);
}
bb >>= recombine;
n_b <<= recombine;
while (n_b & 1) == 0 && tf_change < 0 {
if let Some(lb) = lowband.as_deref_mut() {
haar1(lb, n_b, bb);
}
fill |= fill << bb;
bb <<= 1;
n_b >>= 1;
time_divide += 1;
tf_change += 1;
}
let b0 = bb;
let n_b0 = n_b;
let mut tmp = std::mem::take(&mut self.tmp);
if b0 > 1
&& let Some(lb) = lowband.as_deref_mut()
{
deinterleave_hadamard(lb, n_b >> recombine, b0 << recombine, long_blocks, &mut tmp);
}
let mut cm = self.quant_partition(x, n, b, bb, lowband.as_deref(), lm, gain, fill);
if b0 > 1 {
interleave_hadamard(x, n_b >> recombine, b0 << recombine, long_blocks, &mut tmp);
}
self.tmp = tmp;
let mut n_b = n_b0;
bb = b0;
for _ in 0..time_divide {
bb >>= 1;
n_b <<= 1;
cm |= cm >> bb;
haar1(x, n_b, bb);
}
const BIT_DEINTERLEAVE: [u32; 16] = [0x00, 0x03, 0x0C, 0x0F, 0x30, 0x33, 0x3C, 0x3F, 0xC0, 0xC3, 0xCC, 0xCF, 0xF0, 0xF3, 0xFC, 0xFF];
for k in 0..recombine {
cm = BIT_DEINTERLEAVE[(cm & 0xF) as usize];
haar1(x, n0 >> k, 1 << k);
}
bb <<= recombine;
if let Some(lo) = lowband_out {
let s = (n0 as f32).sqrt();
for j in 0..n0 {
lo[j] = s * x[j];
}
}
cm & ((1u32 << bb) - 1)
}
#[allow(clippy::too_many_arguments)]
pub fn quant_band_stereo(
&mut self,
x: &mut [f32],
y: &mut [f32],
n: usize,
mut b: i32,
bb: usize,
lowband: Option<&mut [f32]>,
lm: i32,
lowband_out: Option<&mut [f32]>,
mut fill: u32,
) -> u32 {
let orig_fill = fill;
if n == 1 {
return self.quant_band_n1(x, Some(y), lowband_out);
}
let sp = self.compute_theta(n as i32, &mut b, bb, bb, lm, true, &mut fill);
let mid = sp.imid as f32 / 32768.0;
let side = sp.iside as f32 / 32768.0;
let mut cm;
if n == 2 {
let mut mbits = b;
let mut sbits = 0;
if sp.itheta != 0 && sp.itheta != 16384 {
sbits = 1 << BITRES;
}
mbits -= sbits;
let c = sp.itheta > 8192;
self.remaining_bits -= sp.qalloc + sbits;
let mut sign = 0;
if sbits != 0 {
sign = self.dec.bits(1) as i32;
}
let sign = (1 - 2 * sign) as f32;
if c {
cm = self.quant_band(y, n, mbits, bb, lowband, lm, lowband_out, 1.0, orig_fill);
x[0] = -sign * y[1];
x[1] = sign * y[0];
} else {
cm = self.quant_band(x, n, mbits, bb, lowband, lm, lowband_out, 1.0, orig_fill);
y[0] = -sign * x[1];
y[1] = sign * x[0];
}
x[0] *= mid;
x[1] *= mid;
y[0] *= side;
y[1] *= side;
let t = x[0];
x[0] = t - y[0];
y[0] += t;
let t = x[1];
x[1] = t - y[1];
y[1] += t;
} else {
let mut mbits = 0.max(b.min((b - sp.delta) / 2));
let mut sbits = b - mbits;
self.remaining_bits -= sp.qalloc;
let mut rebalance = self.remaining_bits;
if mbits >= sbits {
cm = self.quant_band(x, n, mbits, bb, lowband, lm, lowband_out, 1.0, fill);
rebalance = mbits - (rebalance - self.remaining_bits);
if rebalance > 3 << BITRES && sp.itheta != 0 {
sbits += rebalance - (3 << BITRES);
}
cm |= self.quant_band(y, n, sbits, bb, None, lm, None, side, fill >> bb);
} else {
cm = self.quant_band(y, n, sbits, bb, None, lm, None, side, fill >> bb);
rebalance = sbits - (rebalance - self.remaining_bits);
if rebalance > 3 << BITRES && sp.itheta != 16384 {
mbits += rebalance - (3 << BITRES);
}
cm |= self.quant_band(x, n, mbits, bb, lowband, lm, lowband_out, 1.0, fill);
}
}
if n != 2 {
stereo_merge(&mut x[..n], &mut y[..n], mid);
}
if sp.inv {
for v in y[..n].iter_mut() {
*v = -*v;
}
}
cm
}
}
fn stereo_merge(x: &mut [f32], y: &mut [f32], mid: f32) {
let mut xp = 0f32;
let mut side = 0f32;
for (a, b) in x.iter().zip(y.iter()) {
xp += a * b;
side += b * b;
}
xp *= mid;
let mid2 = mid;
let el = mid2 * mid2 + side - 2.0 * xp;
let er = mid2 * mid2 + side + 2.0 * xp;
if er < 6e-4 || el < 6e-4 {
y.copy_from_slice(x);
return;
}
let lgain = 1.0 / el.sqrt();
let rgain = 1.0 / er.sqrt();
for (a, b) in x.iter_mut().zip(y.iter_mut()) {
let l = mid * *a;
let r = *b;
*a = lgain * (l - r);
*b = rgain * (l + r);
}
}
/// Decodes all band shapes (`quant_all_bands`, decoder side).
#[allow(clippy::too_many_arguments)]
pub fn quant_all_bands(
m: &Mode,
start: usize,
end: usize,
x_buf: &mut [f32],
y_buf: Option<&mut [f32]>,
collapse_masks: &mut [u32],
pulses: &[i32; NB_EBANDS],
short_blocks: bool,
spread: usize,
mut dual_stereo: bool,
intensity: usize,
tf_res: &[i32; NB_EBANDS],
total_bits: i32,
mut balance: i32,
dec: &mut RangeDecoder,
lm: usize,
coded_bands: usize,
seed: &mut u32,
disable_inv: bool,
) {
let mm = 1usize << lm;
let bb = if short_blocks { mm } else { 1 };
let norm_offset = mm * EBANDS[start] as usize;
let norm_len = mm * EBANDS[NB_EBANDS - 1] as usize - norm_offset;
let c = if y_buf.is_some() { 2 } else { 1 };
let mut norm = vec![0f32; norm_len];
let mut norm2 = vec![0f32; if c == 2 { norm_len } else { 0 }];
let mut scratch = vec![0f32; mm * (EBANDS[NB_EBANDS] - EBANDS[NB_EBANDS - 1]) as usize];
let mut scratch2 = scratch.clone();
let mut y_buf = y_buf;
let mut lowband_offset = 0usize;
let mut update_lowband = true;
let mut ctx = BandCtx {
m,
dec,
i: 0,
intensity,
spread,
tf_change: 0,
remaining_bits: 0,
seed: *seed,
disable_inv,
avoid_split_noise: bb > 1,
pvq_v: Vec::new(),
iy: Vec::new(),
tmp: Vec::new(),
};
for i in start..end {
ctx.i = i;
let last = i == end - 1;
let xo = mm * EBANDS[i] as usize;
let n = mm * EBANDS[i + 1] as usize - xo;
let tell = ctx.dec.tell_frac() as i32;
if i != start {
balance -= tell;
}
let remaining_bits = total_bits - tell - 1;
ctx.remaining_bits = remaining_bits;
let b = if i < coded_bands {
let curr_balance = sdiv(balance, 3.min(coded_bands as i32 - i as i32));
0.max(16383.min((remaining_bits + 1).min(pulses[i] + curr_balance)))
} else {
0
};
if (mm * EBANDS[i] as usize >= n + norm_offset || i == start + 1) && (update_lowband || lowband_offset == 0) {
lowband_offset = i;
}
if i == start + 1 {
let n1 = mm * (EBANDS[start + 1] - EBANDS[start]) as usize;
let n2 = mm * (EBANDS[start + 2] - EBANDS[start + 1]) as usize;
if n2 > n1 {
norm.copy_within(2 * n1 - n2..n1, n1);
if c == 2 {
norm2.copy_within(2 * n1 - n2..n1, n1);
}
}
}
let tf_change = tf_res[i];
ctx.tf_change = tf_change;
let mut effective_lowband: Option<usize> = None;
let (mut x_cm, mut y_cm);
if lowband_offset != 0 && (spread != SPREAD_AGGRESSIVE || bb > 1 || tf_change < 0) {
let eff = (mm * EBANDS[lowband_offset] as usize).saturating_sub(norm_offset + n);
effective_lowband = Some(eff);
let mut fold_start = lowband_offset;
loop {
fold_start -= 1;
if mm * (EBANDS[fold_start] as usize) <= eff + norm_offset {
break;
}
}
let mut fold_end = lowband_offset - 1;
loop {
fold_end += 1;
if !(fold_end < i && mm * (EBANDS[fold_end] as usize) < eff + norm_offset + n) {
break;
}
}
x_cm = 0;
y_cm = 0;
let mut fi = fold_start;
loop {
x_cm |= collapse_masks[fi * c];
y_cm |= collapse_masks[fi * c + c - 1];
fi += 1;
if fi >= fold_end {
break;
}
}
} else {
x_cm = (1u32 << bb) - 1;
y_cm = x_cm;
}
if dual_stereo && i == intensity {
dual_stereo = false;
for j in 0..(mm * EBANDS[i] as usize - norm_offset) {
norm[j] = 0.5 * (norm[j] + norm2[j]);
}
}
let out_off = xo - norm_offset;
let x = &mut x_buf[xo..xo + n];
if dual_stereo && let Some(y_buf) = y_buf.as_deref_mut() {
let y = &mut y_buf[xo..xo + n];
let lb = effective_lowband.map(|e| {
scratch[..n].copy_from_slice(&norm[e..e + n]);
&mut scratch[..n]
});
let lo = if last { None } else { Some(&mut norm[out_off..out_off + n]) };
x_cm = ctx.quant_band(x, n, b / 2, bb, lb, lm as i32, lo, 1.0, x_cm);
let lb2 = effective_lowband.map(|e| {
scratch2[..n].copy_from_slice(&norm2[e..e + n]);
&mut scratch2[..n]
});
let lo2 = if last { None } else { Some(&mut norm2[out_off..out_off + n]) };
y_cm = ctx.quant_band(y, n, b / 2, bb, lb2, lm as i32, lo2, 1.0, y_cm);
} else {
let lb = effective_lowband.map(|e| {
scratch[..n].copy_from_slice(&norm[e..e + n]);
&mut scratch[..n]
});
let lo = if last { None } else { Some(&mut norm[out_off..out_off + n]) };
if let Some(yb) = y_buf.as_deref_mut() {
let y = &mut yb[xo..xo + n];
x_cm = ctx.quant_band_stereo(x, y, n, b, bb, lb, lm as i32, lo, x_cm | y_cm);
} else {
x_cm = ctx.quant_band(x, n, b, bb, lb, lm as i32, lo, 1.0, x_cm | y_cm);
}
y_cm = x_cm;
}
collapse_masks[i * c] = x_cm;
collapse_masks[i * c + c - 1] = y_cm;
balance += pulses[i] + tell;
update_lowband = b > (n as i32) << BITRES;
ctx.avoid_split_noise = false;
}
*seed = ctx.seed;
}
/// Anti-collapse noise injection (RFC 6716 §4.3.5).
#[allow(clippy::too_many_arguments)]
pub fn anti_collapse(
x_buf: &mut [f32],
collapse_masks: &[u32],
lm: usize,
c: usize,
size: usize,
start: usize,
end: usize,
log_e: &[f32],
prev1: &[f32],
prev2: &[f32],
pulses: &[i32; NB_EBANDS],
mut seed: u32,
) {
for i in start..end {
let n0 = (EBANDS[i + 1] - EBANDS[i]) as usize;
let depth = ((1 + pulses[i]) as u32 / n0 as u32) >> lm;
let thresh = 0.5 * (-0.125 * depth as f32).exp2();
let sqrt_1 = 1.0 / ((n0 << lm) as f32).sqrt();
for ch in 0..c {
let mut p1 = prev1[ch * NB_EBANDS + i];
let mut p2 = prev2[ch * NB_EBANDS + i];
if c == 1 {
p1 = p1.max(prev1[NB_EBANDS + i]);
p2 = p2.max(prev2[NB_EBANDS + i]);
}
let ediff = (log_e[ch * NB_EBANDS + i] - p1.min(p2)).max(0.0);
let mut r = 2.0 * (-ediff).exp2();
if lm == 3 {
r *= std::f32::consts::SQRT_2;
}
r = r.min(thresh);
r *= sqrt_1;
let off = ch * size + ((EBANDS[i] as usize) << lm);
let xb = &mut x_buf[off..off + (n0 << lm)];
let mut renorm = false;
for k in 0..(1usize << lm) {
if collapse_masks[i * c + ch] & (1 << k) == 0 {
for j in 0..n0 {
seed = lcg_rand(seed);
xb[(j << lm) + k] = if seed & 0x8000 != 0 { r } else { -r };
}
renorm = true;
}
}
if renorm {
renormalise_vector(xb, 1.0);
}
}
}
}
/// Multiplies the normalised shapes by the band energies (`denormalise_bands`).
#[allow(clippy::too_many_arguments)]
pub fn denormalise_bands(
x: &[f32],
freq: &mut [f32],
band_log_e: &[f32],
mut start: usize,
mut end: usize,
mm: usize,
downsample: usize,
silence: bool,
) {
let n = mm * SHORT_MDCT;
let mut bound = mm * EBANDS[end] as usize;
if downsample != 1 {
bound = bound.min(n / downsample);
}
if silence {
bound = 0;
start = 0;
end = 0;
}
let s0 = mm * EBANDS[start] as usize;
freq[..s0].fill(0.0);
for i in start..end {
let lo = mm * EBANDS[i] as usize;
let hi = mm * EBANDS[i + 1] as usize;
let lg = band_log_e[i] + E_MEANS[i];
let g = lg.min(32.0).exp2();
for j in lo..hi {
freq[j] = x[j] * g;
}
}
let from = bound;
if from < n {
freq[from..n].fill(0.0);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn bitexact_trig_endpoints() {
assert_eq!(bitexact_cos(0), 32768);
let mid = bitexact_cos(8192);
assert!((mid - 23170).abs() < 20, "{mid}");
assert_eq!(bitexact_log2tan(16384, 16384), 0);
}
#[test]
fn isqrt_exact() {
for v in [0u32, 1, 2, 3, 4, 15, 16, 17, 99, 100, 1 << 20, u32::MAX] {
let r = isqrt32(v) as u64;
assert!(r * r <= v as u64 && (r + 1) * (r + 1) > v as u64);
}
}
#[test]
fn pvq_roundtrip_all_codewords_small() {
use crate::opus::decode::range::enc::RangeEncoder;
// Enumerate every index for small N,K, decode, and check uniqueness + sum |y| == K.
for n in 1..6usize {
for k in 1..5usize {
let total = super::super::rate::pvq_v(n, k) as u32;
let mut seen = std::collections::HashSet::new();
for idx in 0..total {
let mut e = RangeEncoder::new(16);
e.uint(idx, total);
let buf = e.done();
let mut d = RangeDecoder::new(&buf);
let mut y = vec![0i32; n];
let mut v = Vec::new();
let ryy = decode_pulses(&mut y, n, k, &mut d, &mut v);
assert_eq!(y.iter().map(|v| v.unsigned_abs() as usize).sum::<usize>(), k);
assert_eq!(ryy as i32, y.iter().map(|v| v * v).sum::<i32>());
assert!(seen.insert(y.clone()), "duplicate codeword n={n} k={k} idx={idx}");
}
}
}
}
#[test]
fn haar_is_involution() {
let mut x: Vec<f32> = (0..16).map(|i| i as f32).collect();
let orig = x.clone();
haar1(&mut x, 16, 1);
haar1(&mut x, 16, 1);
for (a, b) in x.iter().zip(&orig) {
assert!((a - b).abs() < 1e-4);
}
let mut tmp = Vec::new();
for &h in &[false, true] {
let mut y = orig.clone();
deinterleave_hadamard(&mut y, 4, 4, h, &mut tmp);
interleave_hadamard(&mut y, 4, 4, h, &mut tmp);
assert_eq!(y, orig);
}
}
}
crates/audio/src/opus/decode/celt/energy.rs
+171
View File
@@ -0,0 +1,171 @@
//! Band energy decoding: Laplace-coded coarse energy, fine energy and the final refinement
//! (RFC 6716 §4.3.2).
use super::rate::MAX_FINE_BITS;
use super::tables::{BETA_COEF, BETA_INTRA, E_PROB_MODEL, NB_EBANDS, PRED_COEF, SMALL_ENERGY_ICDF};
use crate::opus::decode::range::RangeDecoder;
const LAPLACE_MINP: u32 = 1;
const LAPLACE_NMIN: u32 = 16;
fn laplace_get_freq1(fs0: u32, decay: u32) -> u32 {
let ft = 32768 - LAPLACE_MINP * (2 * LAPLACE_NMIN) - fs0;
(ft * (16384 - decay)) >> 15
}
/// Decodes a Laplace-distributed integer (`fs` = probability of 0 in Q15, `decay` in Q14).
pub fn laplace_decode(dec: &mut RangeDecoder, mut fs: u32, decay: u32) -> i32 {
let mut val = 0i32;
let fm = dec.decode_bin(15);
let mut fl = 0u32;
if fm >= fs {
val += 1;
fl = fs;
fs = laplace_get_freq1(fs, decay) + LAPLACE_MINP;
while fs > LAPLACE_MINP && fm >= fl + 2 * fs {
fs *= 2;
fl += fs;
fs = (((fs - 2 * LAPLACE_MINP) * decay) >> 15) + LAPLACE_MINP;
val += 1;
}
if fs <= LAPLACE_MINP {
let di = (fm - fl) >> 1;
val += di as i32;
fl += 2 * di * LAPLACE_MINP;
}
if fm < fl + fs {
val = -val;
} else {
fl += fs;
}
}
dec.update_bin(fl, (fl + fs).min(32768), 15);
val
}
/// `unquant_coarse_energy`.
#[allow(clippy::too_many_arguments)]
pub fn unquant_coarse_energy(start: usize, end: usize, old_e: &mut [f32], intra: bool, dec: &mut RangeDecoder, c: usize, lm: usize) {
let prob = &E_PROB_MODEL[lm][intra as usize];
let (coef, beta) = if intra { (0.0, BETA_INTRA) } else { (PRED_COEF[lm], BETA_COEF[lm]) };
let budget = dec.storage() as i32 * 8;
let mut prev = [0f32; 2];
for i in start..end {
for ch in 0..c {
let tell = dec.tell();
let qi = if budget - tell >= 15 {
let pi = 2 * i.min(20);
laplace_decode(dec, (prob[pi] as u32) << 7, (prob[pi + 1] as u32) << 6)
} else if budget - tell >= 2 {
let q = dec.icdf(&SMALL_ENERGY_ICDF, 2) as i32;
(q >> 1) ^ -(q & 1)
} else if budget - tell >= 1 {
-(dec.bit_logp(1) as i32)
} else {
-1
};
let q = qi as f32;
let idx = i + ch * NB_EBANDS;
old_e[idx] = old_e[idx].max(-9.0);
let tmp = coef * old_e[idx] + prev[ch] + q;
old_e[idx] = tmp;
prev[ch] = prev[ch] + q - beta * q;
}
}
}
pub fn unquant_fine_energy(start: usize, end: usize, old_e: &mut [f32], fine_quant: &[i32; NB_EBANDS], dec: &mut RangeDecoder, c: usize) {
for i in start..end {
let fq = fine_quant[i];
if fq <= 0 {
continue;
}
for ch in 0..c {
let q2 = dec.bits(fq as u32);
let offset = (q2 as f32 + 0.5) * (1 << (14 - fq)) as f32 / 16384.0 - 0.5;
old_e[i + ch * NB_EBANDS] += offset;
}
}
}
#[allow(clippy::too_many_arguments)]
pub fn unquant_energy_finalise(
start: usize,
end: usize,
old_e: &mut [f32],
fine_quant: &[i32; NB_EBANDS],
fine_priority: &[i32; NB_EBANDS],
mut bits_left: i32,
dec: &mut RangeDecoder,
c: usize,
) {
for prio in 0..2 {
let mut i = start;
while i < end && bits_left >= c as i32 {
if fine_quant[i] >= MAX_FINE_BITS || fine_priority[i] != prio {
i += 1;
continue;
}
for ch in 0..c {
let q2 = dec.bits(1);
let offset = (q2 as f32 - 0.5) * (1 << (14 - fine_quant[i] - 1)) as f32 / 16384.0;
old_e[i + ch * NB_EBANDS] += offset;
bits_left -= 1;
}
i += 1;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::opus::decode::range::enc::RangeEncoder;
fn laplace_encode(e: &mut RangeEncoder, value: i32, mut fs: u32, decay: u32) {
let mut fl = 0u32;
let mut val = value;
if val != 0 {
let s: i32 = if val < 0 { -1 } else { 0 };
val = val.abs();
fl = fs;
fs = laplace_get_freq1(fs, decay);
let mut i = 1;
while fs > 0 && i < val {
fs *= 2;
fl += fs + 2 * LAPLACE_MINP;
fs = (fs * decay) >> 15;
i += 1;
}
if fs == 0 {
let ndi_max = (32768 - fl + LAPLACE_MINP - 1) as i32;
let ndi_max = (ndi_max - s) >> 1;
let di = (val - i).min(ndi_max - 1);
fl = (fl as i32 + (2 * di + 1 + s) * LAPLACE_MINP as i32) as u32;
fs = LAPLACE_MINP.min(32768 - fl);
} else {
fs += LAPLACE_MINP;
if s == 0 {
fl += fs;
}
}
}
e.encode(fl, fl + fs, 32768);
}
#[test]
fn laplace_roundtrip() {
for &(fs, decay) in &[(72u32 << 7, 127u32 << 6), (24 << 7, 179 << 6), (177 << 7, 11 << 6)] {
let vals: Vec<i32> = (-12..=12).collect();
let mut e = RangeEncoder::new(400);
for &v in &vals {
laplace_encode(&mut e, v, fs, decay);
}
let buf = e.done();
let mut d = RangeDecoder::new(&buf);
for &v in &vals {
assert_eq!(laplace_decode(&mut d, fs, decay), v, "fs={fs} v={v}");
}
}
}
}
crates/audio/src/opus/decode/celt/mdct.rs
+205
View File
@@ -0,0 +1,205 @@
//! Inverse MDCT with the low-overlap window and in-place TDAC (RFC 6716 §4.3.7).
//!
//! The IMDCT is computed through an N/4-point complex FFT (pre-rotation, FFT, post-rotation),
//! producing the N/2 "folded" time samples; the window overlap region is then unfolded against the
//! folded tail of the previous block, which the caller keeps in place in the output buffer.
#[derive(Clone, Copy, Default, Debug)]
pub struct Cpx {
pub re: f32,
pub im: f32,
}
impl Cpx {
#[inline]
fn mul(self, o: Cpx) -> Cpx {
Cpx { re: self.re * o.re - self.im * o.im, im: self.re * o.im + self.im * o.re }
}
#[inline]
fn add(self, o: Cpx) -> Cpx {
Cpx { re: self.re + o.re, im: self.im + o.im }
}
}
/// Mixed-radix (2, 3, 4, 5) forward complex FFT, unscaled.
pub struct Fft {
n: usize,
factors: Vec<usize>,
twiddle: Vec<Cpx>,
}
impl Fft {
pub fn new(n: usize) -> Fft {
let mut factors = Vec::new();
let mut m = n;
for p in [4usize, 2, 3, 5] {
while m.is_multiple_of(p) {
factors.push(p);
m /= p;
}
}
assert_eq!(m, 1, "unsupported FFT size {n}");
let twiddle = (0..n)
.map(|k| {
let ph = -2.0 * std::f64::consts::PI * k as f64 / n as f64;
Cpx { re: ph.cos() as f32, im: ph.sin() as f32 }
})
.collect();
Fft { n, factors, twiddle }
}
pub fn process(&self, data: &mut [Cpx], scratch: &mut Vec<Cpx>) {
scratch.clear();
scratch.extend_from_slice(&data[..self.n]);
self.rec(&mut data[..self.n], scratch, 1, self.n, 0, 1);
}
fn rec(&self, out: &mut [Cpx], inp: &[Cpx], in_stride: usize, n: usize, fi: usize, tw_stride: usize) {
let p = self.factors[fi];
let m = n / p;
if m == 1 {
for (i, o) in out.iter_mut().enumerate().take(p) {
*o = inp[i * in_stride];
}
} else {
for q in 0..p {
self.rec(&mut out[q * m..(q + 1) * m], &inp[q * in_stride..], in_stride * p, m, fi + 1, tw_stride * p);
}
}
let big = self.n;
let mut t = [Cpx::default(); 5];
for k in 0..m {
for q in 0..p {
let w = self.twiddle[(q * k * tw_stride) % big];
t[q] = out[q * m + k].mul(w);
}
for q2 in 0..p {
let mut acc = Cpx::default();
for (q, tq) in t.iter().enumerate().take(p) {
let w = self.twiddle[((q * q2) % p) * (big / p)];
acc = acc.add(tq.mul(w));
}
out[q2 * m + k] = acc;
}
}
}
}
pub struct Imdct {
/// Full MDCT length (2x the number of coefficients).
n: usize,
trig: Vec<f32>,
fft: Fft,
}
impl Imdct {
pub fn new(n: usize) -> Imdct {
let trig = (0..n / 2).map(|i| (2.0 * std::f64::consts::PI * (i as f64 + 0.125) / n as f64).cos() as f32).collect();
Imdct { n, trig, fft: Fft::new(n / 4) }
}
/// Inverse transform of `n/2` coefficients read from `input` at `stride`.
///
/// Writes the folded block to `out[overlap/2 .. overlap/2 + n/2]` and unfolds the first
/// `overlap` samples against the previous block's folded tail already stored in
/// `out[0 .. overlap/2]`.
pub fn backward(&self, input: &[f32], stride: usize, out: &mut [f32], window: &[f32], overlap: usize, work: &mut ImdctScratch) {
let n2 = self.n >> 1;
let n4 = self.n >> 2;
let t = &self.trig;
let z = &mut work.z;
z.clear();
for i in 0..n4 {
let x1 = input[2 * i * stride];
let x2 = input[stride * (n2 - 1 - 2 * i)];
let yr = x2 * t[i] + x1 * t[n4 + i];
let yi = x1 * t[i] - x2 * t[n4 + i];
z.push(Cpx { re: yi, im: yr });
}
self.fft.process(z, &mut work.scratch);
let base = overlap >> 1;
let y = &mut out[base..base + n2];
for i in 0..n4.div_ceil(2) {
let front = z[i];
let back = z[n4 - 1 - i];
let (re, im) = (front.im, front.re);
let (t0, t1) = (t[i], t[n4 + i]);
let yr0 = re * t0 + im * t1;
let yi0 = re * t1 - im * t0;
let (re, im) = (back.im, back.re);
let (t0, t1) = (t[n4 - i - 1], t[n2 - i - 1]);
let yr1 = re * t0 + im * t1;
let yi1 = re * t1 - im * t0;
y[2 * i] = yr0;
y[n2 - 1 - 2 * i] = yi0;
y[n2 - 2 - 2 * i] = yr1;
y[2 * i + 1] = yi1;
}
// Mirror on both sides for TDAC.
for i in 0..overlap / 2 {
let x1 = out[overlap - 1 - i];
let x2 = out[i];
let w1 = window[i];
let w2 = window[overlap - 1 - i];
out[i] = x2 * w2 - x1 * w1;
out[overlap - 1 - i] = x2 * w1 + x1 * w2;
}
}
}
#[derive(Default)]
pub struct ImdctScratch {
z: Vec<Cpx>,
scratch: Vec<Cpx>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fft_matches_dft() {
for &n in &[60usize, 120, 240, 480] {
let fft = Fft::new(n);
let x: Vec<Cpx> = (0..n).map(|i| Cpx { re: ((i * 7) % 13) as f32 - 6.0, im: ((i * 3) % 5) as f32 - 2.0 }).collect();
let mut y = x.clone();
fft.process(&mut y, &mut Vec::new());
for k in [0, 1, 7, n / 2, n - 1] {
let mut acc = (0f64, 0f64);
for (i, v) in x.iter().enumerate() {
let ph = -2.0 * std::f64::consts::PI * (i * k) as f64 / n as f64;
acc.0 += v.re as f64 * ph.cos() - v.im as f64 * ph.sin();
acc.1 += v.re as f64 * ph.sin() + v.im as f64 * ph.cos();
}
assert!((acc.0 - y[k].re as f64).abs() < 1e-2 && (acc.1 - y[k].im as f64).abs() < 1e-2, "n={n} k={k}");
}
}
}
/// The folded output must equal the direct IMDCT formula folded by TDAC symmetry.
#[test]
fn imdct_matches_direct_formula() {
let n = 240;
let n2 = n / 2;
let im = Imdct::new(n);
let x: Vec<f32> = (0..n2).map(|k| ((k * 37 % 11) as f32 - 5.0) / 5.0).collect();
let overlap = 0;
let mut out = vec![0f32; n2];
im.backward(&x, 1, &mut out, &[], overlap, &mut ImdctScratch::default());
// Direct IMDCT y[t] = sum X[k] cos(2pi/N (t + 1/2 + N/4)(k + 1/2)), t in 0..N.
let y: Vec<f64> = (0..n)
.map(|t| {
(0..n2)
.map(|k| x[k] as f64 * (2.0 * std::f64::consts::PI / n as f64 * (t as f64 + 0.5 + n as f64 / 4.0) * (k as f64 + 0.5)).cos())
.sum()
})
.collect();
// The folded half: out[i] relates to y in the middle half; check proportionality.
let mid: Vec<f64> = (0..n2).map(|i| y[n / 4 + i]).collect();
let dot: f64 = mid.iter().zip(&out).map(|(a, b)| a * *b as f64).sum();
let ea: f64 = mid.iter().map(|a| a * a).sum();
let eb: f64 = out.iter().map(|b| (*b as f64).powi(2)).sum();
let corr = dot.abs() / (ea.sqrt() * eb.sqrt());
assert!(corr > 0.999, "corr {corr}");
}
}
crates/audio/src/opus/decode/celt/mod.rs
+709
View File
@@ -0,0 +1,709 @@
//! CELT layer decoder (RFC 6716 §4.3).
pub(crate) mod bands;
pub(crate) mod energy;
pub(crate) mod mdct;
pub(crate) mod rate;
pub(crate) mod tables;
use std::sync::OnceLock;
use bands::{anti_collapse, denormalise_bands, quant_all_bands, renormalise_vector};
use energy::{unquant_coarse_energy, unquant_energy_finalise, unquant_fine_energy};
use mdct::ImdctScratch;
use rate::{Mode, compute_allocation};
use tables::{COMB_GAINS, EBANDS, MAX_LM, NB_EBANDS, OVERLAP, PREEMPH, SHORT_MDCT, SPREAD_ICDF, TAPSET_ICDF, TF_SELECT_TABLE, TRIM_ICDF};
use crate::opus::decode::range::{BITRES, RangeDecoder};
const DECODE_BUFFER_SIZE: usize = 2048;
const COMBFILTER_MINPERIOD: i32 = 15;
const PLC_PITCH_LAG_MAX: usize = 720;
const PLC_PITCH_LAG_MIN: usize = 100;
pub(crate) fn mode() -> &'static Mode {
static MODE: OnceLock<Mode> = OnceLock::new();
MODE.get_or_init(Mode::new)
}
/// CELT decoder state for one (mono or stereo) stream.
pub struct CeltDecoder {
m: &'static Mode,
/// Output channels.
channels: usize,
/// Channels coded in the stream.
pub stream_channels: usize,
downsample: usize,
pub start: usize,
pub end: usize,
pub disable_inv: bool,
pub rng: u32,
loss_count: u32,
/// Pitch lag the concealment found on the first lost frame, reused for the losses after it.
plc_pitch: usize,
skip_plc: bool,
postfilter_period: i32,
postfilter_period_old: i32,
postfilter_gain: f32,
postfilter_gain_old: f32,
postfilter_tapset: usize,
postfilter_tapset_old: usize,
preemph_mem: [f32; 2],
decode_mem: Vec<f32>,
old_band_e: [f32; 2 * NB_EBANDS],
old_log_e: [f32; 2 * NB_EBANDS],
old_log_e2: [f32; 2 * NB_EBANDS],
background_log_e: [f32; 2 * NB_EBANDS],
// scratch
x: Vec<f32>,
freq: Vec<f32>,
imdct: ImdctScratch,
pcm_scratch: Vec<f32>,
}
impl CeltDecoder {
/// `sample_rate` is the output rate (48000, 24000, 16000, 12000 or 8000).
pub fn new(sample_rate: u32, channels: usize) -> CeltDecoder {
let downsample = (48000 / sample_rate) as usize;
let mut d = CeltDecoder {
m: mode(),
channels,
stream_channels: channels,
downsample,
start: 0,
end: NB_EBANDS,
disable_inv: channels == 1,
rng: 0,
loss_count: 0,
plc_pitch: PLC_PITCH_LAG_MIN,
skip_plc: true,
postfilter_period: 0,
postfilter_period_old: 0,
postfilter_gain: 0.0,
postfilter_gain_old: 0.0,
postfilter_tapset: 0,
postfilter_tapset_old: 0,
preemph_mem: [0.0; 2],
decode_mem: vec![0.0; channels * (DECODE_BUFFER_SIZE + OVERLAP)],
old_band_e: [0.0; 2 * NB_EBANDS],
old_log_e: [0.0; 2 * NB_EBANDS],
old_log_e2: [0.0; 2 * NB_EBANDS],
background_log_e: [0.0; 2 * NB_EBANDS],
x: Vec::new(),
freq: Vec::new(),
imdct: ImdctScratch::default(),
pcm_scratch: Vec::new(),
};
d.reset();
d
}
/// `OPUS_RESET_STATE`.
pub fn reset(&mut self) {
self.rng = 0;
self.loss_count = 0;
self.plc_pitch = PLC_PITCH_LAG_MIN;
self.skip_plc = true;
self.postfilter_period = 0;
self.postfilter_period_old = 0;
self.postfilter_gain = 0.0;
self.postfilter_gain_old = 0.0;
self.postfilter_tapset = 0;
self.postfilter_tapset_old = 0;
self.preemph_mem = [0.0; 2];
self.decode_mem.fill(0.0);
self.old_band_e = [0.0; 2 * NB_EBANDS];
self.old_log_e = [-28.0; 2 * NB_EBANDS];
self.old_log_e2 = [-28.0; 2 * NB_EBANDS];
self.background_log_e = [0.0; 2 * NB_EBANDS];
}
fn mem_len(&self) -> usize {
DECODE_BUFFER_SIZE + OVERLAP
}
/// Decodes one CELT frame of `frame_size` samples (at the output rate) into interleaved `pcm`
/// (`channels` wide). With `accum`, the output is added to `pcm`. `data = None` (or fewer than
/// two bytes) runs the packet-loss concealment.
pub fn decode(
&mut self,
data: Option<&[u8]>,
frame_size: usize,
pcm: &mut [f32],
accum: bool,
dec: Option<&mut RangeDecoder>,
) -> Result<usize, crate::opus::decode::Error> {
let n_full = frame_size * self.downsample;
let lm = match n_full {
120 => 0,
240 => 1,
480 => 2,
960 => 3,
_ => return Err(crate::opus::decode::Error::InvalidArgument("CELT frame size")),
};
let data = match data {
Some(d) if d.len() > 1 => d,
_ => {
self.decode_lost(n_full, lm);
self.deemphasis(n_full, pcm, accum);
return Ok(frame_size);
}
};
if data.len() > 1275 {
return Err(crate::opus::decode::Error::InvalidPacket("CELT frame too long"));
}
match dec {
Some(d) => self.decode_frame(d, data.len(), n_full, lm),
None => {
let mut own = RangeDecoder::new(data);
self.decode_frame(&mut own, data.len(), n_full, lm)
}
}
self.deemphasis(n_full, pcm, accum);
Ok(frame_size)
}
#[allow(clippy::cognitive_complexity, reason = "the RFC 6716 CELT frame decode, ported as-is from FilmCraft")]
fn decode_frame(&mut self, dec: &mut RangeDecoder, len: usize, n: usize, lm: usize) {
let cc = self.channels;
let c = self.stream_channels;
let m = self.m;
let mm = 1usize << lm;
let start = self.start;
let end = self.end;
let eff_end = end.min(NB_EBANDS);
if self.loss_count == 0 {
self.skip_plc = false;
}
if c == 1 {
for i in 0..NB_EBANDS {
self.old_band_e[i] = self.old_band_e[i].max(self.old_band_e[NB_EBANDS + i]);
}
}
let total_bits_raw = len as i32 * 8;
let mut tell = dec.tell();
let silence = if tell >= total_bits_raw {
true
} else if tell == 1 {
dec.bit_logp(15)
} else {
false
};
if silence {
// Pretend we've read all the remaining bits.
dec.skip_to_end(total_bits_raw);
tell = total_bits_raw;
}
let mut postfilter_gain = 0.0f32;
let mut postfilter_pitch = 0i32;
let mut postfilter_tapset = 0usize;
if start == 0 && tell + 16 <= total_bits_raw {
if dec.bit_logp(1) {
let octave = dec.uint(6);
postfilter_pitch = ((16 << octave) + dec.bits(4 + octave)) as i32 - 1;
let qg = dec.bits(3);
if dec.tell() + 2 <= total_bits_raw {
postfilter_tapset = dec.icdf(&TAPSET_ICDF, 2);
}
postfilter_gain = 0.09375 * (qg + 1) as f32;
}
tell = dec.tell();
}
let is_transient = if lm > 0 && tell + 3 <= total_bits_raw {
let t = dec.bit_logp(3);
tell = dec.tell();
t
} else {
false
};
let short_blocks = is_transient;
let intra = if tell + 3 <= total_bits_raw { dec.bit_logp(3) } else { false };
unquant_coarse_energy(start, end, &mut self.old_band_e, intra, dec, c, lm);
let mut tf_res = [0i32; NB_EBANDS];
tf_decode(start, end, is_transient, &mut tf_res, lm, dec);
tell = dec.tell();
let mut spread = 2usize;
if tell + 4 <= total_bits_raw {
spread = dec.icdf(&SPREAD_ICDF, 5);
}
let cap = m.init_caps(lm, c);
let mut offsets = [0i32; NB_EBANDS];
let mut dynalloc_logp = 6;
let mut total_bits = total_bits_raw << BITRES;
let mut tellf = dec.tell_frac() as i32;
for i in start..end {
let width = (c as i32 * (EBANDS[i + 1] - EBANDS[i])) << lm;
let quanta = (width << BITRES).min((6 << BITRES).max(width));
let mut loop_logp = dynalloc_logp;
let mut boost = 0;
while tellf + (loop_logp << BITRES) < total_bits && boost < cap[i] {
let flag = dec.bit_logp(loop_logp as u32);
tellf = dec.tell_frac() as i32;
if !flag {
break;
}
boost += quanta;
total_bits -= quanta;
loop_logp = 1;
}
offsets[i] = boost;
if boost > 0 {
dynalloc_logp = 2.max(dynalloc_logp - 1);
}
}
let alloc_trim = if tellf + (6 << BITRES) <= total_bits { dec.icdf(&TRIM_ICDF, 7) as i32 } else { 5 };
let mut bits = ((len as i32 * 8) << BITRES) - dec.tell_frac() as i32 - 1;
let anti_collapse_rsv = if is_transient && lm >= 2 && bits >= ((lm as i32 + 2) << BITRES) { 1 << BITRES } else { 0 };
bits -= anti_collapse_rsv;
let alloc = compute_allocation(m, start, end, &offsets, &cap, alloc_trim, bits, c, lm, dec);
unquant_fine_energy(start, end, &mut self.old_band_e, &alloc.fine_quant, dec, c);
// Shift the decode memory by one frame.
let ml = self.mem_len();
for ch in 0..cc {
let mem = &mut self.decode_mem[ch * ml..(ch + 1) * ml];
mem.copy_within(n..DECODE_BUFFER_SIZE + OVERLAP / 2, 0);
}
let mut collapse_masks = vec![0u32; c * NB_EBANDS];
let mut x = std::mem::take(&mut self.x);
x.clear();
x.resize(c * n, 0.0);
{
let (xa, xb) = x.split_at_mut(n);
let yb = if c == 2 { Some(xb) } else { None };
let mut seed = self.rng;
quant_all_bands(
m,
start,
end,
xa,
yb,
&mut collapse_masks,
&alloc.pulses,
short_blocks,
spread,
alloc.dual_stereo,
alloc.intensity,
&tf_res,
((len as i32) * (8 << BITRES)) - anti_collapse_rsv,
alloc.balance,
dec,
lm,
alloc.coded_bands,
&mut seed,
self.disable_inv,
);
self.rng = seed;
}
let mut anti_collapse_on = false;
if anti_collapse_rsv > 0 {
anti_collapse_on = dec.bits(1) != 0;
}
let bits_left = len as i32 * 8 - dec.tell();
unquant_energy_finalise(start, end, &mut self.old_band_e, &alloc.fine_quant, &alloc.fine_priority, bits_left, dec, c);
if anti_collapse_on {
anti_collapse(
&mut x,
&collapse_masks,
lm,
c,
n,
start,
end,
&self.old_band_e,
&self.old_log_e,
&self.old_log_e2,
&alloc.pulses,
self.rng,
);
}
if silence {
for v in self.old_band_e[..c * NB_EBANDS].iter_mut() {
*v = -28.0;
}
}
self.synthesis(&x, n, start, eff_end, c, is_transient, lm, silence);
self.x = x;
// Post-filter.
for ch in 0..cc {
self.postfilter_period = self.postfilter_period.max(COMBFILTER_MINPERIOD);
self.postfilter_period_old = self.postfilter_period_old.max(COMBFILTER_MINPERIOD);
let mem = &mut self.decode_mem[ch * ml..(ch + 1) * ml];
let off = DECODE_BUFFER_SIZE - n;
comb_filter(
mem,
off,
self.postfilter_period_old,
self.postfilter_period,
SHORT_MDCT,
self.postfilter_gain_old,
self.postfilter_gain,
self.postfilter_tapset_old,
self.postfilter_tapset,
&m.window,
);
if lm != 0 {
comb_filter(
mem,
off + SHORT_MDCT,
self.postfilter_period,
postfilter_pitch,
n - SHORT_MDCT,
self.postfilter_gain,
postfilter_gain,
self.postfilter_tapset,
postfilter_tapset,
&m.window,
);
}
}
self.postfilter_period_old = self.postfilter_period;
self.postfilter_gain_old = self.postfilter_gain;
self.postfilter_tapset_old = self.postfilter_tapset;
self.postfilter_period = postfilter_pitch;
self.postfilter_gain = postfilter_gain;
self.postfilter_tapset = postfilter_tapset;
if lm != 0 {
self.postfilter_period_old = self.postfilter_period;
self.postfilter_gain_old = self.postfilter_gain;
self.postfilter_tapset_old = self.postfilter_tapset;
}
if c == 1 {
let (a, b) = self.old_band_e.split_at_mut(NB_EBANDS);
b.copy_from_slice(a);
}
if !is_transient {
self.old_log_e2 = self.old_log_e;
self.old_log_e = self.old_band_e;
for i in 0..2 * NB_EBANDS {
self.background_log_e[i] = (self.background_log_e[i] + mm as f32 * 0.001).min(self.old_band_e[i]);
}
} else {
for i in 0..2 * NB_EBANDS {
self.old_log_e[i] = self.old_log_e[i].min(self.old_band_e[i]);
}
}
for ch in 0..2 {
for i in (0..start).chain(end..NB_EBANDS) {
self.old_band_e[ch * NB_EBANDS + i] = 0.0;
self.old_log_e[ch * NB_EBANDS + i] = -28.0;
self.old_log_e2[ch * NB_EBANDS + i] = -28.0;
}
}
self.rng = dec.rng();
self.loss_count = 0;
}
/// Denormalisation + IMDCT into the decode memory (`celt_synthesis`).
#[allow(clippy::too_many_arguments)]
fn synthesis(&mut self, x: &[f32], n: usize, start: usize, eff_end: usize, c: usize, is_transient: bool, lm: usize, silence: bool) {
let cc = self.channels;
let m = self.m;
let mm = 1usize << lm;
let (bcount, nb, shift) = if is_transient { (mm, SHORT_MDCT, MAX_LM) } else { (1, SHORT_MDCT << lm, MAX_LM - lm) };
let ml = self.mem_len();
let mut freq = std::mem::take(&mut self.freq);
freq.clear();
freq.resize(2 * n, 0.0);
let (f0, f1) = freq.split_at_mut(n);
let out_off = DECODE_BUFFER_SIZE - n;
if cc == 2 && c == 1 {
denormalise_bands(x, f0, &self.old_band_e, start, eff_end, mm, self.downsample, silence);
f1.copy_from_slice(f0);
for (ch, f) in [(0usize, &*f1), (1usize, &*f0)] {
let mem = &mut self.decode_mem[ch * ml..(ch + 1) * ml];
for b in 0..bcount {
m.mdct[shift].backward(&f[b..], bcount, &mut mem[out_off + nb * b..], &m.window, OVERLAP, &mut self.imdct);
}
}
} else if cc == 1 && c == 2 {
denormalise_bands(x, f0, &self.old_band_e, start, eff_end, mm, self.downsample, silence);
denormalise_bands(&x[n..], f1, &self.old_band_e[NB_EBANDS..], start, eff_end, mm, self.downsample, silence);
for i in 0..n {
f0[i] = 0.5 * (f0[i] + f1[i]);
}
let mem = &mut self.decode_mem[..ml];
for b in 0..bcount {
m.mdct[shift].backward(&f0[b..], bcount, &mut mem[out_off + nb * b..], &m.window, OVERLAP, &mut self.imdct);
}
} else {
for ch in 0..cc {
denormalise_bands(&x[ch * n..], f0, &self.old_band_e[ch * NB_EBANDS..], start, eff_end, mm, self.downsample, silence);
let mem = &mut self.decode_mem[ch * ml..(ch + 1) * ml];
for b in 0..bcount {
m.mdct[shift].backward(&f0[b..], bcount, &mut mem[out_off + nb * b..], &m.window, OVERLAP, &mut self.imdct);
}
}
}
self.freq = freq;
}
fn deemphasis(&mut self, n: usize, pcm: &mut [f32], accum: bool) {
let cc = self.channels;
let ml = self.mem_len();
let ds = self.downsample;
let out_off = DECODE_BUFFER_SIZE - n;
let mut scratch = std::mem::take(&mut self.pcm_scratch);
for ch in 0..cc {
let mem = &self.decode_mem[ch * ml + out_off..ch * ml + out_off + n];
let mut mstate = self.preemph_mem[ch];
scratch.clear();
for &v in mem {
let tmp = v + 1e-30 + mstate;
mstate = PREEMPH * tmp;
scratch.push(tmp);
}
self.preemph_mem[ch] = mstate;
for j in 0..n / ds {
let v = scratch[j * ds] * (1.0 / 32768.0);
let o = &mut pcm[j * cc + ch];
if accum {
*o += v;
} else {
*o = v;
}
}
}
self.pcm_scratch = scratch;
}
/// Packet-loss concealment: pitch-periodic extension for the first lost frames, then
/// decaying shaped noise.
fn decode_lost(&mut self, n: usize, lm: usize) {
let cc = self.channels;
let ml = self.mem_len();
let m = self.m;
let pitch_based = self.loss_count < 5 && self.start == 0 && !self.skip_plc;
if !pitch_based {
let decay = if self.loss_count == 0 { 1.5 } else { 0.5 };
for ch in 0..cc {
for i in self.start..self.end {
let k = ch * NB_EBANDS + i;
self.old_band_e[k] = self.background_log_e[k].max(self.old_band_e[k] - decay);
}
}
let mut seed = self.rng;
let mut x = std::mem::take(&mut self.x);
x.clear();
x.resize(cc * n, 0.0);
let eff_end = self.end.min(NB_EBANDS);
for ch in 0..cc {
for i in self.start..eff_end {
let off = n * ch + ((EBANDS[i] as usize) << lm);
let blen = ((EBANDS[i + 1] - EBANDS[i]) as usize) << lm;
for v in x[off..off + blen].iter_mut() {
seed = bands::lcg_rand(seed);
*v = ((seed as i32) >> 20) as f32;
}
renormalise_vector(&mut x[off..off + blen], 1.0);
}
}
self.rng = seed;
for ch in 0..cc {
let mem = &mut self.decode_mem[ch * ml..(ch + 1) * ml];
mem.copy_within(n..DECODE_BUFFER_SIZE + OVERLAP / 2, 0);
}
let saved_sc = self.stream_channels;
self.stream_channels = cc;
self.synthesis(&x, n, self.start, eff_end, cc, false, lm, false);
self.stream_channels = saved_sc;
self.x = x;
// As libopus: after noise, the history holds no pitch to extend until two good frames
// in a row have been decoded.
self.skip_plc = true;
} else {
// Pitch search on the (mono-mixed) history.
let hist_len = 1024usize;
let base = DECODE_BUFFER_SIZE - hist_len;
let mut mono = vec![0f32; hist_len];
for ch in 0..cc {
for (i, v) in mono.iter_mut().enumerate() {
*v += self.decode_mem[ch * ml + base + i];
}
}
// The post-filter state is left alone: the next good frame cross-fades from it.
let period = if self.loss_count == 0 { find_pitch(&mono) } else { self.plc_pitch };
let period = period.clamp(PLC_PITCH_LAG_MIN, PLC_PITCH_LAG_MAX);
self.plc_pitch = period;
let fade = 0.8f32.powi(self.loss_count as i32 + 1);
let mut ext = vec![0f32; n + OVERLAP];
for ch in 0..cc {
let mem = &mut self.decode_mem[ch * ml..(ch + 1) * ml];
// Energy decay across the last two periods.
let e1: f32 = mem[DECODE_BUFFER_SIZE - period..DECODE_BUFFER_SIZE].iter().map(|v| v * v).sum();
let e0: f32 = mem[DECODE_BUFFER_SIZE - 2 * period..DECODE_BUFFER_SIZE - period].iter().map(|v| v * v).sum();
let decay = if e0 > 0.0 { (e1 / e0).sqrt().min(1.0) } else { 0.0 };
let mut g = 1.0f32;
for (i, e) in ext.iter_mut().enumerate() {
if i > 0 && i % period == 0 {
g *= decay;
}
let src = DECODE_BUFFER_SIZE - period + (i % period);
*e = mem[src] * g * fade;
}
mem.copy_within(n..DECODE_BUFFER_SIZE + OVERLAP / 2, 0);
let out_off = DECODE_BUFFER_SIZE - n;
// Cross-fade the start with the previous overlap so the transition is smooth.
mem[out_off..out_off + n].copy_from_slice(&ext[..n]);
let tail = &ext[n..n + OVERLAP];
for i in 0..OVERLAP / 2 {
mem[DECODE_BUFFER_SIZE + i] = m.window[i] * tail[OVERLAP - 1 - i] + m.window[OVERLAP - 1 - i] * tail[i];
}
}
}
self.loss_count += 1;
}
}
fn find_pitch(x: &[f32]) -> usize {
let n = x.len();
let win = 512.min(n - PLC_PITCH_LAG_MAX);
let tgt = &x[n - win..];
let mut best = PLC_PITCH_LAG_MIN;
let mut best_score = f32::MIN;
let e_t: f32 = tgt.iter().map(|v| v * v).sum::<f32>() + 1e-9;
for lag in PLC_PITCH_LAG_MIN..=PLC_PITCH_LAG_MAX {
let src = &x[n - win - lag..n - lag];
let mut xy = 0f32;
let mut yy = 1e-9f32;
for (a, b) in tgt.iter().zip(src) {
xy += a * b;
yy += b * b;
}
let score = xy / (e_t * yy).sqrt();
if score > best_score {
best_score = score;
best = lag;
}
}
best
}
fn tf_decode(start: usize, end: usize, is_transient: bool, tf_res: &mut [i32; NB_EBANDS], lm: usize, dec: &mut RangeDecoder) {
let mut budget = dec.storage() as i32 * 8;
let mut tell = dec.tell();
let mut logp: u32 = if is_transient { 2 } else { 4 };
let tf_select_rsv = lm > 0 && tell + logp as i32 + 1 <= budget;
budget -= tf_select_rsv as i32;
let mut tf_changed = 0;
let mut curr = 0;
for r in tf_res.iter_mut().take(end).skip(start) {
if tell + logp as i32 <= budget {
curr ^= dec.bit_logp(logp) as i32;
tell = dec.tell();
tf_changed |= curr;
}
*r = curr;
logp = if is_transient { 4 } else { 5 };
}
let mut tf_select = 0;
let ti = 4 * is_transient as usize;
if tf_select_rsv && TF_SELECT_TABLE[lm][ti + tf_changed as usize] != TF_SELECT_TABLE[lm][ti + 2 + tf_changed as usize] {
tf_select = dec.bit_logp(1) as usize;
}
for r in tf_res.iter_mut().take(end).skip(start) {
*r = TF_SELECT_TABLE[lm][ti + 2 * tf_select + *r as usize] as i32;
}
}
/// Pitch post-filter (in place on `buf[off .. off + n]`, reading history before `off`).
#[allow(clippy::too_many_arguments)]
fn comb_filter(buf: &mut [f32], off: usize, t0: i32, t1: i32, n: usize, g0: f32, g1: f32, tapset0: usize, tapset1: usize, window: &[f32; OVERLAP]) {
if g0 == 0.0 && g1 == 0.0 {
return;
}
let t0 = t0.max(COMBFILTER_MINPERIOD) as usize;
let t1 = t1.max(COMBFILTER_MINPERIOD) as usize;
let g00 = g0 * COMB_GAINS[tapset0][0];
let g01 = g0 * COMB_GAINS[tapset0][1];
let g02 = g0 * COMB_GAINS[tapset0][2];
let g10 = g1 * COMB_GAINS[tapset1][0];
let g11 = g1 * COMB_GAINS[tapset1][1];
let g12 = g1 * COMB_GAINS[tapset1][2];
let mut x1 = buf[off + 1 - t1];
let mut x2 = buf[off - t1];
let mut x3 = buf[off - t1 - 1];
let mut x4 = buf[off - t1 - 2];
let overlap = if g0 == g1 && t0 == t1 && tapset0 == tapset1 { 0 } else { OVERLAP.min(n) };
let mut i = 0;
while i < overlap {
let p = off + i;
let x0 = buf[p + 2 - t1];
let f = window[i] * window[i];
let a = 1.0 - f;
buf[p] = buf[p]
+ a * g00 * buf[p - t0]
+ a * g01 * (buf[p + 1 - t0] + buf[p - t0 - 1])
+ a * g02 * (buf[p + 2 - t0] + buf[p - t0 - 2])
+ f * g10 * x2
+ f * g11 * (x1 + x3)
+ f * g12 * (x0 + x4);
x4 = x3;
x3 = x2;
x2 = x1;
x1 = x0;
i += 1;
}
if g1 == 0.0 {
return;
}
// Constant filter for the rest of the block.
let t = t1;
for i in overlap..n {
let p = off + i;
buf[p] = buf[p] + g10 * buf[p - t] + g11 * (buf[p + 1 - t] + buf[p - t - 1]) + g12 * (buf[p + 2 - t] + buf[p - t - 2]);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn concealment_leaves_the_post_filter_state_for_the_next_good_frame() {
let mut d = CeltDecoder::new(48000, 1);
d.skip_plc = false;
// A decoded history with a clear pitch, so the concealment takes the pitch-based path.
let ml = d.mem_len();
for (i, v) in d.decode_mem[..ml].iter_mut().enumerate() {
*v = (i as f32 * std::f32::consts::TAU / 240.0).sin() * 1000.0;
}
d.postfilter_period = 300;
d.postfilter_period_old = 300;
d.postfilter_gain = 0.5;
d.postfilter_gain_old = 0.5;
for _ in 0..3 {
d.decode_lost(960, 3);
}
assert_eq!((d.postfilter_period, d.postfilter_period_old), (300, 300));
assert_eq!(d.plc_pitch % 240, 0, "the lag is a period of the history");
}
#[test]
fn noise_concealment_waits_for_two_good_frames_before_pitch_concealment() {
// Only the noise-based concealment draws from `rng`, so a changed `rng` tells the paths apart.
fn loss_is_noise(d: &mut CeltDecoder) -> bool {
let rng = d.rng;
d.decode(None, 960, &mut [0.0; 960], false, None).unwrap();
d.rng != rng
}
let good = [0x55u8; 40];
let mut d = CeltDecoder::new(48000, 1);
for _ in 0..2 {
d.decode(Some(&good), 960, &mut [0.0; 960], false, None).unwrap();
}
for _ in 0..5 {
assert!(!loss_is_noise(&mut d), "the first losses extend the pitch");
}
assert!(loss_is_noise(&mut d), "the sixth loss in a row is noise");
d.decode(Some(&good), 960, &mut [0.0; 960], false, None).unwrap();
assert!(loss_is_noise(&mut d), "one good frame after noise does not re-enable pitch concealment");
for _ in 0..2 {
d.decode(Some(&good), 960, &mut [0.0; 960], false, None).unwrap();
}
assert!(!loss_is_noise(&mut d), "two good frames in a row do");
}
}
crates/audio/src/opus/decode/celt/rate.rs
+584
View File
@@ -0,0 +1,584 @@
//! Pulse cache and bit allocation (RFC 6716 §4.3.3, §4.3.4.1).
use super::tables::{ALLOC, EBANDS, LOG2_FRAC_TABLE, MAX_LM, NB_EBANDS, OVERLAP};
use crate::opus::decode::range::{BITRES, RangeDecoder};
const MAX_PSEUDO: i32 = 40;
const LOG_MAX_PSEUDO: i32 = 6;
pub const MAX_FINE_BITS: i32 = 8;
const FINE_OFFSET: i32 = 21;
pub const QTHETA_OFFSET: i32 = 4;
pub const QTHETA_OFFSET_TWOPHASE: i32 = 16;
const ALLOC_STEPS: i32 = 6;
/// Number of pulses represented by pseudo-pulse index `i`.
pub fn get_pulses(i: i32) -> i32 {
if i < 8 { i } else { (8 + (i & 7)) << ((i >> 3) - 1) }
}
/// Conservative (rounded up) log2 of `val` with `frac` fractional bits.
pub fn log2_frac(mut val: u32, mut frac: u32) -> i32 {
let mut l = crate::opus::decode::range::ilog(val);
if val & val.wrapping_sub(1) != 0 {
if l > 16 {
val = ((val - 1) >> (l - 16)) + 1;
} else {
val <<= 16 - l;
}
let mut l = (l - 1) << frac;
loop {
let b = (val >> 16) as i32;
l += b << frac;
val = (val + b as u32) >> b;
val = (val * val + 0x7FFF) >> 15;
if frac == 0 {
break;
}
frac -= 1;
}
l + (val > 0x8000) as i32
} else {
l -= 1;
l << frac
}
}
/// PVQ codebook size V(N, K) as u128 (saturating far above 2^32).
pub fn pvq_v(n: usize, k: usize) -> u128 {
// V(N,K) = V(N-1,K) + V(N,K-1) + V(N-1,K-1); computed row by row.
if k == 0 {
return 1;
}
if n == 0 {
return 0;
}
let mut row = vec![0u128; k + 1]; // V(0, k)
row[0] = 1;
for _ in 0..n {
let mut prev_diag = row[0]; // V(n-1, 0)
// V(n, 0) = 1
for kk in 1..=k {
let up = row[kk]; // V(n-1, kk)
let v = up.saturating_add(row[kk - 1]).saturating_add(prev_diag);
prev_diag = up;
row[kk] = v.min(u128::MAX >> 8);
}
row[0] = 1;
}
row[k]
}
fn fits_in32(n: usize, k: i32) -> bool {
pvq_v(n, k as usize) <= u32::MAX as u128
}
/// Precomputed per-mode data shared by all CELT decoders.
pub struct Mode {
/// log2 of band width in 1/8 bits.
pub log_n: [i32; NB_EBANDS],
/// Index into `cache_bits` for `(LM+1)*NB_EBANDS + band` (-1 when unused).
pub cache_index: Vec<i16>,
/// `cache[0]` = max pseudo-pulses; `cache[k]` = bits(k) - 1 in 1/8 bits.
pub cache_bits: Vec<u8>,
/// Per-band maximum allocation: `[(2*LM + C-1)*NB_EBANDS + band]`.
pub caps: Vec<u8>,
pub window: [f32; OVERLAP],
pub mdct: [super::mdct::Imdct; MAX_LM + 1],
}
impl Mode {
pub fn new() -> Mode {
let mut log_n = [0i32; NB_EBANDS];
for (i, l) in log_n.iter_mut().enumerate() {
*l = log2_frac((EBANDS[i + 1] - EBANDS[i]) as u32, BITRES);
}
let (cache_index, cache_bits) = compute_pulse_cache(MAX_LM as i32);
let mut m = Mode {
log_n,
cache_index,
cache_bits,
caps: Vec::new(),
window: [0.0; OVERLAP],
mdct: [super::mdct::Imdct::new(1920), super::mdct::Imdct::new(960), super::mdct::Imdct::new(480), super::mdct::Imdct::new(240)],
};
m.caps = compute_caps(&m);
for i in 0..OVERLAP {
let x = std::f64::consts::FRAC_PI_2 * (i as f64 + 0.5) / OVERLAP as f64;
let s = x.sin();
m.window[i] = (std::f64::consts::FRAC_PI_2 * s * s).sin() as f32;
}
m
}
#[inline]
pub fn cache(&self, lm: i32, band: usize) -> &[u8] {
let idx = self.cache_index[((lm + 1) as usize) * NB_EBANDS + band];
&self.cache_bits[idx.max(0) as usize..]
}
pub fn bits2pulses(&self, band: usize, lm: i32, bits: i32) -> i32 {
let cache = self.cache(lm, band);
let mut lo = 0i32;
let mut hi = cache[0] as i32;
let bits = bits - 1;
for _ in 0..LOG_MAX_PSEUDO {
let mid = (lo + hi + 1) >> 1;
if cache[mid as usize] as i32 >= bits {
hi = mid;
} else {
lo = mid;
}
}
let lo_bits = if lo == 0 { -1 } else { cache[lo as usize] as i32 };
if bits - lo_bits <= cache[hi as usize] as i32 - bits { lo } else { hi }
}
pub fn pulses2bits(&self, band: usize, lm: i32, pulses: i32) -> i32 {
if pulses == 0 { 0 } else { self.cache(lm, band)[pulses as usize] as i32 + 1 }
}
/// Per-band allocation caps for this frame (`init_caps`).
pub fn init_caps(&self, lm: usize, c: usize) -> [i32; NB_EBANDS] {
let mut cap = [0i32; NB_EBANDS];
for (i, cp) in cap.iter_mut().enumerate() {
let n = (EBANDS[i + 1] - EBANDS[i]) << lm;
*cp = (self.caps[NB_EBANDS * (2 * lm + c - 1) + i] as i32 + 64) * c as i32 * n >> 2;
}
cap
}
}
fn compute_pulse_cache(lm_max: i32) -> (Vec<i16>, Vec<u8>) {
let nb = NB_EBANDS;
let rows = (lm_max + 2) as usize;
let mut index = vec![-1i16; nb * rows];
let mut entries: Vec<(usize, i32, usize)> = Vec::new(); // (N, K, offset)
let mut curr = 0usize;
for i in 0..rows {
for j in 0..nb {
let n = (((EBANDS[j + 1] - EBANDS[j]) << i) >> 1) as usize;
index[i * nb + j] = -1;
// Reuse the entry of an earlier band with the same size.
'search: for k in 0..=i {
for nn in 0..nb {
if k == i && nn >= j {
break;
}
if n == (((EBANDS[nn + 1] - EBANDS[nn]) << k) >> 1) as usize {
index[i * nb + j] = index[k * nb + nn];
break 'search;
}
}
}
if index[i * nb + j] == -1 && n != 0 {
let mut k = 0;
while k < MAX_PSEUDO && fits_in32(n, get_pulses(k + 1)) {
k += 1;
}
index[i * nb + j] = curr as i16;
entries.push((n, k, curr));
curr += (k + 1) as usize;
}
}
}
let mut bits = vec![0u8; curr];
for &(n, k, off) in &entries {
let maxk = get_pulses(k) as usize;
let tmp = required_bits(n, maxk, BITRES);
for j in 1..=k {
bits[off + j as usize] = (tmp[get_pulses(j) as usize] - 1) as u8;
}
bits[off] = k as u8;
}
(index, bits)
}
/// Bits (1/2^frac units, rounded up) needed for each K in `0..=maxk` for dimension `n`.
fn required_bits(n: usize, maxk: usize, frac: u32) -> Vec<i32> {
let mut out = vec![0i32; maxk + 1];
if n == 1 {
for o in out.iter_mut().skip(1) {
*o = 1 << frac;
}
} else {
for (k, o) in out.iter_mut().enumerate().skip(1) {
let v = pvq_v(n, k);
*o = log2_frac(v as u32, frac);
}
}
out
}
fn compute_caps(m: &Mode) -> Vec<u8> {
let nb = NB_EBANDS;
let mut caps = vec![0u8; (MAX_LM + 1) * 2 * nb];
for i in 0..=MAX_LM as i32 {
for c in 1..=2i32 {
for j in 0..nb {
let mut n0 = EBANDS[j + 1] - EBANDS[j];
let max_bits = if n0 << i == 1 {
c * (1 + MAX_FINE_BITS) << BITRES
} else {
let mut lm0 = 0;
if n0 > 2 {
n0 >>= 1;
lm0 -= 1;
} else if n0 <= 1 {
lm0 = i.min(1);
n0 <<= lm0;
}
let pcache = m.cache(lm0, j);
let mut mb = pcache[pcache[0] as usize] as i32 + 1;
let mut n = n0;
for k in 0..(i - lm0) {
mb <<= 1;
let offset = ((m.log_n[j] + ((lm0 + k) << BITRES)) >> 1) - QTHETA_OFFSET;
let num = 459 * ((2 * n - 1) * offset + mb);
let den = ((2 * n - 1) << 9) - 459;
let qb = ((num + (den >> 1)) / den).min(57);
mb += qb;
n <<= 1;
}
if c == 2 {
mb <<= 1;
let offset = ((m.log_n[j] + (i << BITRES)) >> 1) - if n == 2 { QTHETA_OFFSET_TWOPHASE } else { QTHETA_OFFSET };
let ndof = 2 * n - 1 - (n == 2) as i32;
let p = if n == 2 { 512 } else { 487 };
let num = p * (mb + ndof * offset);
let den = (ndof << 9) - p;
let qb = ((num + (den >> 1)) / den).min(if n == 2 { 64 } else { 61 });
mb += qb;
}
let ndof = c * n + if c == 2 && n > 2 { 1 } else { 0 };
let mut offset = ((m.log_n[j] + (i << BITRES)) >> 1) - FINE_OFFSET;
if n == 2 {
offset += (1 << BITRES) >> 2;
}
let num = mb + ndof * offset;
let den = (ndof - 1) << BITRES;
let qb = ((num + (den >> 1)) / den).min(MAX_FINE_BITS);
mb += c * qb << BITRES;
mb
};
let w = c * ((EBANDS[j + 1] - EBANDS[j]) << i);
let v = (4 * max_bits / w) - 64;
caps[((i * 2 + c - 1) as usize) * nb + j] = v.clamp(0, 255) as u8;
}
}
}
caps
}
/// Result of the allocation process.
pub struct Allocation {
pub coded_bands: usize,
pub intensity: usize,
pub dual_stereo: bool,
pub balance: i32,
pub pulses: [i32; NB_EBANDS],
pub fine_quant: [i32; NB_EBANDS],
pub fine_priority: [i32; NB_EBANDS],
}
/// `clt_compute_allocation` (decoder side).
#[allow(clippy::too_many_arguments)]
pub fn compute_allocation(
m: &Mode,
start: usize,
end: usize,
offsets: &[i32; NB_EBANDS],
cap: &[i32; NB_EBANDS],
alloc_trim: i32,
total: i32,
c: usize,
lm: usize,
dec: &mut RangeDecoder,
) -> Allocation {
let ci = c as i32;
let lmi = lm as i32;
let mut total = total.max(0);
let mut skip_start = start;
let skip_rsv = if total >= 1 << BITRES { 1 << BITRES } else { 0 };
total -= skip_rsv;
let mut intensity_rsv = 0;
let mut dual_stereo_rsv = 0;
if c == 2 {
intensity_rsv = LOG2_FRAC_TABLE[end - start];
if intensity_rsv > total {
intensity_rsv = 0;
} else {
total -= intensity_rsv;
dual_stereo_rsv = if total >= 1 << BITRES { 1 << BITRES } else { 0 };
total -= dual_stereo_rsv;
}
}
let mut bits1 = [0i32; NB_EBANDS];
let mut bits2 = [0i32; NB_EBANDS];
let mut thresh = [0i32; NB_EBANDS];
let mut trim_offset = [0i32; NB_EBANDS];
for j in start..end {
let w = EBANDS[j + 1] - EBANDS[j];
thresh[j] = (ci << BITRES).max((3 * w << lmi << BITRES) >> 4);
trim_offset[j] = ci * w * (alloc_trim - 5 - lmi) * (end as i32 - j as i32 - 1) * (1 << (lmi + BITRES as i32)) >> 6;
if w << lmi == 1 {
trim_offset[j] -= ci << BITRES;
}
}
let nb_alloc = ALLOC.len() as i32;
let mut lo = 1;
let mut hi = nb_alloc - 1;
loop {
let mut done = false;
let mut psum = 0;
let mid = (lo + hi) >> 1;
for j in (start..end).rev() {
let n = EBANDS[j + 1] - EBANDS[j];
let mut bitsj = ci * n * (ALLOC[mid as usize][j] as i32) << lmi >> 2;
if bitsj > 0 {
bitsj = (bitsj + trim_offset[j]).max(0);
}
bitsj += offsets[j];
if bitsj >= thresh[j] || done {
done = true;
psum += bitsj.min(cap[j]);
} else if bitsj >= ci << BITRES {
psum += ci << BITRES;
}
}
if psum > total {
hi = mid - 1;
} else {
lo = mid + 1;
}
if lo > hi {
break;
}
}
hi = lo;
lo -= 1;
for j in start..end {
let n = EBANDS[j + 1] - EBANDS[j];
let mut bits1j = ci * n * (ALLOC[lo as usize][j] as i32) << lmi >> 2;
let mut bits2j = if hi >= nb_alloc { cap[j] } else { ci * n * (ALLOC[hi as usize][j] as i32) << lmi >> 2 };
if bits1j > 0 {
bits1j = (bits1j + trim_offset[j]).max(0);
}
if bits2j > 0 {
bits2j = (bits2j + trim_offset[j]).max(0);
}
if lo > 0 {
bits1j += offsets[j];
}
bits2j += offsets[j];
if offsets[j] > 0 {
skip_start = j;
}
bits2j = (bits2j - bits1j).max(0);
bits1[j] = bits1j;
bits2[j] = bits2j;
}
interp_bits2pulses(m, start, end, skip_start, &bits1, &bits2, &thresh, cap, total, skip_rsv, intensity_rsv, dual_stereo_rsv, c, lm, dec)
}
#[allow(clippy::too_many_arguments)]
fn interp_bits2pulses(
m: &Mode,
start: usize,
end: usize,
skip_start: usize,
bits1: &[i32; NB_EBANDS],
bits2: &[i32; NB_EBANDS],
thresh: &[i32; NB_EBANDS],
cap: &[i32; NB_EBANDS],
mut total: i32,
skip_rsv: i32,
mut intensity_rsv: i32,
mut dual_stereo_rsv: i32,
c: usize,
lm: usize,
dec: &mut RangeDecoder,
) -> Allocation {
let ci = c as i32;
let alloc_floor = ci << BITRES;
let stereo = (c > 1) as i32;
let log_m = (lm as i32) << BITRES;
let mut lo = 0;
let mut hi = 1 << ALLOC_STEPS;
for _ in 0..ALLOC_STEPS {
let mid = (lo + hi) >> 1;
let mut psum = 0;
let mut done = false;
for j in (start..end).rev() {
let tmp = bits1[j] + (mid * bits2[j] >> ALLOC_STEPS);
if tmp >= thresh[j] || done {
done = true;
psum += tmp.min(cap[j]);
} else if tmp >= alloc_floor {
psum += alloc_floor;
}
}
if psum > total {
hi = mid;
} else {
lo = mid;
}
}
let mut bits = [0i32; NB_EBANDS];
let mut psum = 0;
let mut done = false;
for j in (start..end).rev() {
let mut tmp = bits1[j] + (lo * bits2[j] >> ALLOC_STEPS);
if tmp < thresh[j] && !done {
tmp = if tmp >= alloc_floor { alloc_floor } else { 0 };
} else {
done = true;
}
tmp = tmp.min(cap[j]);
bits[j] = tmp;
psum += tmp;
}
let s = start;
let mut coded_bands = end;
loop {
let j = coded_bands - 1;
if j <= skip_start {
total += skip_rsv;
break;
}
let mut left = total - psum;
let span = EBANDS[coded_bands] - EBANDS[s];
let percoeff = left / span;
left -= span * percoeff;
let rem = (left - (EBANDS[j] - EBANDS[s])).max(0);
let band_width = EBANDS[coded_bands] - EBANDS[j];
let mut band_bits = bits[j] + percoeff * band_width + rem;
if band_bits >= thresh[j].max(alloc_floor + (1 << BITRES)) {
if dec.bit_logp(1) {
break;
}
psum += 1 << BITRES;
band_bits -= 1 << BITRES;
}
psum -= bits[j] + intensity_rsv;
if intensity_rsv > 0 {
intensity_rsv = LOG2_FRAC_TABLE[j - s];
}
psum += intensity_rsv;
if band_bits >= alloc_floor {
psum += alloc_floor;
bits[j] = alloc_floor;
} else {
bits[j] = 0;
}
coded_bands -= 1;
}
let intensity = if intensity_rsv > 0 { s + dec.uint((coded_bands + 1 - s) as u32) as usize } else { 0 };
if intensity <= s {
total += dual_stereo_rsv;
dual_stereo_rsv = 0;
}
let dual_stereo = if dual_stereo_rsv > 0 { dec.bit_logp(1) } else { false };
let mut left = total - psum;
let span = EBANDS[coded_bands] - EBANDS[s];
let percoeff = left / span;
left -= span * percoeff;
for j in s..coded_bands {
bits[j] += percoeff * (EBANDS[j + 1] - EBANDS[j]);
}
for j in s..coded_bands {
let tmp = left.min(EBANDS[j + 1] - EBANDS[j]);
bits[j] += tmp;
left -= tmp;
}
let mut ebits = [0i32; NB_EBANDS];
let mut fine_priority = [0i32; NB_EBANDS];
let mut balance = 0;
for j in s..coded_bands {
let n0 = EBANDS[j + 1] - EBANDS[j];
let n = n0 << lm;
let bit = bits[j] + balance;
let mut excess;
if n > 1 {
excess = (bit - cap[j]).max(0);
bits[j] = bit - excess;
let den = ci * n + if c == 2 && n > 2 && !dual_stereo && j < intensity { 1 } else { 0 };
let nclogn = den * (m.log_n[j] + log_m);
let mut offset = (nclogn >> 1) - den * FINE_OFFSET;
if n == 2 {
offset += (den << BITRES) >> 2;
}
if bits[j] + offset < den * 2 << BITRES {
offset += nclogn >> 2;
} else if bits[j] + offset < den * 3 << BITRES {
offset += nclogn >> 3;
}
let mut eb = (bits[j] + offset + (den << (BITRES - 1))).max(0);
eb = (eb / den) >> BITRES;
if ci * eb > (bits[j] >> BITRES) {
eb = bits[j] >> stereo >> BITRES;
}
eb = eb.min(MAX_FINE_BITS);
ebits[j] = eb;
fine_priority[j] = (eb * (den << BITRES) >= bits[j] + offset) as i32;
bits[j] -= ci * eb << BITRES;
} else {
excess = (bit - (ci << BITRES)).max(0);
bits[j] = bit - excess;
ebits[j] = 0;
fine_priority[j] = 1;
}
if excess > 0 {
let extra_fine = (excess >> (stereo + BITRES as i32)).min(MAX_FINE_BITS - ebits[j]);
ebits[j] += extra_fine;
let extra_bits = extra_fine * ci << BITRES;
fine_priority[j] = (extra_bits >= excess - balance) as i32;
excess -= extra_bits;
}
balance = excess;
}
for j in coded_bands..end {
ebits[j] = bits[j] >> stereo >> BITRES;
bits[j] = 0;
fine_priority[j] = (ebits[j] < 1) as i32;
}
Allocation { coded_bands, intensity, dual_stereo, balance, pulses: bits, fine_quant: ebits, fine_priority }
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn log2_frac_values() {
assert_eq!(log2_frac(1, 3), 0);
assert_eq!(log2_frac(2, 3), 8);
assert_eq!(log2_frac(4, 3), 16);
assert_eq!(log2_frac(6, 3), 21);
assert_eq!(log2_frac(18, 3), 34);
assert_eq!(log2_frac(22, 3), 36);
}
#[test]
fn pvq_counts() {
// V(N,1) = 2N, V(1,K) = 2 (K>0), V(2,K) = 4K.
assert_eq!(pvq_v(5, 1), 10);
assert_eq!(pvq_v(1, 7), 2);
assert_eq!(pvq_v(2, 9), 36);
assert_eq!(pvq_v(3, 2), 18);
assert_eq!(pvq_v(0, 0), 1);
assert_eq!(pvq_v(0, 3), 0);
}
#[test]
fn caps_match_reference_shape() {
let m = Mode::new();
// 1-bin bands at LM=0 (mono and stereo) are capped at 224.
assert!(m.caps[..8].iter().all(|&c| c == 224));
assert!(m.caps[21..29].iter().all(|&c| c == 224));
let first_rows: [u8; 42] = [
224, 224, 224, 224, 224, 224, 224, 224, 160, 160, 160, 160, 185, 185, 185, 178, 178, 168, 134, 61, 37, 224, 224, 224, 224, 224, 224, 224,
224, 240, 240, 240, 240, 207, 207, 207, 198, 198, 183, 144, 66, 40,
];
assert_eq!(&m.caps[..42], &first_rows[..]);
}
}
crates/audio/src/opus/decode/celt/tables.rs
+130
View File
@@ -0,0 +1,130 @@
//! Constant tables of the 48 kHz CELT mode (RFC 6716 §4.3).
/// Number of energy bands.
pub const NB_EBANDS: usize = 21;
/// Band edges in units of 2.5 ms MDCT bins (Table 55).
pub const EBANDS: [i32; NB_EBANDS + 1] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 16, 20, 24, 28, 34, 40, 48, 60, 78, 100];
/// Overlap of the low-overlap window (samples at 48 kHz).
pub const OVERLAP: usize = 120;
/// Size of the shortest MDCT (2.5 ms).
pub const SHORT_MDCT: usize = 120;
/// Largest LM (20 ms frames).
pub const MAX_LM: usize = 3;
/// Pre-emphasis coefficient.
pub const PREEMPH: f32 = 0.850_006_1;
/// Static allocation table (Table 57), indexed `[quality][band]`, in 1/32 bit per MDCT bin.
pub const ALLOC: [[u8; NB_EBANDS]; 11] = transpose([
[0, 90, 110, 118, 126, 134, 144, 152, 162, 172, 200],
[0, 80, 100, 110, 119, 127, 137, 145, 155, 165, 200],
[0, 75, 90, 103, 112, 120, 130, 138, 148, 158, 200],
[0, 69, 84, 93, 104, 114, 124, 132, 142, 152, 200],
[0, 63, 78, 86, 95, 103, 113, 123, 133, 143, 200],
[0, 56, 71, 80, 89, 97, 107, 117, 127, 137, 200],
[0, 49, 65, 75, 83, 91, 101, 111, 121, 131, 200],
[0, 40, 58, 70, 78, 85, 95, 105, 115, 125, 200],
[0, 34, 51, 65, 72, 78, 88, 98, 108, 118, 198],
[0, 29, 45, 59, 66, 72, 82, 92, 102, 112, 193],
[0, 20, 39, 53, 60, 66, 76, 86, 96, 106, 188],
[0, 18, 32, 47, 54, 60, 70, 80, 90, 100, 183],
[0, 10, 26, 40, 47, 54, 64, 74, 84, 94, 178],
[0, 0, 20, 31, 39, 47, 57, 67, 77, 87, 173],
[0, 0, 12, 23, 32, 41, 51, 61, 71, 81, 168],
[0, 0, 0, 15, 25, 35, 45, 55, 65, 75, 163],
[0, 0, 0, 4, 17, 29, 39, 49, 59, 69, 158],
[0, 0, 0, 0, 12, 23, 33, 43, 53, 63, 153],
[0, 0, 0, 0, 1, 16, 26, 36, 46, 56, 148],
[0, 0, 0, 0, 0, 10, 15, 20, 30, 45, 129],
[0, 0, 0, 0, 0, 1, 1, 1, 1, 20, 104],
]);
const fn transpose(t: [[u8; 11]; NB_EBANDS]) -> [[u8; NB_EBANDS]; 11] {
let mut out = [[0u8; NB_EBANDS]; 11];
let mut b = 0;
while b < NB_EBANDS {
let mut q = 0;
while q < 11 {
out[q][b] = t[b][q];
q += 1;
}
b += 1;
}
out
}
/// Mean band energy (log2 domain) removed before coarse quantisation.
pub const E_MEANS: [f32; 25] = [
6.4375, 6.25, 5.75, 5.3125, 5.0625, 4.8125, 4.5, 4.375, 4.875, 4.6875, 4.5625, 4.4375, 4.875, 4.625, 4.3125, 4.5, 4.375, 4.625, 4.75, 4.4375,
3.75, 3.75, 3.75, 3.75, 3.75,
];
/// Inter-frame prediction coefficient (alpha) per LM, Q15.
pub const PRED_COEF: [f32; 4] = [29440.0 / 32768.0, 26112.0 / 32768.0, 21248.0 / 32768.0, 16384.0 / 32768.0];
/// Inter-band prediction coefficient (beta) per LM for inter frames, Q15.
pub const BETA_COEF: [f32; 4] = [30147.0 / 32768.0, 22282.0 / 32768.0, 12124.0 / 32768.0, 6554.0 / 32768.0];
/// Beta for intra frames.
pub const BETA_INTRA: f32 = 4915.0 / 32768.0;
/// Laplace model parameters for coarse energy: `[LM][intra][2*band + {0: prob, 1: decay}]`.
pub const E_PROB_MODEL: [[[u8; 42]; 2]; 4] = [
[
[
72, 127, 65, 129, 66, 128, 65, 128, 64, 128, 62, 128, 64, 128, 64, 128, 92, 78, 92, 79, 92, 78, 90, 79, 116, 41, 115, 40, 114, 40, 132,
26, 132, 26, 145, 17, 161, 12, 176, 10, 177, 11,
],
[
24, 179, 48, 138, 54, 135, 54, 132, 53, 134, 56, 133, 55, 132, 55, 132, 61, 114, 70, 96, 74, 88, 75, 88, 87, 74, 89, 66, 91, 67, 100, 59,
108, 50, 120, 40, 122, 37, 97, 43, 78, 50,
],
],
[
[
83, 78, 84, 81, 88, 75, 86, 74, 87, 71, 90, 73, 93, 74, 93, 74, 109, 40, 114, 36, 117, 34, 117, 34, 143, 17, 145, 18, 146, 19, 162, 12,
165, 10, 178, 7, 189, 6, 190, 8, 177, 9,
],
[
23, 178, 54, 115, 63, 102, 66, 98, 69, 99, 74, 89, 71, 91, 73, 91, 78, 89, 86, 80, 92, 66, 93, 64, 102, 59, 103, 60, 104, 60, 117, 52,
123, 44, 138, 35, 133, 31, 97, 38, 77, 45,
],
],
[
[
61, 90, 93, 60, 105, 42, 107, 41, 110, 45, 116, 38, 113, 38, 112, 38, 124, 26, 132, 27, 136, 19, 140, 20, 155, 14, 159, 16, 158, 18, 170,
13, 177, 10, 187, 8, 192, 6, 175, 9, 159, 10,
],
[
21, 178, 59, 110, 71, 86, 75, 85, 84, 83, 91, 66, 88, 73, 87, 72, 92, 75, 98, 72, 105, 58, 107, 54, 115, 52, 114, 55, 112, 56, 129, 51,
132, 40, 150, 33, 140, 29, 98, 35, 77, 42,
],
],
[
[
42, 121, 96, 66, 108, 43, 111, 40, 117, 44, 123, 32, 120, 36, 119, 33, 127, 33, 134, 34, 139, 21, 147, 23, 152, 20, 158, 25, 154, 26,
166, 21, 173, 16, 184, 13, 184, 10, 150, 13, 139, 15,
],
[
22, 178, 63, 114, 74, 82, 84, 83, 92, 82, 103, 62, 96, 72, 96, 67, 101, 73, 107, 72, 113, 55, 118, 52, 125, 52, 118, 52, 117, 55, 135,
49, 137, 39, 157, 32, 145, 29, 97, 33, 77, 40,
],
],
];
/// Inverse CDF for coarse energy when only a few bits remain.
pub const SMALL_ENERGY_ICDF: [u8; 3] = [2, 1, 0];
/// Conservative log2 in 1/8 bits of the number of coded bands (intensity reservation).
pub const LOG2_FRAC_TABLE: [i32; 24] = [0, 8, 13, 16, 19, 21, 23, 24, 26, 27, 28, 29, 30, 31, 32, 32, 33, 34, 34, 35, 36, 36, 37, 37];
/// TF resolution changes: `[LM][4*transient + 2*tf_select + tf_flag]`.
pub const TF_SELECT_TABLE: [[i8; 8]; 4] =
[[0, -1, 0, -1, 0, -1, 0, -1], [0, -1, 0, -2, 1, 0, 1, -1], [0, -2, 0, -3, 2, 0, 1, -1], [0, -2, 0, -3, 3, 0, 1, -1]];
/// Spreading decision `{7, 2, 21, 2}/32`.
pub const SPREAD_ICDF: [u8; 4] = [25, 23, 2, 0];
/// Allocation trim (Table 58).
pub const TRIM_ICDF: [u8; 11] = [126, 124, 119, 109, 87, 41, 19, 9, 4, 2, 0];
/// Post-filter tapset `{2, 1, 1}/4`.
pub const TAPSET_ICDF: [u8; 3] = [2, 1, 0];
/// Post-filter tap gains per tapset.
pub const COMB_GAINS: [[f32; 3]; 3] =
[[0.306_640_62, 0.217_041_02, 0.129_638_67], [0.463_867_2, 0.268_066_4, 0.0], [0.799_804_7, 0.100_097_656, 0.0]];
/// Hadamard "sequency" ordering for 2, 4, 8 and 16 blocks, concatenated.
pub const ORDERY: [usize; 30] = [1, 0, 3, 0, 2, 1, 7, 0, 4, 3, 6, 1, 5, 2, 15, 0, 8, 7, 12, 3, 11, 4, 14, 1, 9, 6, 13, 2, 10, 5];
crates/audio/src/opus/decode/decoder.rs
+419
View File
@@ -0,0 +1,419 @@
//! Single-stream Opus decoder: mode switching, hybrid summation, redundancy and transitions
//! (RFC 6716 §4, §4.5).
use crate::opus::decode::celt::CeltDecoder;
use crate::opus::decode::celt::tables::OVERLAP;
use crate::opus::decode::packet::{Bandwidth, Mode, Packet};
use crate::opus::decode::range::RangeDecoder;
use crate::opus::decode::silk::{LostFlag, SilkDecoder};
use crate::opus::decode::{Error, Result};
/// Output sample rates supported by the decoder.
pub const SAMPLE_RATES: [u32; 5] = [8000, 12000, 16000, 24000, 48000];
/// Decoder for one elementary Opus stream (mono or stereo), like `OpusDecoder`.
pub struct StreamDecoder {
rate: u32,
channels: usize,
celt: CeltDecoder,
silk: SilkDecoder,
stream_channels: usize,
bandwidth: Bandwidth,
mode: Option<Mode>,
prev_mode: Option<Mode>,
frame_size: usize,
prev_redundancy: bool,
last_packet_duration: usize,
range_final: u32,
/// Output gain in Q8 dB (OpusHead output gain).
gain_q8: i16,
}
impl StreamDecoder {
/// Creates a decoder producing `channels` (1 or 2) at `rate` (8/12/16/24/48 kHz).
pub fn new(rate: u32, channels: usize) -> Result<StreamDecoder> {
if !SAMPLE_RATES.contains(&rate) {
return Err(Error::InvalidArgument("sample rate must be 8000, 12000, 16000, 24000 or 48000"));
}
if !(1..=2).contains(&channels) {
return Err(Error::InvalidArgument("channels must be 1 or 2"));
}
Ok(StreamDecoder {
rate,
channels,
celt: CeltDecoder::new(rate, channels),
silk: SilkDecoder::new(rate, channels),
stream_channels: channels,
bandwidth: Bandwidth::Fullband,
mode: None,
prev_mode: None,
frame_size: rate as usize / 400,
prev_redundancy: false,
last_packet_duration: 0,
range_final: 0,
gain_q8: 0,
})
}
/// Output sampling rate in Hz.
pub fn sample_rate(&self) -> u32 {
self.rate
}
/// Output channels (1 or 2).
pub fn channels(&self) -> usize {
self.channels
}
/// Final range coder state of the last decoded frame (for conformance checks).
pub fn final_range(&self) -> u32 {
self.range_final
}
/// Duration (samples per channel) of the last decoded packet.
pub fn last_packet_duration(&self) -> usize {
self.last_packet_duration
}
/// Sets the output gain in Q8 dB (as in the OpusHead header).
pub fn set_gain_q8(&mut self, gain: i16) {
self.gain_q8 = gain;
}
/// Disables the 180-degree intensity-stereo phase inversion (RFC 8251 §10).
pub fn set_phase_inversion_disabled(&mut self, disabled: bool) {
self.celt.disable_inv = disabled;
}
/// `OPUS_RESET_STATE`.
pub fn reset(&mut self) {
self.celt.reset();
self.silk.reset();
self.stream_channels = self.channels;
self.bandwidth = Bandwidth::Fullband;
self.mode = None;
self.prev_mode = None;
self.frame_size = self.rate as usize / 400;
self.prev_redundancy = false;
self.last_packet_duration = 0;
self.range_final = 0;
}
/// Decodes a packet into interleaved samples appended to `out`. `None` conceals one lost
/// packet of the last packet's duration. Returns samples per channel.
pub fn decode_interleaved(&mut self, packet: Option<&[u8]>, out: &mut Vec<f32>) -> Result<usize> {
match packet {
Some(p) if !p.is_empty() => self.decode_packet(p, out),
_ => {
let n = if self.last_packet_duration > 0 { self.last_packet_duration } else { self.rate as usize / 50 };
self.conceal(n, out)
}
}
}
/// Conceals `n` samples (a multiple of 2.5 ms).
pub fn conceal(&mut self, n: usize, out: &mut Vec<f32>) -> Result<usize> {
let f2_5 = self.rate as usize / 400;
if n == 0 || !n.is_multiple_of(f2_5) {
return Err(Error::InvalidArgument("concealment length must be a multiple of 2.5 ms"));
}
let start = out.len();
out.resize(start + n * self.channels, 0.0);
let mut done = 0;
while done < n {
let r = self.decode_frame(None, &mut out[start + done * self.channels..], n - done, false)?;
done += r;
}
self.last_packet_duration = n;
Ok(n)
}
/// Decodes the in-band FEC (LBRR) of `packet` for the previous (lost) packet of `n` samples.
pub fn decode_fec(&mut self, packet: &[u8], n: usize, out: &mut Vec<f32>) -> Result<usize> {
let p = Packet::parse(packet)?;
let pfs = p.toc.frame_samples(self.rate);
if n < pfs || p.toc.mode == Mode::CeltOnly || self.mode == Some(Mode::CeltOnly) {
return self.conceal(n, out);
}
let start = out.len();
if n > pfs {
self.conceal(n - pfs, out)?;
}
self.mode = Some(p.toc.mode);
self.bandwidth = p.toc.bandwidth;
self.frame_size = pfs;
self.stream_channels = p.toc.channels();
let off = out.len();
out.resize(off + pfs * self.channels, 0.0);
self.decode_frame(Some(p.frames[0]), &mut out[off..], pfs, true)?;
self.last_packet_duration = n;
debug_assert_eq!(out.len() - start, n * self.channels);
Ok(n)
}
fn decode_packet(&mut self, data: &[u8], out: &mut Vec<f32>) -> Result<usize> {
let p = Packet::parse(data)?;
let fs = p.toc.frame_samples(self.rate);
self.mode = Some(p.toc.mode);
self.bandwidth = p.toc.bandwidth;
self.frame_size = fs;
self.stream_channels = p.toc.channels();
let start = out.len();
out.resize(start + fs * p.frames.len() * self.channels, 0.0);
let mut nb = 0;
for f in &p.frames {
let r = self.decode_frame(Some(f), &mut out[start + nb * self.channels..], fs, false)?;
nb += r;
}
out.truncate(start + nb * self.channels);
self.last_packet_duration = nb;
Ok(nb)
}
/// `opus_decode_frame`: decodes one frame (or conceals when `data` is `None`/≤1 byte).
#[allow(clippy::cognitive_complexity, reason = "the RFC 6716 frame decoding sequence, ported as-is from FilmCraft")]
fn decode_frame(&mut self, data: Option<&[u8]>, pcm: &mut [f32], frame_size: usize, fec: bool) -> Result<usize> {
let ch = self.channels;
let fs = self.rate as usize;
let f20 = fs / 50;
let f10 = f20 >> 1;
let f5 = f10 >> 1;
let f2_5 = f5 >> 1;
let mut frame_size = frame_size.min(fs / 25 * 3);
let data = match data {
Some(d) if d.len() > 1 => Some(d),
_ => {
frame_size = frame_size.min(self.frame_size);
None
}
};
let (mut audiosize, mode, bandwidth);
match data {
Some(_) => {
audiosize = self.frame_size;
mode = self.mode.unwrap_or(Mode::CeltOnly);
bandwidth = Some(self.bandwidth);
}
None => {
audiosize = frame_size;
bandwidth = None;
mode = match self.prev_mode {
None => {
pcm[..audiosize * ch].fill(0.0);
return Ok(audiosize);
}
Some(m) => m,
};
if audiosize > f20 {
let mut done = 0;
while done < audiosize {
let r = self.decode_frame(None, &mut pcm[done * ch..], (audiosize - done).min(f20), false)?;
done += r;
}
return Ok(frame_size);
} else if audiosize < f20 {
if audiosize > f10 {
audiosize = f10;
} else if mode != Mode::SilkOnly && audiosize > f5 && audiosize < f10 {
audiosize = f5;
}
}
}
}
let data_len = data.map_or(0, |d| d.len());
let mut dec_storage;
let mut dec: Option<&mut RangeDecoder> = match data {
Some(d) => {
dec_storage = RangeDecoder::new(d);
Some(&mut dec_storage)
}
None => None,
};
let mut transition = false;
if data.is_some()
&& self.prev_mode.is_some()
&& ((mode == Mode::CeltOnly && self.prev_mode != Some(Mode::CeltOnly) && !self.prev_redundancy)
|| (mode != Mode::CeltOnly && self.prev_mode == Some(Mode::CeltOnly)))
{
transition = true;
}
let mut pcm_transition = Vec::new();
if transition && mode == Mode::CeltOnly {
pcm_transition.resize(f5 * ch, 0.0);
let n = f5.min(audiosize);
self.decode_frame(None, &mut pcm_transition, n, false)?;
}
if audiosize > frame_size {
return Err(Error::InvalidArgument("frame larger than output buffer"));
}
frame_size = audiosize;
// SILK.
let mut pcm_silk: Vec<f32> = Vec::new();
if mode != Mode::CeltOnly {
pcm_silk.resize(frame_size.max(f10) * ch, 0.0);
if self.prev_mode == Some(Mode::CeltOnly) {
self.silk.reset();
}
let payload_ms = (1000 * audiosize / fs).max(10);
let internal_rate = if mode == Mode::SilkOnly {
match bandwidth {
Some(Bandwidth::Narrowband) => 8000,
Some(Bandwidth::Mediumband) => 12000,
_ => 16000,
}
} else {
16000
};
if data.is_some() {
self.silk.set_stream(self.stream_channels, internal_rate);
}
let lost = if data.is_none() {
LostFlag::Lost
} else if fec {
LostFlag::Lbrr
} else {
LostFlag::Normal
};
let mut decoded = 0;
while decoded < frame_size {
let first = decoded == 0;
let n = match self.silk.decode(dec.as_deref_mut(), lost, first, payload_ms, &mut pcm_silk[decoded * ch..]) {
Ok(n) => n,
Err(e) => {
if lost != LostFlag::Normal {
let n = frame_size - decoded;
pcm_silk[decoded * ch..(decoded + n) * ch].fill(0.0);
n
} else {
return Err(e);
}
}
};
if n == 0 {
break;
}
decoded += n;
}
}
let mut start_band = 0;
let mut redundancy = false;
let mut celt_to_silk = false;
let mut redundancy_bytes = 0usize;
let mut len = data_len;
if !fec
&& mode != Mode::CeltOnly
&& let Some(d) = dec.as_deref_mut()
&& d.tell() + 17 + 20 * (self.mode == Some(Mode::Hybrid)) as i32 <= 8 * len as i32
{
redundancy = if mode == Mode::Hybrid { d.bit_logp(12) } else { true };
if redundancy {
celt_to_silk = d.bit_logp(1);
redundancy_bytes = if mode == Mode::Hybrid { d.uint(256) as usize + 2 } else { len - ((d.tell() as usize + 7) >> 3) };
if redundancy_bytes > len || (len - redundancy_bytes) * 8 < d.tell() as usize {
len = 0;
redundancy_bytes = 0;
redundancy = false;
} else {
len -= redundancy_bytes;
}
d.shrink_storage(redundancy_bytes);
}
}
if mode != Mode::CeltOnly {
start_band = 17;
}
if redundancy {
transition = false;
}
if transition && mode != Mode::CeltOnly {
pcm_transition.resize(f5 * ch, 0.0);
let n = f5.min(audiosize);
self.decode_frame(None, &mut pcm_transition, n, false)?;
}
if let Some(bw) = bandwidth {
self.celt.end = bw.celt_end_band();
}
self.celt.stream_channels = self.stream_channels;
let mut redundant_audio = vec![0f32; if redundancy { f5 * ch } else { 0 }];
let mut redundant_rng = 0u32;
let red_data = data.map(|d| &d[len..len + redundancy_bytes]);
if redundancy && celt_to_silk {
self.celt.start = 0;
self.celt.decode(red_data, f5, &mut redundant_audio, false, None)?;
redundant_rng = self.celt.rng;
}
self.celt.start = start_band;
if mode != Mode::SilkOnly {
let celt_frame = f20.min(frame_size);
if self.prev_mode.is_some() && Some(mode) != self.prev_mode && !self.prev_redundancy {
self.celt.reset();
}
let cdata = if fec { None } else { data.map(|d| &d[..len]) };
let cdec = if fec { None } else { dec.as_deref_mut() };
self.celt.decode(cdata, celt_frame, pcm, false, cdec)?;
} else {
pcm[..frame_size * ch].fill(0.0);
if self.prev_mode == Some(Mode::Hybrid) && !(redundancy && celt_to_silk && self.prev_redundancy) {
self.celt.start = 0;
let silence = [0xFFu8, 0xFF];
self.celt.decode(Some(&silence), f2_5, pcm, true, None)?;
}
}
if mode != Mode::CeltOnly {
for (o, s) in pcm[..frame_size * ch].iter_mut().zip(&pcm_silk) {
*o += *s;
}
}
let window = &crate::opus::decode::celt::mode().window;
if redundancy && !celt_to_silk {
self.celt.reset();
self.celt.start = 0;
self.celt.decode(red_data, f5, &mut redundant_audio, false, None)?;
redundant_rng = self.celt.rng;
let o = ch * (frame_size - f2_5);
let tail: Vec<f32> = pcm[o..o + f2_5 * ch].to_vec();
smooth_fade(&tail, &redundant_audio[ch * f2_5..], &mut pcm[o..], f2_5, ch, window, self.rate);
}
if redundancy && celt_to_silk {
pcm[..f2_5 * ch].copy_from_slice(&redundant_audio[..f2_5 * ch]);
let cur: Vec<f32> = pcm[ch * f2_5..ch * 2 * f2_5].to_vec();
smooth_fade(&redundant_audio[ch * f2_5..], &cur, &mut pcm[ch * f2_5..], f2_5, ch, window, self.rate);
}
if transition {
if audiosize >= f5 {
pcm[..f2_5 * ch].copy_from_slice(&pcm_transition[..f2_5 * ch]);
let cur: Vec<f32> = pcm[ch * f2_5..ch * 2 * f2_5].to_vec();
smooth_fade(&pcm_transition[ch * f2_5..], &cur, &mut pcm[ch * f2_5..], f2_5, ch, window, self.rate);
} else {
let cur: Vec<f32> = pcm[..ch * f2_5].to_vec();
smooth_fade(&pcm_transition, &cur, pcm, f2_5, ch, window, self.rate);
}
}
if self.gain_q8 != 0 {
let g = (6.488_141e-4f32 * self.gain_q8 as f32).exp2();
for v in pcm[..frame_size * ch].iter_mut() {
*v *= g;
}
}
self.range_final = match dec.as_deref() {
Some(d) if data_len > 1 => d.rng() ^ redundant_rng,
_ => 0,
};
self.prev_mode = Some(mode);
self.prev_redundancy = redundancy && !celt_to_silk;
Ok(audiosize)
}
}
fn smooth_fade(in1: &[f32], in2: &[f32], out: &mut [f32], overlap: usize, ch: usize, window: &[f32; OVERLAP], rate: u32) {
let inc = (48000 / rate) as usize;
for c in 0..ch {
for i in 0..overlap {
let w = window[i * inc] * window[i * inc];
out[i * ch + c] = w * in2[i * ch + c] + (1.0 - w) * in1[i * ch + c];
}
}
}
crates/audio/src/opus/decode/header.rs
+234
View File
@@ -0,0 +1,234 @@
//! Opus identification header (`OpusHead`, RFC 7845 §5.1) and the ISO-BMFF `dOps` box payload.
use crate::opus::decode::{Error, Result};
/// Decoder configuration: channel layout, multistream mapping, pre-skip and output gain.
#[non_exhaustive]
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct OpusHead {
/// Output channel count (1..=255).
pub channels: usize,
/// Samples (at 48 kHz) to discard from the start of the decoded stream.
pub pre_skip: u16,
/// Sample rate of the original input (informational only).
pub input_sample_rate: u32,
/// Output gain in Q7.8 dB.
pub output_gain: i16,
/// Channel mapping family (0: mono/stereo, 1: Vorbis order surround, 255: unspecified order).
pub mapping_family: u8,
/// Number of elementary streams.
pub streams: usize,
/// Number of coupled (stereo) streams; these come first.
pub coupled_streams: usize,
/// Output channel → decoded channel index (255 = silence).
pub mapping: Vec<u8>,
}
impl OpusHead {
/// A family-0 configuration for plain mono/stereo streams (no header present).
pub fn simple(channels: usize) -> Result<OpusHead> {
if !(1..=2).contains(&channels) {
return Err(Error::InvalidArgument("family 0 supports 1 or 2 channels"));
}
Ok(OpusHead {
channels,
pre_skip: 0,
input_sample_rate: 48000,
output_gain: 0,
mapping_family: 0,
streams: 1,
coupled_streams: channels - 1,
mapping: (0..channels as u8).collect(),
})
}
/// Parses an `OpusHead` packet (Ogg first packet, Matroska `CodecPrivate`).
pub fn parse(data: &[u8]) -> Result<OpusHead> {
if data.len() < 19 || &data[..8] != b"OpusHead" {
return Err(Error::InvalidHeader("missing OpusHead magic"));
}
let version = data[8];
if version >> 4 != 0 {
return Err(Error::Unsupported("OpusHead major version"));
}
let channels = data[9] as usize;
let pre_skip = u16::from_le_bytes([data[10], data[11]]);
let input_sample_rate = u32::from_le_bytes([data[12], data[13], data[14], data[15]]);
let output_gain = i16::from_le_bytes([data[16], data[17]]);
let family = data[18];
Self::build(channels, pre_skip, input_sample_rate, output_gain, family, &data[19..])
}
/// Parses the payload of an ISO-BMFF `dOps` box (Opus in MP4, big-endian fields).
pub fn from_dops(data: &[u8]) -> Result<OpusHead> {
if data.len() < 11 {
return Err(Error::InvalidHeader("dOps too short"));
}
if data[0] != 0 {
return Err(Error::Unsupported("dOps version"));
}
let channels = data[1] as usize;
let pre_skip = u16::from_be_bytes([data[2], data[3]]);
let input_sample_rate = u32::from_be_bytes([data[4], data[5], data[6], data[7]]);
let output_gain = i16::from_be_bytes([data[8], data[9]]);
Self::build(channels, pre_skip, input_sample_rate, output_gain, data[10], &data[11..])
}
/// Serialises as an `OpusHead` packet.
pub fn to_bytes(&self) -> Vec<u8> {
let mut v = b"OpusHead".to_vec();
v.push(1);
v.push(self.channels as u8);
v.extend(self.pre_skip.to_le_bytes());
v.extend(self.input_sample_rate.to_le_bytes());
v.extend(self.output_gain.to_le_bytes());
v.push(self.mapping_family);
if self.mapping_family != 0 {
v.push(self.streams as u8);
v.push(self.coupled_streams as u8);
v.extend(&self.mapping);
}
v
}
/// Validate a header independently of its wire parser. This is required for callers of
/// [`crate::opus::decode::Decoder::from_head`], which may construct an `OpusHead` directly with `usize`
/// values that cannot occur in the byte format.
pub fn validate(&self) -> Result<()> {
if !(1..=255).contains(&self.channels) || self.mapping.len() != self.channels {
return Err(Error::InvalidHeader("bad channel count or mapping length"));
}
if self.mapping_family == 0 {
if self.channels > 2
|| self.streams != 1
|| self.coupled_streams != self.channels - 1
|| self.mapping.iter().copied().ne(0..self.channels as u8)
{
return Err(Error::InvalidHeader("invalid family 0 mapping"));
}
return Ok(());
}
if self.mapping_family == 3 {
return Err(Error::Unsupported("channel mapping family 3 (projection)"));
}
if self.streams == 0
|| self.streams > 255
|| self.coupled_streams > self.streams
|| self.streams.checked_add(self.coupled_streams).is_none_or(|n| n > 255)
{
return Err(Error::InvalidHeader("bad stream counts"));
}
if self.mapping_family == 1 && !(1..=8).contains(&self.channels) {
return Err(Error::InvalidHeader("family 1 allows 1 to 8 channels"));
}
if self.mapping.iter().any(|&m| m != 255 && m as usize >= self.streams + self.coupled_streams) {
return Err(Error::InvalidHeader("channel mapping out of range"));
}
Ok(())
}
fn build(channels: usize, pre_skip: u16, input_sample_rate: u32, output_gain: i16, family: u8, rest: &[u8]) -> Result<OpusHead> {
if channels == 0 {
return Err(Error::InvalidHeader("zero channels"));
}
let (streams, coupled, mapping) = if family == 0 {
if channels > 2 {
return Err(Error::InvalidHeader("family 0 allows at most 2 channels"));
}
(1, channels - 1, (0..channels as u8).collect::<Vec<_>>())
} else {
if family == 3 {
return Err(Error::Unsupported("channel mapping family 3 (projection)"));
}
if rest.len() < 2 + channels {
return Err(Error::InvalidHeader("truncated channel mapping table"));
}
let streams = rest[0] as usize;
let coupled = rest[1] as usize;
if streams == 0 || coupled > streams || streams + coupled > 255 {
return Err(Error::InvalidHeader("bad stream counts"));
}
if family == 1 && !(1..=8).contains(&channels) {
return Err(Error::InvalidHeader("family 1 allows 1 to 8 channels"));
}
let mapping = rest[2..2 + channels].to_vec();
if mapping.iter().any(|&m| m != 255 && m as usize >= streams + coupled) {
return Err(Error::InvalidHeader("channel mapping out of range"));
}
(streams, coupled, mapping)
};
let head =
OpusHead { channels, pre_skip, input_sample_rate, output_gain, mapping_family: family, streams, coupled_streams: coupled, mapping };
head.validate()?;
Ok(head)
}
/// Linear output gain factor.
pub fn gain_linear(&self) -> f32 {
10f32.powf(self.output_gain as f32 / (20.0 * 256.0))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_family0_and_roundtrip() {
let mut h = b"OpusHead".to_vec();
h.extend([1, 2, 0x38, 0x01, 0x80, 0xBB, 0, 0, 0x00, 0x01, 0]);
let head = OpusHead::parse(&h).unwrap();
assert_eq!(head.channels, 2);
assert_eq!(head.pre_skip, 312);
assert_eq!(head.input_sample_rate, 48000);
assert_eq!(head.output_gain, 256);
assert_eq!((head.streams, head.coupled_streams), (1, 1));
assert_eq!(head.to_bytes(), h);
assert!((head.gain_linear() - 1.122).abs() < 1e-3);
}
#[test]
fn parse_family1_51() {
let mut h = b"OpusHead".to_vec();
h.extend([1, 6, 0x38, 0x01, 0x80, 0xBB, 0, 0, 0, 0, 1, 4, 2, 0, 4, 1, 2, 3, 5]);
let head = OpusHead::parse(&h).unwrap();
assert_eq!((head.channels, head.streams, head.coupled_streams), (6, 4, 2));
assert_eq!(head.mapping, vec![0, 4, 1, 2, 3, 5]);
assert_eq!(OpusHead::parse(&head.to_bytes()).unwrap(), head);
}
#[test]
fn rejects_garbage() {
assert!(OpusHead::parse(b"OpusHea").is_err());
assert!(OpusHead::parse(b"OpusTags...........").is_err());
let mut h = b"OpusHead".to_vec();
h.extend([1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0]);
assert!(OpusHead::parse(&h).is_err());
let mut h = b"OpusHead".to_vec();
h.extend([1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 5]);
assert!(OpusHead::parse(&h).is_err());
}
#[test]
fn dops() {
let d = [0u8, 2, 0x01, 0x38, 0, 0, 0xBB, 0x80, 0, 0, 0];
let head = OpusHead::from_dops(&d).unwrap();
assert_eq!(head.pre_skip, 312);
assert_eq!(head.channels, 2);
}
#[test]
fn directly_constructed_headers_are_checked_before_decoder_allocation() {
let head = OpusHead {
channels: 1,
pre_skip: 0,
input_sample_rate: 48_000,
output_gain: 0,
mapping_family: 255,
streams: usize::MAX,
coupled_streams: 0,
mapping: vec![0],
};
assert!(crate::opus::decode::Decoder::from_head(head, 48_000).is_err());
}
}
crates/audio/src/opus/decode/mod.rs
+47
View File
@@ -0,0 +1,47 @@
//! The decoder half of [`crate::opus`]: FilmCraft's `filmcraft-opus` (storytold/filmcraft@8fda2f2).
// The fixed-point parts mirror the normative integer arithmetic; keep C-like shift expressions.
#![allow(clippy::precedence, clippy::int_plus_one)]
pub(crate) mod celt;
mod decoder;
mod header;
mod multistream;
pub mod packet;
pub(crate) mod range;
pub(crate) mod silk;
pub use decoder::{SAMPLE_RATES, StreamDecoder};
pub use header::OpusHead;
pub use multistream::Decoder;
pub use packet::{Mode, Packet, Toc};
/// Errors from header parsing and decoding.
#[non_exhaustive]
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Error {
/// An argument is out of range (sampling rate, channel count, concealment length, output size).
InvalidArgument(&'static str),
/// A malformed packet (RFC 6716 §3.4).
InvalidPacket(&'static str),
/// A malformed `OpusHead` (RFC 7845 §5.1) or MP4 `dOps` box.
InvalidHeader(&'static str),
/// A valid stream uses a feature this decoder does not implement.
Unsupported(&'static str),
}
impl std::fmt::Display for Error {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Error::InvalidArgument(s) => write!(f, "invalid argument: {s}"),
Error::InvalidPacket(s) => write!(f, "invalid packet: {s}"),
Error::InvalidHeader(s) => write!(f, "invalid header: {s}"),
Error::Unsupported(s) => write!(f, "unsupported: {s}"),
}
}
}
impl std::error::Error for Error {}
/// Result with this module's [`Error`].
pub type Result<T> = std::result::Result<T, Error>;
crates/audio/src/opus/decode/multistream.rs
+196
View File
@@ -0,0 +1,196 @@
//! Public decoder: OpusHead-configured (multistream) decoding to planar f32.
use crate::opus::decode::decoder::StreamDecoder;
use crate::opus::decode::header::OpusHead;
use crate::opus::decode::packet::Packet;
use crate::opus::decode::{Error, Result};
/// Opus decoder for an `OpusHead` configuration (mono, stereo, or multistream surround).
///
/// Produces planar f32 at the chosen output rate. Pre-skip is *not* removed unless
/// [`Decoder::set_trim_pre_skip`] is enabled (containers usually handle it via edit lists /
/// granule positions). The header's output gain is applied.
pub struct Decoder {
head: OpusHead,
rate: u32,
streams: Vec<StreamDecoder>,
trim_pre_skip: bool,
skip_remaining: usize,
scratch: Vec<Vec<f32>>,
}
impl Decoder {
/// Creates a 48 kHz decoder from an `OpusHead` packet (Ogg / Matroska `CodecPrivate`).
pub fn new(opus_head: &[u8]) -> Result<Decoder> {
Self::from_head(OpusHead::parse(opus_head)?, 48000)
}
/// Creates a decoder for plain mono/stereo packets without a header.
pub fn simple(sample_rate: u32, channels: usize) -> Result<Decoder> {
Self::from_head(OpusHead::simple(channels)?, sample_rate)
}
/// Creates a decoder for a parsed header at `sample_rate` (8/12/16/24/48 kHz).
pub fn from_head(head: OpusHead, sample_rate: u32) -> Result<Decoder> {
head.validate()?;
let mut streams = Vec::with_capacity(head.streams);
for s in 0..head.streams {
let ch = if s < head.coupled_streams { 2 } else { 1 };
let mut d = StreamDecoder::new(sample_rate, ch)?;
d.set_gain_q8(head.output_gain);
streams.push(d);
}
Ok(Decoder { rate: sample_rate, streams, trim_pre_skip: false, skip_remaining: 0, scratch: vec![Vec::new(); head.streams], head })
}
/// The `OpusHead` the decoder was created from.
pub fn head(&self) -> &OpusHead {
&self.head
}
/// Output sampling rate in Hz.
pub fn sample_rate(&self) -> u32 {
self.rate
}
/// Output channels (the header's channel count).
pub fn channels(&self) -> usize {
self.head.channels
}
/// Pre-skip in samples at the output rate.
pub fn pre_skip(&self) -> usize {
self.head.pre_skip as usize * self.rate as usize / 48000
}
/// When enabled, the first [`Self::pre_skip`] output samples are dropped.
pub fn set_trim_pre_skip(&mut self, enable: bool) {
self.trim_pre_skip = enable;
self.skip_remaining = if enable { self.pre_skip() } else { 0 };
}
/// Disables intensity-stereo phase inversion (useful when downmixing; RFC 8251 section 10).
pub fn set_phase_inversion_disabled(&mut self, disabled: bool) {
for s in &mut self.streams {
s.set_phase_inversion_disabled(disabled);
}
}
/// Final range coder state of the first stream (conformance testing).
pub fn final_range(&self) -> u32 {
self.streams.first().map_or(0, |s| s.final_range())
}
/// Reset every stream's decoder state (e.g. after a seek). With [`Self::set_trim_pre_skip`]
/// on, trimming starts again too, which is right only when decoding restarts from the start of
/// the stream (RFC 7845 §4 applies pre-skip once); turn it off before a seek anywhere else.
pub fn reset(&mut self) {
for s in &mut self.streams {
s.reset();
}
self.skip_remaining = if self.trim_pre_skip { self.pre_skip() } else { 0 };
}
/// Decodes one packet (`None` = lost packet, concealed) into planar channels.
pub fn decode(&mut self, packet: Option<&[u8]>) -> Result<Vec<Vec<f32>>> {
self.decode_or_conceal(packet, None)
}
/// Conceals one lost or undecodable packet. With its bytes at hand and a TOC and framing that
/// parse, the concealment is as long as the packet; otherwise as long as the last packet
/// (20 ms before any).
pub fn conceal(&mut self, packet: Option<&[u8]>) -> Result<Vec<Vec<f32>>> {
let dur = packet.and_then(|d| self.packet_duration(d));
self.decode_or_conceal(None, dur)
}
/// Samples per channel, at the output rate, in a packet whose framing parses.
fn packet_duration(&self, data: &[u8]) -> Option<usize> {
let p = if self.streams.len() > 1 { Packet::parse_self_delimited(data).ok()?.0 } else { Packet::parse(data).ok()? };
Some(p.toc.frame_samples(self.rate) * p.frames.len()).filter(|&n| n > 0)
}
fn decode_or_conceal(&mut self, packet: Option<&[u8]>, conceal_len: Option<usize>) -> Result<Vec<Vec<f32>>> {
let n = self.streams.len();
for b in &mut self.scratch {
b.clear();
}
let mut samples = None;
match packet {
Some(data) if !data.is_empty() => {
let mut off = 0;
for s in 0..n {
let rest = &data[off..];
if rest.is_empty() {
return Err(Error::InvalidPacket("truncated multistream packet"));
}
let used = if s + 1 < n { Packet::parse_self_delimited(rest)?.1 } else { rest.len() };
let pkt = &rest[..used];
let r = if s + 1 < n {
// Re-frame the self-delimited packet as a regular one for the stream decoder.
let (p, _) = Packet::parse_self_delimited(pkt)?;
let regular = reframe(&p);
self.streams[s].decode_interleaved(Some(&regular), &mut self.scratch[s])?
} else {
self.streams[s].decode_interleaved(Some(pkt), &mut self.scratch[s])?
};
if *samples.get_or_insert(r) != r {
return Err(Error::InvalidPacket("multistream frame durations differ"));
}
off += used;
}
}
_ => {
let dur = conceal_len.unwrap_or(match self.streams[0].last_packet_duration() {
0 => self.rate as usize / 50,
d => d,
});
for s in 0..n {
self.streams[s].conceal(dur, &mut self.scratch[s])?;
}
samples = Some(dur);
}
}
let samples = samples.unwrap_or(0);
let mut out = vec![vec![0f32; samples]; self.head.channels];
for (c, o) in out.iter_mut().enumerate() {
let m = self.head.mapping[c] as usize;
if m == 255 {
continue;
}
let (s, ch, nch) = if m < 2 * self.head.coupled_streams { (m / 2, m % 2, 2) } else { (m - self.head.coupled_streams, 0, 1) };
let src = &self.scratch[s];
for (i, v) in o.iter_mut().enumerate() {
*v = src[i * nch + ch];
}
}
if self.skip_remaining > 0 {
let k = self.skip_remaining.min(samples);
for o in &mut out {
o.drain(..k);
}
self.skip_remaining -= k;
}
Ok(out)
}
}
/// Rebuilds a regular (code 3 VBR) packet from a parsed self-delimited one.
fn reframe(p: &Packet) -> Vec<u8> {
let mut v = vec![(p.toc.config << 3) | if p.toc.stereo { 4 } else { 0 } | 3];
v.push(0x80 | p.frames.len() as u8);
for f in &p.frames[..p.frames.len() - 1] {
let s = f.len();
if s < 252 {
v.push(s as u8);
} else {
let b0 = 252 + (s & 3);
v.push(b0 as u8);
v.push(((s - b0) >> 2) as u8);
}
}
for f in &p.frames {
v.extend_from_slice(f);
}
v
}
crates/audio/src/opus/decode/packet.rs
+338
View File
@@ -0,0 +1,338 @@
//! TOC byte and frame packing (RFC 6716 §3).
use crate::opus::decode::Error;
/// Coding mode of an Opus frame.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Mode {
/// SILK only (TOC configs 0-11).
SilkOnly,
/// SILK and CELT (TOC configs 12-15).
Hybrid,
/// CELT only (TOC configs 16-31).
CeltOnly,
}
pub use crate::opus::Bandwidth;
impl Bandwidth {
/// Number of CELT bands coded for this bandwidth.
pub fn celt_end_band(self) -> usize {
match self {
Bandwidth::Narrowband => 13,
Bandwidth::Mediumband | Bandwidth::Wideband => 17,
Bandwidth::SuperWideband => 19,
Bandwidth::Fullband => 21,
}
}
}
/// Decoded table-of-contents byte.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Toc {
/// Configuration number (0..=31, RFC 6716 Table 2).
pub config: u8,
/// Coding mode, from `config`.
pub mode: Mode,
/// Audio bandwidth, from `config`.
pub bandwidth: Bandwidth,
/// Stereo flag (`s` bit).
pub stereo: bool,
/// Frame count code (0..=3).
pub code: u8,
}
impl Toc {
/// Decode a TOC byte (RFC 6716 §3.1).
pub fn parse(b: u8) -> Toc {
let config = b >> 3;
let (mode, bandwidth) = match config {
0..=3 => (Mode::SilkOnly, Bandwidth::Narrowband),
4..=7 => (Mode::SilkOnly, Bandwidth::Mediumband),
8..=11 => (Mode::SilkOnly, Bandwidth::Wideband),
12..=13 => (Mode::Hybrid, Bandwidth::SuperWideband),
14..=15 => (Mode::Hybrid, Bandwidth::Fullband),
16..=19 => (Mode::CeltOnly, Bandwidth::Narrowband),
20..=23 => (Mode::CeltOnly, Bandwidth::Wideband),
24..=27 => (Mode::CeltOnly, Bandwidth::SuperWideband),
_ => (Mode::CeltOnly, Bandwidth::Fullband),
};
Toc { config, mode, bandwidth, stereo: b & 4 != 0, code: b & 3 }
}
/// Samples per frame at 48 kHz.
pub fn frame_samples_48k(&self) -> usize {
self.frame_samples(48000)
}
/// Samples per frame at `rate`.
pub fn frame_samples(&self, rate: u32) -> usize {
let rate = rate as usize;
let c = self.config as usize;
match self.mode {
Mode::CeltOnly => (rate << (c & 3)) / 400,
Mode::Hybrid => {
if c & 1 != 0 {
rate / 50
} else {
rate / 100
}
}
Mode::SilkOnly => match c & 3 {
3 => rate * 60 / 1000,
s => (rate << s) / 100,
},
}
}
/// Channels the frames are coded with: 2 when `stereo`, else 1.
pub fn channels(&self) -> usize {
if self.stereo { 2 } else { 1 }
}
}
/// A parsed packet: TOC plus the byte ranges of each frame.
#[derive(Clone, Debug)]
pub struct Packet<'a> {
/// The packet's TOC byte.
pub toc: Toc,
/// Compressed data of each frame.
pub frames: Vec<&'a [u8]>,
/// Bytes of padding (code 3 only).
pub padding: usize,
}
fn parse_size(data: &[u8]) -> Option<(usize, usize)> {
match data.first() {
None => None,
Some(&b) if b < 252 => Some((b as usize, 1)),
Some(&b) => data.get(1).map(|&b1| (4 * b1 as usize + b as usize, 2)),
}
}
impl<'a> Packet<'a> {
/// Parses a packet (RFC 6716 section 3.2), enforcing requirements R1 to R7.
pub fn parse(data: &'a [u8]) -> Result<Packet<'a>, Error> {
Self::parse_impl(data, false).map(|(p, _)| p)
}
/// Parses a self-delimited packet (RFC 6716 Appendix B, used for all but the last stream of
/// a multistream packet); returns the packet and the number of bytes it occupies.
pub fn parse_self_delimited(data: &'a [u8]) -> Result<(Packet<'a>, usize), Error> {
Self::parse_impl(data, true)
}
fn parse_impl(data: &'a [u8], self_delimited: bool) -> Result<(Packet<'a>, usize), Error> {
const BAD: Error = Error::InvalidPacket("malformed packet framing");
if data.is_empty() {
return Err(Error::InvalidPacket("empty packet"));
}
let toc = Toc::parse(data[0]);
let fs = toc.frame_samples_48k();
let mut pos = 1usize;
let mut len = data.len() as isize - 1;
let mut sizes: Vec<isize> = Vec::new();
let mut padding = 0usize;
let mut cbr = false;
let count;
let mut last_size = len;
let read_size = |pos: &mut usize, len: &mut isize| -> Result<isize, Error> {
let avail = &data[*pos..(*pos + (*len).max(0) as usize).min(data.len())];
let (s, n) = parse_size(avail).ok_or(BAD)?;
*pos += n;
*len -= n as isize;
Ok(s as isize)
};
match toc.code {
0 => count = 1,
1 => {
count = 2;
cbr = true;
if !self_delimited {
if len & 1 != 0 {
return Err(BAD);
}
last_size = len / 2;
sizes.push(last_size);
}
}
2 => {
count = 2;
let s0 = read_size(&mut pos, &mut len)?;
if s0 > len {
return Err(BAD);
}
sizes.push(s0);
last_size = len - s0;
}
_ => {
if len < 1 {
return Err(BAD);
}
let ch = data[pos];
pos += 1;
len -= 1;
count = (ch & 0x3F) as usize;
if count == 0 || fs * count > 5760 {
return Err(BAD);
}
if ch & 0x40 != 0 {
loop {
if len <= 0 {
return Err(BAD);
}
let b = data[pos];
pos += 1;
len -= 1;
let tmp = if b == 255 { 254 } else { b as isize };
len -= tmp;
padding += tmp as usize;
if b != 255 {
break;
}
}
}
if len < 0 {
return Err(BAD);
}
cbr = ch & 0x80 == 0;
if !cbr {
last_size = len;
for _ in 0..count - 1 {
let before = len;
let s = read_size(&mut pos, &mut len)?;
if s > len {
return Err(BAD);
}
sizes.push(s);
last_size -= (before - len) + s;
}
if last_size < 0 {
return Err(BAD);
}
} else if !self_delimited {
last_size = len / count as isize;
if last_size * count as isize != len {
return Err(BAD);
}
for _ in 0..count - 1 {
sizes.push(last_size);
}
}
}
}
if self_delimited {
let before = len;
let s = read_size(&mut pos, &mut len)?;
if s > len {
return Err(BAD);
}
if cbr {
if s * count as isize > len {
return Err(BAD);
}
sizes = vec![s; count - 1];
} else if (before - len) + s > last_size {
return Err(BAD);
}
last_size = s;
}
if last_size > 1275 || sizes.iter().any(|&s| s > 1275) {
return Err(BAD);
}
sizes.push(last_size);
let mut frames = Vec::with_capacity(count);
for &s in &sizes {
let s = s as usize;
if pos + s > data.len() {
return Err(BAD);
}
frames.push(&data[pos..pos + s]);
pos += s;
}
let total = if self_delimited { pos + padding } else { data.len() };
if total > data.len() {
return Err(BAD);
}
Ok((Packet { toc, frames, padding }, total))
}
}
/// Number of samples (at 48 kHz) in a packet, or an error for malformed framing.
pub fn packet_samples_48k(data: &[u8]) -> Result<usize, Error> {
let p = Packet::parse(data)?;
Ok(p.toc.frame_samples_48k() * p.frames.len())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn toc_configs() {
for cfg in 0u8..32 {
let t = Toc::parse(cfg << 3);
let ms10 = t.frame_samples(48000) * 10 / 48; // tenths of ms
let expect = match cfg {
0..=11 => [100, 200, 400, 600][cfg as usize & 3],
12..=15 => [100, 200][cfg as usize & 1],
_ => [25, 50, 100, 200][cfg as usize & 3],
};
assert_eq!(ms10, expect, "config {cfg}");
}
assert!(Toc::parse(0x04).stereo);
}
#[test]
fn frame_codes() {
// code 0
let p = Packet::parse(&[0x00, 1, 2, 3]).unwrap();
assert_eq!(p.frames, vec![&[1u8, 2, 3][..]]);
// code 1 (odd payload is invalid)
assert!(Packet::parse(&[0x01, 1, 2, 3]).is_err());
let p = Packet::parse(&[0x01, 1, 2, 3, 4]).unwrap();
assert_eq!(p.frames.len(), 2);
// code 2
let p = Packet::parse(&[0x02, 1, 9, 8, 7]).unwrap();
assert_eq!(p.frames, vec![&[9u8][..], &[8u8, 7][..]]);
assert!(Packet::parse(&[0x02, 5, 1]).is_err());
// code 3 CBR with padding: 2 frames of 2 bytes, 3 bytes padding
let p = Packet::parse(&[0x03, 0x42, 3, 1, 2, 3, 4, 0, 0, 0]).unwrap();
assert_eq!(p.frames, vec![&[1u8, 2][..], &[3u8, 4][..]]);
assert_eq!(p.padding, 3);
// code 3 VBR
let p = Packet::parse(&[0x03, 0x83, 1, 2, 10, 20, 21, 30]).unwrap();
assert_eq!(p.frames, vec![&[10u8][..], &[20u8, 21][..], &[30u8][..]]);
// too many frames (> 120 ms)
assert!(Packet::parse(&[0x03 | (3 << 3), 0x03, 0]).is_err());
// zero frames
assert!(Packet::parse(&[0x03, 0x00]).is_err());
assert!(Packet::parse(&[]).is_err());
}
#[test]
fn two_byte_sizes() {
let mut pkt = vec![0x02u8, 252, 1];
pkt.extend(std::iter::repeat_n(7u8, 256));
pkt.extend([1, 2]);
let p = Packet::parse(&pkt).unwrap();
assert_eq!(p.frames[0].len(), 256);
assert_eq!(p.frames[1], &[1, 2]);
}
#[test]
fn self_delimited() {
// code 0 with explicit size, followed by trailing data of the next stream.
let (p, used) = Packet::parse_self_delimited(&[0x00, 2, 5, 6, 0xAA, 0xBB]).unwrap();
assert_eq!(p.frames, vec![&[5u8, 6][..]]);
assert_eq!(used, 4);
// code 1: one size for both frames.
let (p, used) = Packet::parse_self_delimited(&[0x01, 1, 5, 6, 9]).unwrap();
assert_eq!(p.frames, vec![&[5u8][..], &[6u8][..]]);
assert_eq!(used, 4);
// code 3 VBR, 2 frames: size of frame 0, then size of last frame.
let (p, used) = Packet::parse_self_delimited(&[0x03, 0x82, 1, 2, 7, 8, 9, 0xFF]).unwrap();
assert_eq!(p.frames, vec![&[7u8][..], &[8u8, 9][..]]);
assert_eq!(used, 7);
}
}
crates/audio/src/opus/decode/range.rs
+497
View File
@@ -0,0 +1,497 @@
//! Range decoder (RFC 6716 §4.1).
//!
//! The decoder reads range-coded symbols from the front of the buffer and raw bits from the back.
//! Reading past either end yields zeros (never panics); callers detect corruption through
//! [`RangeDecoder::tell`] exceeding the budget or through out-of-range values.
const SYM_BITS: u32 = 8;
const CODE_BITS: u32 = 32;
const SYM_MAX: u32 = (1 << SYM_BITS) - 1;
#[cfg(test)]
const CODE_SHIFT: u32 = CODE_BITS - SYM_BITS - 1;
const CODE_TOP: u32 = 1 << (CODE_BITS - 1);
const CODE_BOT: u32 = CODE_TOP >> SYM_BITS;
const CODE_EXTRA: u32 = (CODE_BITS - 2) % SYM_BITS + 1;
/// Bit resolution of [`RangeDecoder::tell_frac`] (1/8 bit).
pub const BITRES: u32 = 3;
/// Number of bits needed to represent `x` (0 for 0).
#[inline]
pub fn ilog(x: u32) -> i32 {
(32 - x.leading_zeros()) as i32
}
#[derive(Clone, Debug)]
pub struct RangeDecoder<'a> {
buf: &'a [u8],
/// Bytes available to the decoder (may be reduced, e.g. for redundancy).
storage: usize,
offs: usize,
end_offs: usize,
end_window: u32,
nend_bits: i32,
nbits_total: i32,
rng: u32,
val: u32,
rem: u32,
/// Set when an out-of-range value was decoded.
pub error: bool,
}
impl<'a> RangeDecoder<'a> {
pub fn new(buf: &'a [u8]) -> Self {
let mut d = RangeDecoder {
buf,
storage: buf.len(),
offs: 0,
end_offs: 0,
end_window: 0,
nend_bits: 0,
nbits_total: (CODE_BITS + 1 - ((CODE_BITS - CODE_EXTRA) / SYM_BITS) * SYM_BITS) as i32,
rng: 1 << CODE_EXTRA,
val: 0,
rem: 0,
error: false,
};
d.rem = d.read_byte();
d.val = d.rng - 1 - (d.rem >> (SYM_BITS - CODE_EXTRA));
d.normalize();
d
}
/// Size of the buffer in bytes as seen by the decoder.
pub fn storage(&self) -> usize {
self.storage
}
/// Shrinks the buffer seen by the decoder (raw bits are then read from the new end).
pub fn shrink_storage(&mut self, by: usize) {
self.storage = self.storage.saturating_sub(by);
}
pub fn rng(&self) -> u32 {
self.rng
}
#[inline]
fn read_byte(&mut self) -> u32 {
if self.offs < self.storage {
let b = self.buf[self.offs];
self.offs += 1;
b as u32
} else {
0
}
}
#[inline]
fn read_byte_from_end(&mut self) -> u32 {
if self.end_offs < self.storage {
self.end_offs += 1;
self.buf[self.storage - self.end_offs] as u32
} else {
0
}
}
#[inline]
fn normalize(&mut self) {
while self.rng <= CODE_BOT {
self.nbits_total += SYM_BITS as i32;
self.rng <<= SYM_BITS;
let sym = self.rem;
self.rem = self.read_byte();
let sym = ((sym << SYM_BITS) | self.rem) >> (SYM_BITS - CODE_EXTRA);
self.val = ((self.val << SYM_BITS).wrapping_add(SYM_MAX & !sym)) & (CODE_TOP - 1);
}
}
/// Returns the cumulative frequency of the next symbol for total `ft` (call [`Self::update`] next).
#[inline]
pub fn decode(&mut self, ft: u32) -> u32 {
let ext = self.rng / ft;
let s = self.val / ext;
// stash ext in rem-free form: recomputed by update()
ft - (s + 1).min(ft)
}
/// Like [`Self::decode`] with `ft = 1 << bits`.
#[inline]
pub fn decode_bin(&mut self, bits: u32) -> u32 {
let ext = self.rng >> bits;
let s = self.val / ext;
(1 << bits) - (s + 1).min(1 << bits)
}
/// Consumes the symbol with cumulative range `[fl, fh)` of total `ft`.
#[inline]
pub fn update(&mut self, fl: u32, fh: u32, ft: u32) {
let ext = self.rng / ft;
let s = ext * (ft - fh);
self.val = self.val.wrapping_sub(s);
self.rng = if fl > 0 { ext * (fh - fl) } else { self.rng - s };
self.normalize();
}
/// Like [`Self::update`] for a symbol decoded with [`Self::decode_bin`].
#[inline]
pub fn update_bin(&mut self, fl: u32, fh: u32, bits: u32) {
let ext = self.rng >> bits;
let ft = 1u32 << bits;
let s = ext * (ft - fh);
self.val = self.val.wrapping_sub(s);
self.rng = if fl > 0 { ext * (fh - fl) } else { self.rng - s };
self.normalize();
}
/// Decodes a binary symbol whose probability of being 1 is `1/2^logp`.
#[inline]
pub fn bit_logp(&mut self, logp: u32) -> bool {
let r = self.rng;
let d = self.val;
let s = r >> logp;
let ret = d < s;
if !ret {
self.val = d - s;
}
self.rng = if ret { s } else { r - s };
self.normalize();
ret
}
/// Decodes a symbol with an "inverse CDF" table (`icdf[i] = ft - cdf[i+1]`, `ft = 1 << ftb`).
#[inline]
pub fn icdf(&mut self, icdf: &[u8], ftb: u32) -> usize {
let mut s = self.rng;
let d = self.val;
let r = s >> ftb;
let mut ret = 0usize;
let mut t;
loop {
t = s;
let v = icdf.get(ret).copied().unwrap_or(0) as u32;
s = r * v;
if d >= s || ret + 1 >= icdf.len() {
break;
}
ret += 1;
}
// If the table is exhausted without d >= s, force the last symbol (only on corrupt tables).
if d < s {
s = 0;
}
self.val = d - s;
self.rng = t - s;
self.normalize();
ret
}
/// Decodes a uniformly distributed integer in `[0, ft)` (`ft >= 2`).
pub fn uint(&mut self, ft: u32) -> u32 {
debug_assert!(ft > 1);
let ft1 = ft - 1;
let ftb = ilog(ft1);
if ftb > 8 {
let ftb = (ftb - 8) as u32;
let top = (ft1 >> ftb) + 1;
let s = self.decode(top);
self.update(s, s + 1, top);
let t = (s << ftb) | self.bits(ftb);
if t <= ft1 {
return t;
}
self.error = true;
ft1
} else {
let s = self.decode(ft);
self.update(s, s + 1, ft);
s
}
}
/// Reads `bits` raw bits (0..=25) from the end of the buffer.
pub fn bits(&mut self, bits: u32) -> u32 {
if bits == 0 {
return 0;
}
let mut window = self.end_window;
let mut available = self.nend_bits;
if (available as u32) < bits {
loop {
window |= self.read_byte_from_end() << available;
available += SYM_BITS as i32;
if available > (CODE_BITS - SYM_BITS) as i32 {
break;
}
}
}
let ret = window & ((1u32 << bits) - 1);
window >>= bits;
available -= bits as i32;
self.end_window = window;
self.nend_bits = available;
self.nbits_total += bits as i32;
ret
}
/// Advances the bit counter so that [`Self::tell`] reports `total` (CELT silence frames).
pub fn skip_to_end(&mut self, total: i32) {
let t = self.tell();
self.nbits_total += total - t;
}
/// Number of bits consumed so far (rounded up).
#[inline]
pub fn tell(&self) -> i32 {
self.nbits_total - ilog(self.rng)
}
/// Number of 1/8 bits consumed so far (rounded up).
pub fn tell_frac(&self) -> u32 {
let nbits = (self.nbits_total as u32) << BITRES;
let mut l = ilog(self.rng) as u32;
let mut r = self.rng >> (l - 16);
for _ in 0..BITRES {
r = (r * r) >> 15;
let b = r >> 16;
l = (l << 1) | b;
r >>= b;
}
nbits.wrapping_sub(l)
}
}
#[cfg(test)]
pub(crate) mod enc {
//! Minimal range encoder (RFC 6716 §5.1), used only to build test streams.
use super::*;
pub struct RangeEncoder {
out: Vec<u8>,
end: Vec<u8>,
end_window: u32,
nend_bits: i32,
rng: u32,
low: u32,
rem: i32,
ext: u32,
size: usize,
}
impl RangeEncoder {
pub fn new(size: usize) -> Self {
RangeEncoder { out: Vec::new(), end: Vec::new(), end_window: 0, nend_bits: 0, rng: CODE_TOP, low: 0, rem: -1, ext: 0, size }
}
fn carry_out(&mut self, c: u32) {
if c != SYM_MAX {
let carry = c >> SYM_BITS;
if self.rem >= 0 {
self.out.push((self.rem as u32 + carry) as u8);
}
if self.ext > 0 {
let sym = (SYM_MAX + carry) & SYM_MAX;
for _ in 0..self.ext {
self.out.push(sym as u8);
}
self.ext = 0;
}
self.rem = (c & SYM_MAX) as i32;
} else {
self.ext += 1;
}
}
fn normalize(&mut self) {
while self.rng <= CODE_BOT {
self.carry_out(self.low >> CODE_SHIFT);
self.low = (self.low << SYM_BITS) & (CODE_TOP - 1);
self.rng <<= SYM_BITS;
}
}
pub fn encode(&mut self, fl: u32, fh: u32, ft: u32) {
let r = self.rng / ft;
if fl > 0 {
self.low += self.rng - r * (ft - fl);
self.rng = r * (fh - fl);
} else {
self.rng -= r * (ft - fh);
}
self.normalize();
}
pub fn bit_logp(&mut self, val: bool, logp: u32) {
let r = self.rng;
let s = r >> logp;
let r2 = r - s;
if val {
self.low += r2;
self.rng = s;
} else {
self.rng = r2;
}
self.normalize();
}
pub fn icdf(&mut self, s: usize, icdf: &[u8], ftb: u32) {
let r = self.rng >> ftb;
if s > 0 {
self.low += self.rng - r * icdf[s - 1] as u32;
self.rng = r * (icdf[s - 1] as u32 - icdf[s] as u32);
} else {
self.rng -= r * icdf[s] as u32;
}
self.normalize();
}
pub fn uint(&mut self, fl: u32, ft: u32) {
let ft1 = ft - 1;
let ftb = ilog(ft1);
if ftb > 8 {
let ftb = (ftb - 8) as u32;
let top = (ft1 >> ftb) + 1;
let v = fl >> ftb;
self.encode(v, v + 1, top);
self.bits(fl & ((1 << ftb) - 1), ftb);
} else {
self.encode(fl, fl + 1, ft);
}
}
pub fn bits(&mut self, val: u32, bits: u32) {
let mut window = self.end_window;
let mut used = self.nend_bits;
if used as u32 + bits > CODE_BITS {
while used >= SYM_BITS as i32 {
self.end.push((window & SYM_MAX) as u8);
window >>= SYM_BITS;
used -= SYM_BITS as i32;
}
}
window |= val << used;
used += bits as i32;
self.end_window = window;
self.nend_bits = used;
}
/// Finalises and returns a buffer of exactly `size` bytes.
pub fn done(mut self) -> Vec<u8> {
let mut l = CODE_BITS as i32 - ilog(self.rng);
let mut msk = (CODE_TOP - 1) >> l;
let mut end = (self.low + msk) & !msk;
if (end | msk) >= self.low + self.rng {
l += 1;
msk >>= 1;
end = (self.low + msk) & !msk;
}
while l > 0 {
self.carry_out(end >> CODE_SHIFT);
end = (end << SYM_BITS) & (CODE_TOP - 1);
l -= SYM_BITS as i32;
}
if self.rem >= 0 || self.ext > 0 {
self.carry_out(0);
}
let mut window = self.end_window;
let mut used = self.nend_bits;
while used > 0 {
self.end.push((window & SYM_MAX) as u8);
window >>= SYM_BITS;
used -= SYM_BITS as i32;
}
assert!(self.out.len() + self.end.len() <= self.size, "range encoder overflow");
let mut buf = self.out;
buf.resize(self.size - self.end.len(), 0);
for &b in self.end.iter().rev() {
buf.push(b);
}
buf
}
}
}
#[cfg(test)]
mod tests {
use super::enc::RangeEncoder;
use super::*;
struct Lcg(u32);
impl Lcg {
fn next(&mut self) -> u32 {
self.0 = self.0.wrapping_mul(1664525).wrapping_add(1013904223);
self.0 >> 8
}
}
#[test]
fn roundtrip_mixed_symbols() {
let mut rng = Lcg(1);
for trial in 0..200 {
let n = 1 + (rng.next() % 300) as usize;
let mut ops = Vec::new();
for _ in 0..n {
match rng.next() % 5 {
0 => {
let ft = 2 + rng.next() % 1000;
let v = rng.next() % ft;
ops.push((0u8, v, ft));
}
1 => {
let logp = 1 + rng.next() % 15;
ops.push((1, rng.next().is_multiple_of(7) as u32, logp));
}
2 => {
let bits = 1 + rng.next() % 24;
ops.push((2, rng.next() & ((1 << bits) - 1), bits));
}
3 => {
let ft = 2 + rng.next() % 100_000_000;
ops.push((3, rng.next() % ft, ft));
}
_ => ops.push((4, rng.next() % 4, 0)),
}
}
let icdf: [u8; 4] = [200, 120, 30, 0];
let mut e = RangeEncoder::new(8 * n + 16);
for &(k, v, p) in &ops {
match k {
0 => e.encode(v, v + 1, p),
1 => e.bit_logp(v != 0, p),
2 => e.bits(v, p),
3 => e.uint(v, p),
_ => e.icdf(v as usize, &icdf, 8),
}
}
let buf = e.done();
let mut d = RangeDecoder::new(&buf);
for (i, &(k, v, p)) in ops.iter().enumerate() {
let got = match k {
0 => {
let s = d.decode(p);
d.update(s, s + 1, p);
s
}
1 => d.bit_logp(p) as u32,
2 => d.bits(p),
3 => d.uint(p),
_ => d.icdf(&icdf, 8) as u32,
};
assert_eq!(got, v, "trial {trial} op {i} kind {k}");
}
assert!(!d.error);
}
}
#[test]
fn tell_starts_at_one_bit() {
let buf = [0u8; 8];
let d = RangeDecoder::new(&buf);
assert_eq!(d.tell(), 1);
assert_eq!(d.tell_frac(), 8);
}
#[test]
fn empty_and_garbage_never_panic() {
let mut d = RangeDecoder::new(&[]);
for i in 0..100 {
d.bits(1 + i % 20);
d.uint(3 + i * 77);
d.icdf(&[250, 100, 0], 8);
d.bit_logp(1 + i % 15);
}
let _ = d.tell_frac();
}
}
crates/audio/src/opus/decode/silk/lpc.rs
+377
View File
@@ -0,0 +1,377 @@
//! Fixed-point NLSF decoding, NLSF→LPC conversion and filter stabilisation
//! (RFC 6716 §4.2.7.5, with the RFC 8251 §6/§7 overflow fixes).
use super::tables::{
COS_Q12, LSF_ORDER_NBMB, LSF_ORDER_WB, NLSF_CB_NBMB, NLSF_CB_WB, NLSF_MIN_SPACING_NBMB, NLSF_MIN_SPACING_WB, NLSF_PRED_NBMB, NLSF_PRED_WB,
NLSF_PSEL_NBMB, NLSF_PSEL_WB,
};
pub const MAX_LPC_ORDER: usize = 16;
#[inline]
pub fn smulwb(a: i32, b: i32) -> i32 {
((a as i64 * (b as i16) as i64) >> 16) as i32
}
#[inline]
pub fn smlawb(a: i32, b: i32, c: i32) -> i32 {
a.wrapping_add(smulwb(b, c))
}
#[inline]
pub fn smulww(a: i32, b: i32) -> i32 {
((a as i64 * b as i64) >> 16) as i32
}
#[inline]
pub fn smmul(a: i32, b: i32) -> i32 {
((a as i64 * b as i64) >> 32) as i32
}
#[inline]
pub fn rshift_round(a: i32, s: u32) -> i32 {
if s == 1 { (a >> 1) + (a & 1) } else { ((a >> (s - 1)) + 1) >> 1 }
}
#[inline]
pub fn rshift_round64(a: i64, s: u32) -> i64 {
if s == 1 { (a >> 1) + (a & 1) } else { ((a >> (s - 1)) + 1) >> 1 }
}
#[inline]
pub fn sat16(a: i32) -> i32 {
a.clamp(-32768, 32767)
}
/// `silk_INVERSE32_varQ`: approximation of `(1 << q_res) / b32`.
pub fn inverse32_varq(b32: i32, q_res: i32) -> i32 {
let b_headrm = b32.unsigned_abs().leading_zeros() as i32 - 1;
let b32_nrm = b32.wrapping_shl(b_headrm as u32);
let b32_inv = (i32::MAX >> 2) / (b32_nrm >> 16);
let mut result = b32_inv << 16;
let err_q32 = ((1i32 << 29).wrapping_sub(smulwb(b32_nrm, b32_inv))).wrapping_shl(3);
result = result.wrapping_add(((err_q32 as i64 * b32_inv as i64) >> 16) as i32);
let lshift = 61 - b_headrm - q_res;
if lshift <= 0 {
let s = (-lshift) as u32;
let lim_hi = i32::MAX >> s;
let lim_lo = i32::MIN >> s;
result.clamp(lim_lo, lim_hi) << s
} else if lshift < 32 {
result >> lshift
} else {
0
}
}
/// `silk_DIV32_varQ`: approximation of `(a32 << q_res) / b32`.
pub fn div32_varq(a32: i32, b32: i32, q_res: i32) -> i32 {
let a_headrm = a32.unsigned_abs().leading_zeros() as i32 - 1;
let a32_nrm = a32.wrapping_shl(a_headrm as u32);
let b_headrm = b32.unsigned_abs().leading_zeros() as i32 - 1;
let b32_nrm = b32.wrapping_shl(b_headrm as u32);
let b32_inv = (i32::MAX >> 2) / (b32_nrm >> 16);
let mut result = smulwb(a32_nrm, b32_inv);
let a32_nrm = a32_nrm.wrapping_sub(((smmul(b32_nrm, result) as i64) << 3) as i32);
result = smlawb(result, a32_nrm, b32_inv);
let lshift = 29 + a_headrm - b_headrm - q_res;
if lshift < 0 {
let s = (-lshift) as u32;
let lim_hi = i32::MAX >> s;
let lim_lo = i32::MIN >> s;
result.clamp(lim_lo, lim_hi) << s
} else if lshift < 32 {
result >> lshift
} else {
0
}
}
fn sqrt_approx(x: i32) -> i32 {
if x <= 0 {
return 0;
}
let lz = x.leading_zeros() as i32;
let frac = (x.rotate_right((24 - lz).rem_euclid(32) as u32)) & 0x7f;
let mut y = if lz & 1 != 0 { 32768 } else { 46214 };
y >>= lz >> 1;
smlawb(y, y, 213 * frac)
}
/// Decodes the stage-1/stage-2 indices into stabilised NLSFs (Q15).
pub fn nlsf_decode(indices: &[i32], wb: bool) -> [i32; MAX_LPC_ORDER] {
let order = if wb { 16 } else { 10 };
let i1 = indices[0] as usize;
let cb1: &[i32] = if wb { &NLSF_CB_WB[i1] } else { &NLSF_CB_NBMB[i1] };
let qstep = if wb { 9830 } else { 11796 };
// Residual dequantisation with backwards prediction.
let mut res_q10 = [0i32; MAX_LPC_ORDER];
let mut out_q10 = 0i32;
for i in (0..order).rev() {
let pred = if i + 1 < order {
let sel = if wb { NLSF_PSEL_WB[i1][i] } else { NLSF_PSEL_NBMB[i1][i] } as usize;
let w = if wb { NLSF_PRED_WB[sel][i] } else { NLSF_PRED_NBMB[sel][i] };
(out_q10 * w) >> 8
} else {
0
};
let mut v = indices[i + 1] << 10;
if v > 0 {
v -= 102;
} else if v < 0 {
v += 102;
}
out_q10 = smlawb(pred, v, qstep);
res_q10[i] = out_q10;
}
// Laroia weights.
let mut nlsf = [0i32; MAX_LPC_ORDER];
for k in 0..order {
let prev = if k == 0 { 0 } else { cb1[k - 1] };
let next = if k + 1 == order { 256 } else { cb1[k + 1] };
let w = (1024 / (cb1[k] - prev).max(1) + 1024 / (next - cb1[k]).max(1)).min(32767);
let w_q9 = sqrt_approx(w << 16);
let v = (cb1[k] << 7) + (res_q10[k] << 14) / w_q9.max(1);
nlsf[k] = v.clamp(0, 32767);
}
let dmin: &[i32] = if wb { &NLSF_MIN_SPACING_WB } else { &NLSF_MIN_SPACING_NBMB };
nlsf_stabilize(&mut nlsf[..order], dmin);
nlsf
}
pub fn nlsf_stabilize(nlsf: &mut [i32], dmin: &[i32]) {
let l = nlsf.len();
for _ in 0..20 {
let mut min_diff = nlsf[0] - dmin[0];
let mut idx = 0;
for i in 1..l {
let d = nlsf[i] - (nlsf[i - 1] + dmin[i]);
if d < min_diff {
min_diff = d;
idx = i;
}
}
let d = 32768 - (nlsf[l - 1] + dmin[l]);
if d < min_diff {
min_diff = d;
idx = l;
}
if min_diff >= 0 {
return;
}
if idx == 0 {
nlsf[0] = dmin[0];
} else if idx == l {
nlsf[l - 1] = 32768 - dmin[l];
} else {
let mut min_center = dmin[..idx].iter().sum::<i32>();
min_center += dmin[idx] >> 1;
let mut max_center = 32768 - dmin[idx + 1..=l].iter().sum::<i32>();
max_center -= dmin[idx] >> 1;
let c = rshift_round(nlsf[idx - 1] + nlsf[idx], 1);
let c = if min_center > max_center { c.clamp(max_center, min_center) } else { c.clamp(min_center, max_center) };
nlsf[idx - 1] = c - (dmin[idx] >> 1);
nlsf[idx] = nlsf[idx - 1] + dmin[idx];
}
}
// Fallback.
nlsf.sort_unstable();
nlsf[0] = nlsf[0].max(dmin[0]);
for i in 1..l {
nlsf[i] = nlsf[i].max(sat16(nlsf[i - 1] + dmin[i]));
}
nlsf[l - 1] = nlsf[l - 1].min(32768 - dmin[l]);
for i in (0..l - 1).rev() {
nlsf[i] = nlsf[i].min(nlsf[i + 1] - dmin[i + 1]);
}
}
fn bwexpander_32(ar: &mut [i32], mut chirp_q16: i32) {
let d = ar.len();
let chirp_minus_one = chirp_q16 - 65536;
for v in ar.iter_mut().take(d - 1) {
*v = smulww(chirp_q16, *v);
chirp_q16 += rshift_round(chirp_q16.wrapping_mul(chirp_minus_one), 16);
}
ar[d - 1] = smulww(chirp_q16, ar[d - 1]);
}
/// `silk_bwexpander` on Q12 coefficients (16-bit).
pub fn bwexpander(ar: &mut [i32], mut chirp_q16: i32) {
let d = ar.len();
let chirp_minus_one = chirp_q16 - 65536;
for v in ar.iter_mut().take(d - 1) {
*v = rshift_round(chirp_q16.wrapping_mul(*v), 16);
chirp_q16 += rshift_round(chirp_q16.wrapping_mul(chirp_minus_one), 16);
}
ar[d - 1] = rshift_round(chirp_q16.wrapping_mul(ar[d - 1]), 16);
}
fn lpc_inverse_pred_gain(a_q12: &[i32]) -> i32 {
const QA: u32 = 24;
const A_LIMIT: i32 = 16_773_022;
let order = a_q12.len();
let mut a = [0i32; MAX_LPC_ORDER];
let mut dc = 0;
for k in 0..order {
dc += a_q12[k];
a[k] = a_q12[k] << (QA - 12);
}
if dc >= 4096 {
return 0;
}
let mut inv_gain: i32 = 1 << 30;
let mul32_frac_q31 = |x: i32, y: i32| rshift_round64(x as i64 * y as i64, 31) as i32;
for k in (1..order).rev() {
if a[k] > A_LIMIT || a[k] < -A_LIMIT {
return 0;
}
let rc_q31 = -(a[k] << (31 - QA));
let rc_mult1 = (1i32 << 30) - smmul(rc_q31, rc_q31);
inv_gain = smmul(inv_gain, rc_mult1) << 2;
if inv_gain < 107_374 {
return 0;
}
let mult2q = 32 - rc_mult1.unsigned_abs().leading_zeros() as i32;
let rc_mult2 = inverse32_varq(rc_mult1, mult2q + 30);
for n in 0..(k + 1) >> 1 {
let t1 = a[n];
let t2 = a[k - n - 1];
let v1 = rshift_round64(t1.saturating_sub(mul32_frac_q31(t2, rc_q31)) as i64 * rc_mult2 as i64, mult2q as u32);
if v1 > i32::MAX as i64 || v1 < i32::MIN as i64 {
return 0;
}
let v2 = rshift_round64(t2.saturating_sub(mul32_frac_q31(t1, rc_q31)) as i64 * rc_mult2 as i64, mult2q as u32);
if v2 > i32::MAX as i64 || v2 < i32::MIN as i64 {
return 0;
}
a[n] = v1 as i32;
a[k - n - 1] = v2 as i32;
}
}
if a[0] > A_LIMIT || a[0] < -A_LIMIT {
return 0;
}
let rc_q31 = -(a[0] << (31 - QA));
let rc_mult1 = (1i32 << 30) - smmul(rc_q31, rc_q31);
inv_gain = smmul(inv_gain, rc_mult1) << 2;
if inv_gain < 107_374 {
return 0;
}
inv_gain
}
/// Converts NLSFs (Q15) to stable Q12 LPC coefficients (`silk_NLSF2A`).
pub fn nlsf2a(nlsf: &[i32]) -> [i32; MAX_LPC_ORDER] {
let d = nlsf.len();
let ordering: &[usize] = if d == 16 { &LSF_ORDER_WB } else { &LSF_ORDER_NBMB };
let mut c_q17 = [0i32; MAX_LPC_ORDER];
for k in 0..d {
let i = (nlsf[k] >> 8) as usize;
let f = nlsf[k] & 255;
let c0 = COS_Q12[i];
let c1 = COS_Q12[(i + 1).min(128)];
c_q17[ordering[k]] = (c0 * 256 + (c1 - c0) * f + 4) >> 3;
}
let d2 = d / 2;
let find_poly = |off: usize| -> [i64; 10] {
let mut out = [0i64; 10];
out[0] = 1 << 16;
out[1] = -(c_q17[off] as i64);
for k in 1..d2 {
let ftmp = c_q17[2 * k + off] as i64;
out[k + 1] = 2 * out[k - 1] - ((ftmp * out[k] + 32768) >> 16);
for n in (2..=k).rev() {
out[n] += out[n - 2] - ((ftmp * out[n - 1] + 32768) >> 16);
}
out[1] -= ftmp;
}
out
};
let p = find_poly(0);
let q = find_poly(1);
let mut a32 = [0i32; MAX_LPC_ORDER];
for k in 0..d2 {
let ptmp = p[k + 1] + p[k];
let qtmp = q[k + 1] - q[k];
a32[k] = (-qtmp - ptmp) as i32;
a32[d - k - 1] = (qtmp - ptmp) as i32;
}
let a32 = &mut a32[..d];
// Limit the dynamic range (silk_LPC_fit with QIN = 17, QOUT = 12).
let mut a_q12 = [0i32; MAX_LPC_ORDER];
let mut i = 0;
while i < 10 {
let mut maxabs = 0i32;
let mut idx = 0;
for (k, &v) in a32.iter().enumerate() {
let av = v.saturating_abs();
if av > maxabs {
maxabs = av;
idx = k;
}
}
let maxabs = rshift_round(maxabs, 5);
if maxabs > 32767 {
let maxabs = maxabs.min(163_838);
let chirp = 65470 - ((maxabs - 32767) << 14) / ((maxabs * (idx as i32 + 1)) >> 2);
bwexpander_32(a32, chirp);
} else {
break;
}
i += 1;
}
if i == 10 {
for k in 0..d {
a_q12[k] = sat16(rshift_round(a32[k], 5));
a32[k] = a_q12[k] << 5;
}
} else {
for k in 0..d {
a_q12[k] = rshift_round(a32[k], 5);
}
}
for i in 0..16 {
if lpc_inverse_pred_gain(&a_q12[..d]) == 0 {
bwexpander_32(a32, 65536 - (2 << i));
for k in 0..d {
a_q12[k] = rshift_round(a32[k], 5);
}
} else {
break;
}
}
a_q12
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn flat_nlsf_gives_small_lpc() {
// Uniformly spaced NLSFs correspond to an (almost) flat spectrum.
let nlsf: Vec<i32> = (1..=16).map(|k| k * 32768 / 17).collect();
let a = nlsf2a(&nlsf);
assert!(a.iter().all(|v| v.abs() < 600), "{a:?}");
let nlsf: Vec<i32> = (1..=10).map(|k| k * 32768 / 11).collect();
let a = nlsf2a(&nlsf);
assert!(a[..10].iter().all(|v| v.abs() < 600), "{a:?}");
}
#[test]
fn stabilize_enforces_spacing() {
let mut v = [100, 90, 80, 30000, 30001, 30002, 31000, 32000, 32700, 32760];
nlsf_stabilize(&mut v, &NLSF_MIN_SPACING_NBMB);
assert!(v[0] >= NLSF_MIN_SPACING_NBMB[0]);
for i in 1..10 {
assert!(v[i] - v[i - 1] >= NLSF_MIN_SPACING_NBMB[i], "{v:?}");
}
assert!(32768 - v[9] >= NLSF_MIN_SPACING_NBMB[10]);
}
#[test]
fn inverse_approximations() {
for &b in &[65536, 100_000, 1 << 20, 81920, 1_686_110_208] {
let inv = inverse32_varq(b, 47) as f64;
let exact = 2f64.powi(47) / b as f64;
assert!((inv - exact).abs() / exact < 1e-3, "{b}");
}
let q = div32_varq(65536, 131072, 16) as f64;
assert!((q - 32768.0).abs() < 2.0);
}
}
crates/audio/src/opus/decode/silk/mod.rs
+890
View File
@@ -0,0 +1,890 @@
//! SILK layer decoder (RFC 6716 §4.2).
//!
//! Parameter decoding and reconstruction follow the normative fixed-point arithmetic so that the
//! output matches the reference decoder sample for sample at the internal rate. Resampling to the
//! output rate uses our own linear-phase interpolator with the delays of RFC 6716 Table 54.
pub(crate) mod lpc;
mod resampler;
#[allow(dead_code)]
pub(crate) mod tables;
use lpc::{MAX_LPC_ORDER, bwexpander, div32_varq, inverse32_varq, nlsf_decode, nlsf2a, rshift_round, sat16, smlawb, smulwb, smulww};
use resampler::Resampler;
use tables::{
CB_NB_10, CB_NB_20, CB_WB_10, CB_WB_20, DELTA_GAIN, EXC_LSB, EXC_SIGN, GAIN_MSB, LBRR_FLAGS, LTP_FILTER, LTP_PERIODICITY, LTP_SCALE, LTP_TAPS_0,
LTP_TAPS_1, LTP_TAPS_2, MID_ONLY, NLSF_EXT, NLSF_INTERP, NLSF_SEL_NBMB, NLSF_SEL_WB, NLSF_STAGE1, NLSF_STAGE2, PITCH_CONTOUR, PITCH_DELTA,
PITCH_HIGH, PITCH_LOW, PULSE_COUNT, RATE_LEVEL, SHELL_2, SHELL_4, SHELL_8, SHELL_16, STEREO_PRED_JOINT, STEREO_WEIGHTS_Q13, TYPE_ACTIVE,
TYPE_INACTIVE, UNIFORM3, UNIFORM4, UNIFORM5, UNIFORM8,
};
use crate::opus::decode::range::RangeDecoder;
use crate::opus::decode::{Error, Result};
const MAX_FRAME: usize = 320;
const MAX_SUBFR: usize = 80;
const LTP_ORDER: usize = 5;
const OUT_BUF: usize = MAX_FRAME + 2 * MAX_SUBFR;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum LostFlag {
Normal,
Lost,
Lbrr,
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum Cond {
Independently,
IndependentlyNoLtpScaling,
Conditionally,
}
#[derive(Clone, Default)]
struct Indices {
gains: [i32; 4],
ltp_index: [usize; 4],
nlsf: [i32; 17],
lag_index: i32,
contour_index: usize,
signal_type: usize,
quant_offset_type: usize,
nlsf_interp_q2: i32,
per_index: usize,
ltp_scale_index: usize,
seed: i32,
}
#[derive(Clone)]
struct Channel {
fs_khz: usize,
nb_subfr: usize,
frame_length: usize,
subfr_length: usize,
ltp_mem_length: usize,
lpc_order: usize,
prev_gain_q16: i32,
s_lpc_q14: [i32; MAX_LPC_ORDER],
out_buf: [i16; OUT_BUF],
lag_prev: i32,
last_gain_index: i32,
prev_signal_type: usize,
first_frame_after_reset: bool,
prev_nlsf_q15: [i32; MAX_LPC_ORDER],
ec_prev_signal_type: usize,
ec_prev_lag_index: i32,
vad_flags: [bool; 3],
lbrr_flag: bool,
lbrr_flags: [bool; 3],
n_frames_decoded: usize,
n_frames_per_packet: usize,
loss_cnt: u32,
indices: Indices,
resampler: Resampler,
// PLC memory.
plc_lpc_q12: [i32; MAX_LPC_ORDER],
plc_gain_q16: i32,
plc_pitch: i32,
plc_rand: u32,
}
impl Channel {
fn new() -> Channel {
Channel {
fs_khz: 0,
nb_subfr: 4,
frame_length: 0,
subfr_length: 0,
ltp_mem_length: 0,
lpc_order: 0,
prev_gain_q16: 65536,
s_lpc_q14: [0; MAX_LPC_ORDER],
out_buf: [0; OUT_BUF],
lag_prev: 100,
last_gain_index: 10,
prev_signal_type: 0,
first_frame_after_reset: true,
prev_nlsf_q15: [0; MAX_LPC_ORDER],
ec_prev_signal_type: 0,
ec_prev_lag_index: 0,
vad_flags: [false; 3],
lbrr_flag: false,
lbrr_flags: [false; 3],
n_frames_decoded: 0,
n_frames_per_packet: 1,
loss_cnt: 0,
indices: Indices::default(),
resampler: Resampler::new(16000, 16000),
plc_lpc_q12: [0; MAX_LPC_ORDER],
plc_gain_q16: 0,
plc_pitch: 0,
plc_rand: 22222,
}
}
fn set_fs(&mut self, fs_khz: usize, api_rate: u32) {
self.subfr_length = 5 * fs_khz;
let frame_length = self.nb_subfr * self.subfr_length;
if self.fs_khz != fs_khz || self.resampler.rates() != (fs_khz as u32 * 1000, api_rate) {
self.resampler = Resampler::new(fs_khz as u32 * 1000, api_rate);
}
if self.fs_khz != fs_khz || frame_length != self.frame_length {
if self.fs_khz != fs_khz {
self.ltp_mem_length = 20 * fs_khz;
self.lpc_order = if fs_khz == 16 { 16 } else { 10 };
self.first_frame_after_reset = true;
self.lag_prev = 100;
self.last_gain_index = 10;
self.prev_signal_type = 0;
self.out_buf = [0; OUT_BUF];
self.s_lpc_q14 = [0; MAX_LPC_ORDER];
}
self.fs_khz = fs_khz;
self.frame_length = frame_length;
}
}
fn decode_indices(&mut self, dec: &mut RangeDecoder, frame_index: usize, decode_lbrr: bool, cond: Cond) {
let ix = if decode_lbrr || self.vad_flags[frame_index] { dec.icdf(&TYPE_ACTIVE, 8) + 2 } else { dec.icdf(&TYPE_INACTIVE, 8) };
let ind = &mut self.indices;
ind.signal_type = ix >> 1;
ind.quant_offset_type = ix & 1;
if cond == Cond::Conditionally {
ind.gains[0] = dec.icdf(&DELTA_GAIN, 8) as i32;
} else {
ind.gains[0] = (dec.icdf(GAIN_MSB[ind.signal_type], 8) as i32) << 3;
ind.gains[0] += dec.icdf(&UNIFORM8, 8) as i32;
}
for i in 1..self.nb_subfr {
ind.gains[i] = dec.icdf(&DELTA_GAIN, 8) as i32;
}
let wb = self.lpc_order == 16;
let s1 = (ind.signal_type >> 1) + if wb { 2 } else { 0 };
let i1 = dec.icdf(NLSF_STAGE1[s1], 8);
ind.nlsf[0] = i1 as i32;
for i in 0..self.lpc_order {
let cb = if wb { 8 + NLSF_SEL_WB[i1][i] as usize } else { NLSF_SEL_NBMB[i1][i] as usize };
let mut v = dec.icdf(NLSF_STAGE2[cb], 8) as i32;
if v == 0 {
v -= dec.icdf(&NLSF_EXT, 8) as i32;
} else if v == 8 {
v += dec.icdf(&NLSF_EXT, 8) as i32;
}
ind.nlsf[i + 1] = v - 4;
}
ind.nlsf_interp_q2 = if self.nb_subfr == 4 { dec.icdf(&NLSF_INTERP, 8) as i32 } else { 4 };
if ind.signal_type == 2 {
let mut absolute = true;
if cond == Cond::Conditionally && self.ec_prev_signal_type == 2 {
let d = dec.icdf(&PITCH_DELTA, 8) as i32;
if d > 0 {
ind.lag_index = self.ec_prev_lag_index + d - 9;
absolute = false;
}
}
if absolute {
ind.lag_index = dec.icdf(&PITCH_HIGH, 8) as i32 * (self.fs_khz as i32 >> 1);
let low = match self.fs_khz {
8 => PITCH_LOW[0],
12 => PITCH_LOW[1],
_ => PITCH_LOW[2],
};
ind.lag_index += dec.icdf(low, 8) as i32;
}
self.ec_prev_lag_index = ind.lag_index;
let contour = match (self.fs_khz, self.nb_subfr) {
(8, 2) => PITCH_CONTOUR[0],
(8, _) => PITCH_CONTOUR[1],
(_, 2) => PITCH_CONTOUR[2],
_ => PITCH_CONTOUR[3],
};
ind.contour_index = dec.icdf(contour, 8);
ind.per_index = dec.icdf(&LTP_PERIODICITY, 8);
for k in 0..self.nb_subfr {
ind.ltp_index[k] = dec.icdf(LTP_FILTER[ind.per_index], 8);
}
ind.ltp_scale_index = if cond == Cond::Independently { dec.icdf(&LTP_SCALE, 8) } else { 0 };
}
self.ec_prev_signal_type = ind.signal_type;
ind.seed = dec.icdf(&UNIFORM4, 8) as i32;
}
fn decode_frame(&mut self, dec: Option<&mut RangeDecoder>, out: &mut [i16], lost: LostFlag, cond: Cond) {
let l = self.frame_length;
let decode = match (lost, &dec) {
(LostFlag::Normal, Some(_)) => true,
(LostFlag::Lbrr, Some(_)) => self.lbrr_flags[self.n_frames_decoded],
_ => false,
};
if decode && let Some(dec) = dec {
self.decode_indices(dec, self.n_frames_decoded, lost == LostFlag::Lbrr, cond);
let mut pulses = [0i32; MAX_FRAME + 16];
decode_pulses(dec, &mut pulses, self.indices.signal_type, self.indices.quant_offset_type, l);
let ctrl = self.decode_parameters(cond);
self.decode_core(&ctrl, &pulses, out);
self.plc_lpc_q12 = ctrl.pred_coef_q12[1];
self.plc_gain_q16 = ctrl.gains_q16[self.nb_subfr - 1];
self.plc_pitch = if self.indices.signal_type == 2 { ctrl.pitch_l[self.nb_subfr - 1] } else { 0 };
self.loss_cnt = 0;
self.prev_signal_type = self.indices.signal_type;
self.first_frame_after_reset = false;
self.lag_prev = ctrl.pitch_l[self.nb_subfr - 1];
} else {
self.conceal(out);
}
// Update the output buffer.
let mv = self.ltp_mem_length - l;
self.out_buf.copy_within(l..l + mv, 0);
self.out_buf[mv..mv + l].copy_from_slice(&out[..l]);
}
fn decode_parameters(&mut self, cond: Cond) -> Ctrl {
let mut ctrl = Ctrl::default();
let nb = self.nb_subfr;
// Gains.
let mut prev = self.last_gain_index;
for k in 0..nb {
let ind = self.indices.gains[k];
if k == 0 && cond != Cond::Conditionally {
prev = ind.max(prev - 16);
} else {
let tmp = ind - 4;
let thr = 2 * 36 - 64 + prev;
if tmp > thr {
prev += (tmp << 1) - thr;
} else {
prev += tmp;
}
}
prev = prev.clamp(0, 63);
ctrl.gains_q16[k] = log2lin((smulwb(0x1D1C71, prev) + 2090).min(3967));
}
self.last_gain_index = prev;
// LPC.
let order = self.lpc_order;
let nlsf = nlsf_decode(&self.indices.nlsf, order == 16);
ctrl.pred_coef_q12[1] = nlsf2a(&nlsf[..order]);
if self.first_frame_after_reset {
self.indices.nlsf_interp_q2 = 4;
}
if self.indices.nlsf_interp_q2 < 4 {
let mut n0 = [0i32; MAX_LPC_ORDER];
for i in 0..order {
n0[i] = self.prev_nlsf_q15[i] + ((self.indices.nlsf_interp_q2 * (nlsf[i] - self.prev_nlsf_q15[i])) >> 2);
}
ctrl.pred_coef_q12[0] = nlsf2a(&n0[..order]);
} else {
ctrl.pred_coef_q12[0] = ctrl.pred_coef_q12[1];
}
self.prev_nlsf_q15 = nlsf;
if self.loss_cnt > 0 {
bwexpander(&mut ctrl.pred_coef_q12[0][..order], 63570);
bwexpander(&mut ctrl.pred_coef_q12[1][..order], 63570);
}
if self.indices.signal_type == 2 {
let fs = self.fs_khz as i32;
let min_lag = 2 * fs;
let max_lag = 18 * fs;
let lag = min_lag + self.indices.lag_index;
for k in 0..nb {
let off = match (self.fs_khz, nb) {
(8, 2) => CB_NB_10[self.indices.contour_index][k],
(8, _) => CB_NB_20[self.indices.contour_index][k],
(_, 2) => CB_WB_10[self.indices.contour_index][k],
_ => CB_WB_20[self.indices.contour_index][k],
};
ctrl.pitch_l[k] = (lag + off).clamp(min_lag, max_lag);
}
for k in 0..nb {
let ix = self.indices.ltp_index[k];
let taps = match self.indices.per_index {
0 => LTP_TAPS_0[ix.min(7)],
1 => LTP_TAPS_1[ix.min(15)],
_ => LTP_TAPS_2[ix.min(31)],
};
for i in 0..LTP_ORDER {
ctrl.ltp_coef_q14[k * LTP_ORDER + i] = taps[i] << 7;
}
}
ctrl.ltp_scale_q14 = [15565, 12288, 8192][self.indices.ltp_scale_index];
}
ctrl
}
fn decode_core(&mut self, ctrl: &Ctrl, pulses: &[i32], xq: &mut [i16]) {
let ind = &self.indices;
let l = self.frame_length;
let sub = self.subfr_length;
let order = self.lpc_order;
let ltp_mem = self.ltp_mem_length;
let mut s_ltp = [0i16; MAX_FRAME];
let mut s_ltp_q15 = [0i32; 2 * MAX_FRAME];
let mut res_q14 = [0i32; MAX_SUBFR];
let mut s_lpc = [0i32; MAX_SUBFR + MAX_LPC_ORDER];
let offset_q10 = [[100, 240], [32, 100]][ind.signal_type >> 1][ind.quant_offset_type];
let interp = ind.nlsf_interp_q2 < 4;
let mut exc = [0i32; MAX_FRAME];
let mut seed = ind.seed;
for i in 0..l {
seed = seed.wrapping_mul(196_314_165).wrapping_add(907_633_515);
let mut e = pulses[i] << 14;
if e > 0 {
e -= 80 << 4;
} else if e < 0 {
e += 80 << 4;
}
e += offset_q10 << 4;
if seed < 0 {
e = -e;
}
exc[i] = e;
seed = seed.wrapping_add(pulses[i]);
}
s_lpc[..MAX_LPC_ORDER].copy_from_slice(&self.s_lpc_q14);
let mut ltp_buf_idx = ltp_mem;
for k in 0..self.nb_subfr {
let a_q12 = &ctrl.pred_coef_q12[k >> 1];
let mut b_q14 = [0i32; LTP_ORDER];
b_q14.copy_from_slice(&ctrl.ltp_coef_q14[k * LTP_ORDER..(k + 1) * LTP_ORDER]);
let mut signal_type = ind.signal_type;
let gain_q10 = ctrl.gains_q16[k] >> 6;
let mut inv_gain_q31 = inverse32_varq(ctrl.gains_q16[k], 47);
let gain_adj_q16 = if ctrl.gains_q16[k] != self.prev_gain_q16 {
let g = div32_varq(self.prev_gain_q16, ctrl.gains_q16[k], 16);
for v in s_lpc[..MAX_LPC_ORDER].iter_mut() {
*v = smulww(g, *v);
}
g
} else {
1 << 16
};
self.prev_gain_q16 = ctrl.gains_q16[k];
let mut lag = ctrl.pitch_l[k];
if self.loss_cnt > 0 && self.prev_signal_type == 2 && signal_type != 2 && k < 2 {
b_q14 = [0; LTP_ORDER];
b_q14[LTP_ORDER / 2] = 1 << 12;
signal_type = 2;
lag = self.lag_prev;
}
if signal_type == 2 {
let lag_u = lag as usize;
if k == 0 || (k == 2 && interp) {
let start_idx = ltp_mem as isize - lag as isize - order as isize - (LTP_ORDER / 2) as isize;
let start_idx = start_idx.max(0) as usize;
if k == 2 {
self.out_buf[ltp_mem..ltp_mem + 2 * sub].copy_from_slice(&xq[..2 * sub]);
}
let src_off = start_idx + k * sub;
lpc_analysis_filter(&mut s_ltp[start_idx..ltp_mem], &self.out_buf[src_off..src_off + ltp_mem - start_idx], a_q12, order);
if k == 0 {
inv_gain_q31 = smulwb(inv_gain_q31, ctrl.ltp_scale_q14) << 2;
}
for i in 0..(lag_u + LTP_ORDER / 2).min(ltp_mem) {
s_ltp_q15[ltp_buf_idx - i - 1] = smulwb(inv_gain_q31, s_ltp[ltp_mem - i - 1] as i32);
}
} else if gain_adj_q16 != 1 << 16 {
for i in 0..(lag_u + LTP_ORDER / 2).min(ltp_buf_idx) {
let p = ltp_buf_idx - i - 1;
s_ltp_q15[p] = smulww(gain_adj_q16, s_ltp_q15[p]);
}
}
let base = ltp_buf_idx as isize - lag as isize + (LTP_ORDER / 2) as isize;
for i in 0..sub {
let p = base + i as isize;
let g = |o: isize| -> i32 { if p - o >= 0 { s_ltp_q15[(p - o) as usize] } else { 0 } };
let mut pred = 2i32;
pred = smlawb(pred, g(0), b_q14[0]);
pred = smlawb(pred, g(1), b_q14[1]);
pred = smlawb(pred, g(2), b_q14[2]);
pred = smlawb(pred, g(3), b_q14[3]);
pred = smlawb(pred, g(4), b_q14[4]);
res_q14[i] = exc[k * sub + i].wrapping_add(pred << 1);
s_ltp_q15[ltp_buf_idx] = res_q14[i] << 1;
ltp_buf_idx += 1;
}
} else {
res_q14[..sub].copy_from_slice(&exc[k * sub..(k + 1) * sub]);
}
for i in 0..sub {
let mut pred = (order >> 1) as i32;
for j in 0..order {
pred = smlawb(pred, s_lpc[MAX_LPC_ORDER + i - 1 - j], a_q12[j]);
}
let v = res_q14[i].saturating_add(pred.saturating_mul(16));
s_lpc[MAX_LPC_ORDER + i] = v;
xq[k * sub + i] = sat16(rshift_round(smulww(v, gain_q10), 8)) as i16;
}
s_lpc.copy_within(sub..sub + MAX_LPC_ORDER, 0);
}
self.s_lpc_q14.copy_from_slice(&s_lpc[..MAX_LPC_ORDER]);
}
/// Basic concealment: pitch-periodic repetition for voiced history, otherwise LPC-shaped
/// noise at the residual level of the history; both fade out over successive losses.
fn conceal(&mut self, out: &mut [i16]) {
let l = self.frame_length;
if l == 0 {
return;
}
self.loss_cnt += 1;
let order = self.lpc_order;
let mem = self.ltp_mem_length;
let fade = 0.85f32.powi(self.loss_cnt as i32);
let hist: Vec<f32> = self.out_buf[..mem].iter().map(|&v| v as f32).collect();
let pitch = self.plc_pitch as usize;
let mut y = hist.clone();
if pitch > 0 && pitch < mem {
for _ in 0..l {
let v = y[y.len() - pitch] * fade;
y.push(v);
}
} else {
let mut a12 = self.plc_lpc_q12;
bwexpander(&mut a12[..order], 64880);
self.plc_lpc_q12 = a12;
let a: Vec<f32> = a12[..order].iter().map(|&v| v as f32 / 4096.0).collect();
let mut e = 0f32;
for i in order..mem {
let mut r = hist[i];
for j in 0..order {
r -= a[j] * hist[i - 1 - j];
}
e += r * r;
}
let rms = (e / (mem - order).max(1) as f32).sqrt() * fade;
for _ in 0..l {
self.plc_rand = self.plc_rand.wrapping_mul(196_314_165).wrapping_add(907_633_515);
let mut v = ((self.plc_rand as i32) >> 16) as f32 / 32768.0 * rms * 1.7;
for j in 0..order {
v += a[j] * y[y.len() - 1 - j];
}
y.push(v.clamp(-32768.0, 32767.0));
}
}
for (o, v) in out.iter_mut().zip(&y[mem..mem + l]) {
*o = v.clamp(-32768.0, 32767.0) as i16;
}
// Keep the LPC synthesis state consistent for the next decoded frame.
let g = self.prev_gain_q16.max(1) as i64;
for j in 0..MAX_LPC_ORDER {
let v = out[l - MAX_LPC_ORDER + j] as i64;
self.s_lpc_q14[j] = ((v << 30) / g).clamp(i32::MIN as i64, i32::MAX as i64) as i32;
}
self.lag_prev = self.plc_pitch.max(1);
self.first_frame_after_reset = false;
}
}
#[derive(Default)]
struct Ctrl {
gains_q16: [i32; 4],
pred_coef_q12: [[i32; MAX_LPC_ORDER]; 2],
pitch_l: [i32; 4],
ltp_coef_q14: [i32; 4 * LTP_ORDER],
ltp_scale_q14: i32,
}
fn log2lin(in_log_q7: i32) -> i32 {
if in_log_q7 < 0 {
return 0;
}
if in_log_q7 >= 3967 {
return i32::MAX;
}
let out = 1i32 << (in_log_q7 >> 7);
let frac = in_log_q7 & 0x7F;
if in_log_q7 < 2048 {
out + ((out * smlawb(frac, frac * (128 - frac), -174)) >> 7)
} else {
out + (out >> 7) * smlawb(frac, frac * (128 - frac), -174)
}
}
fn lpc_analysis_filter(out: &mut [i16], input: &[i16], b: &[i32], d: usize) {
let len = out.len();
for ix in d..len {
let mut acc: i32 = 0;
for j in 0..d {
acc = acc.wrapping_add((input[ix - 1 - j] as i32).wrapping_mul(b[j]));
}
let v = ((input[ix] as i32) << 12).wrapping_sub(acc);
out[ix] = sat16(rshift_round(v, 12)) as i16;
}
for v in out.iter_mut().take(d.min(len)) {
*v = 0;
}
}
fn decode_pulses(dec: &mut RangeDecoder, pulses: &mut [i32], signal_type: usize, quant_offset_type: usize, frame_length: usize) {
let rate_level = dec.icdf(RATE_LEVEL[signal_type >> 1], 8);
let mut iter = frame_length >> 4;
if iter * 16 < frame_length {
iter += 1;
}
let mut sum_pulses = [0i32; 20];
let mut n_lshifts = [0i32; 20];
for i in 0..iter {
let mut s = dec.icdf(PULSE_COUNT[rate_level], 8) as i32;
while s == 17 {
n_lshifts[i] += 1;
let tab = if n_lshifts[i] == 10 { PULSE_COUNT[10] } else { PULSE_COUNT[9] };
s = dec.icdf(tab, 8) as i32;
}
sum_pulses[i] = s;
}
for i in 0..iter {
let blk = &mut pulses[i * 16..(i + 1) * 16];
if sum_pulses[i] > 0 {
shell_decode(dec, blk, sum_pulses[i]);
} else {
blk.fill(0);
}
}
for i in 0..iter {
let nls = n_lshifts[i];
if nls > 0 {
for p in pulses[i * 16..(i + 1) * 16].iter_mut() {
let mut a = *p;
for _ in 0..nls {
a = (a << 1) + dec.icdf(&EXC_LSB, 8) as i32;
}
*p = a;
}
sum_pulses[i] |= nls << 5;
}
}
// Signs.
let base = 7 * (quant_offset_type + (signal_type << 1));
let len = (frame_length + 8) >> 4;
for i in 0..len {
let p = sum_pulses[i];
if p > 0 {
let tab = EXC_SIGN[base + ((p & 0x1F) as usize).min(6)];
for q in pulses[i * 16..(i + 1) * 16].iter_mut() {
if *q > 0 && dec.icdf(tab, 8) == 0 {
*q = -*q;
}
}
}
}
}
fn shell_decode(dec: &mut RangeDecoder, out: &mut [i32], total: i32) {
fn split(dec: &mut RangeDecoder, out: &mut [i32], n: usize, count: i32) {
if n == 1 {
out[0] = count;
return;
}
if count == 0 {
out[..n].fill(0);
return;
}
let tab = match n {
16 => SHELL_16[(count - 1) as usize],
8 => SHELL_8[(count - 1) as usize],
4 => SHELL_4[(count - 1) as usize],
_ => SHELL_2[(count - 1) as usize],
};
let left = dec.icdf(tab, 8) as i32;
let h = n / 2;
let (a, b) = out.split_at_mut(h);
split(dec, a, h, left);
split(dec, b, h, count - left);
}
split(dec, out, 16, total.min(16));
}
#[derive(Default, Clone)]
struct Stereo {
pred_prev_q13: [i32; 2],
s_mid: [i16; 2],
s_side: [i16; 2],
}
fn stereo_decode_pred(dec: &mut RangeDecoder) -> [i32; 2] {
let n = dec.icdf(&STEREO_PRED_JOINT, 8);
let mut ix = [[0usize; 3]; 2];
ix[0][2] = n / 5;
ix[1][2] = n - 5 * ix[0][2];
for v in ix.iter_mut() {
v[0] = dec.icdf(&UNIFORM3, 8);
v[1] = dec.icdf(&UNIFORM5, 8);
}
let mut pred = [0i32; 2];
for (p, v) in pred.iter_mut().zip(ix.iter_mut()) {
v[0] += 3 * v[2];
let low = STEREO_WEIGHTS_Q13[v[0]];
let step = smulwb(STEREO_WEIGHTS_Q13[v[0] + 1] - low, 6554);
*p = low + step * (2 * v[1] as i32 + 1);
}
pred[0] -= pred[1];
pred
}
fn stereo_ms_to_lr(st: &mut Stereo, x1: &mut [i16], x2: &mut [i16], pred_q13: [i32; 2], fs_khz: usize, len: usize) {
x1[..2].copy_from_slice(&st.s_mid);
x2[..2].copy_from_slice(&st.s_side);
st.s_mid.copy_from_slice(&x1[len..len + 2]);
st.s_side.copy_from_slice(&x2[len..len + 2]);
let mut p0 = st.pred_prev_q13[0];
let mut p1 = st.pred_prev_q13[1];
let denom_q16 = (1 << 16) / (8 * fs_khz as i32);
let d0 = rshift_round((pred_q13[0] - st.pred_prev_q13[0]) as i16 as i32 * denom_q16 as i16 as i32, 16);
let d1 = rshift_round((pred_q13[1] - st.pred_prev_q13[1]) as i16 as i32 * denom_q16 as i16 as i32, 16);
let interp = 8 * fs_khz;
for n in 0..len {
if n < interp {
p0 += d0;
p1 += d1;
} else if n == interp {
p0 = pred_q13[0];
p1 = pred_q13[1];
}
let sum = ((x1[n] as i32 + x1[n + 2] as i32) + ((x1[n + 1] as i32) << 1)) << 9;
let mut s = smlawb((x2[n + 1] as i32) << 8, sum, p0);
s = smlawb(s, (x1[n + 1] as i32) << 11, p1);
x2[n + 1] = sat16(rshift_round(s, 8)) as i16;
}
st.pred_prev_q13 = pred_q13;
for n in 0..len {
let a = x1[n + 1] as i32;
let b = x2[n + 1] as i32;
x1[n + 1] = sat16(a + b) as i16;
x2[n + 1] = sat16(a - b) as i16;
}
}
/// SILK decoder for up to two internal channels (`silk_decoder`).
pub struct SilkDecoder {
api_rate: u32,
api_channels: usize,
ch: [Channel; 2],
stereo: Stereo,
channels_internal: usize,
channels_internal_prev: usize,
internal_rate: u32,
prev_decode_only_middle: bool,
ms_pred: [i32; 2],
}
impl SilkDecoder {
pub fn new(api_rate: u32, api_channels: usize) -> SilkDecoder {
SilkDecoder {
api_rate,
api_channels,
ch: [Channel::new(), Channel::new()],
stereo: Stereo::default(),
channels_internal: api_channels,
channels_internal_prev: 0,
internal_rate: 16000,
prev_decode_only_middle: false,
ms_pred: [0; 2],
}
}
pub fn reset(&mut self) {
self.ch = [Channel::new(), Channel::new()];
self.stereo = Stereo::default();
self.prev_decode_only_middle = false;
self.channels_internal_prev = 0;
}
pub fn set_stream(&mut self, channels: usize, internal_rate: u32) {
self.channels_internal = channels;
self.internal_rate = internal_rate;
}
/// Decodes one SILK frame (10 or 20 ms) into interleaved `out` at the API rate.
#[allow(clippy::cognitive_complexity, reason = "the RFC 6716 SILK frame decode, ported as-is from FilmCraft")]
pub fn decode(
&mut self,
mut dec: Option<&mut RangeDecoder>,
lost: LostFlag,
new_packet: bool,
payload_ms: usize,
out: &mut [f32],
) -> Result<usize> {
let nci = self.channels_internal;
if new_packet {
for c in self.ch.iter_mut() {
c.n_frames_decoded = 0;
}
}
if nci > self.channels_internal_prev && self.channels_internal_prev != 0 {
self.ch[1] = Channel::new();
}
let stereo_to_mono = nci == 1 && self.channels_internal_prev == 2 && self.internal_rate == 1000 * self.ch[0].fs_khz as u32;
if self.ch[0].n_frames_decoded == 0 {
for c in self.ch.iter_mut().take(nci) {
let (nfpp, nb) = match payload_ms {
10 => (1, 2),
20 => (1, 4),
40 => (2, 4),
60 => (3, 4),
_ => return Err(Error::InvalidPacket("bad SILK frame size")),
};
c.n_frames_per_packet = nfpp;
c.nb_subfr = nb;
let fs_khz = ((self.internal_rate >> 10) + 1) as usize;
c.set_fs(fs_khz, self.api_rate);
}
}
if self.api_channels == 2 && nci == 2 && self.channels_internal_prev == 1 {
self.stereo.pred_prev_q13 = [0; 2];
self.stereo.s_side = [0; 2];
self.ch[1].resampler = self.ch[0].resampler.clone();
}
self.channels_internal_prev = nci;
if lost != LostFlag::Lost && self.ch[0].n_frames_decoded == 0 {
let d = dec.as_deref_mut().ok_or(Error::InvalidPacket("missing SILK data"))?;
for c in self.ch.iter_mut().take(nci) {
for i in 0..c.n_frames_per_packet {
c.vad_flags[i] = d.bit_logp(1);
}
c.lbrr_flag = d.bit_logp(1);
}
for c in self.ch.iter_mut().take(nci) {
c.lbrr_flags = [false; 3];
if c.lbrr_flag {
if c.n_frames_per_packet == 1 {
c.lbrr_flags[0] = true;
} else {
let sym = d.icdf(LBRR_FLAGS[c.n_frames_per_packet - 2], 8) + 1;
for i in 0..c.n_frames_per_packet {
c.lbrr_flags[i] = (sym >> i) & 1 != 0;
}
}
}
}
if lost == LostFlag::Normal {
// Skip LBRR data.
for i in 0..self.ch[0].n_frames_per_packet {
for n in 0..nci {
if self.ch[n].lbrr_flags[i] {
if nci == 2 && n == 0 {
stereo_decode_pred(d);
if !self.ch[1].lbrr_flags[i] {
d.icdf(&MID_ONLY, 8);
}
}
let cond = if i > 0 && self.ch[n].lbrr_flags[i - 1] { Cond::Conditionally } else { Cond::Independently };
let c = &mut self.ch[n];
c.decode_indices(d, i, true, cond);
let mut pulses = [0i32; MAX_FRAME + 16];
decode_pulses(d, &mut pulses, c.indices.signal_type, c.indices.quant_offset_type, c.frame_length);
}
}
}
}
}
// Stereo prediction. Each frame starts out coding the side channel, as in libopus.
let mut decode_only_middle = false;
if nci == 2 {
let fi = self.ch[0].n_frames_decoded;
let coded = lost == LostFlag::Normal || (lost == LostFlag::Lbrr && self.ch[0].lbrr_flags[fi]);
match dec.as_deref_mut() {
Some(d) if coded => {
self.ms_pred = stereo_decode_pred(d);
let side_inactive =
(lost == LostFlag::Normal && !self.ch[1].vad_flags[fi]) || (lost == LostFlag::Lbrr && !self.ch[1].lbrr_flags[fi]);
decode_only_middle = if side_inactive { d.icdf(&MID_ONLY, 8) == 1 } else { false };
}
_ => {
self.ms_pred = self.stereo.pred_prev_q13;
}
}
}
if nci == 2 && !decode_only_middle && self.prev_decode_only_middle {
let c = &mut self.ch[1];
c.out_buf = [0; OUT_BUF];
c.s_lpc_q14 = [0; MAX_LPC_ORDER];
c.lag_prev = 100;
c.last_gain_index = 10;
c.prev_signal_type = 0;
c.first_frame_after_reset = true;
}
let l = self.ch[0].frame_length;
let mut buf = [[0i16; MAX_FRAME + 2]; 2];
let has_side = if lost == LostFlag::Normal {
!decode_only_middle
} else {
!self.prev_decode_only_middle || (nci == 2 && lost == LostFlag::Lbrr && self.ch[1].lbrr_flags[self.ch[1].n_frames_decoded])
};
for n in 0..nci {
if n == 0 || has_side {
let frame_index = self.ch[0].n_frames_decoded as isize - n as isize;
let cond = if frame_index <= 0 {
Cond::Independently
} else if lost == LostFlag::Lbrr {
if self.ch[n].lbrr_flags[frame_index as usize - 1] { Cond::Conditionally } else { Cond::Independently }
} else if n > 0 && self.prev_decode_only_middle {
Cond::IndependentlyNoLtpScaling
} else {
Cond::Conditionally
};
let c = &mut self.ch[n];
c.decode_frame(dec.as_deref_mut(), &mut buf[n][2..], lost, cond);
} else {
buf[n][2..2 + l].fill(0);
}
self.ch[n].n_frames_decoded += 1;
}
if self.api_channels == 2 && nci == 2 {
let (a, b) = buf.split_at_mut(1);
stereo_ms_to_lr(&mut self.stereo, &mut a[0], &mut b[0], self.ms_pred, self.ch[0].fs_khz, l);
} else {
buf[0][..2].copy_from_slice(&self.stereo.s_mid);
self.stereo.s_mid.copy_from_slice(&buf[0][l..l + 2]);
}
let n_out = l * self.api_rate as usize / (self.ch[0].fs_khz * 1000);
let ach = self.api_channels;
let mut tmp = Vec::with_capacity(n_out);
for n in 0..ach.min(nci) {
tmp.clear();
self.ch[n].resampler.process(&buf[n][1..1 + l], &mut tmp);
for (i, &v) in tmp.iter().enumerate().take(n_out) {
out[i * ach + n] = v / 32768.0;
}
}
if ach == 2 && nci == 1 {
if stereo_to_mono {
tmp.clear();
self.ch[1].resampler.process(&buf[0][1..1 + l], &mut tmp);
for (i, &v) in tmp.iter().enumerate().take(n_out) {
out[i * 2 + 1] = v / 32768.0;
}
} else {
for i in 0..n_out {
out[i * 2 + 1] = out[i * 2];
}
}
}
if lost == LostFlag::Lost {
for c in self.ch.iter_mut().take(nci) {
c.last_gain_index = 10;
}
} else {
self.prev_decode_only_middle = decode_only_middle;
}
Ok(n_out)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_lost_frame_after_a_mid_only_frame_resets_the_side_channel() {
let mut d = SilkDecoder::new(48000, 2);
d.set_stream(2, 16000);
// A first frame sets the internal rate, which resets both channels by itself.
d.decode(None, LostFlag::Lost, true, 20, &mut [0.0; 2 * 960]).unwrap();
// What a coded mid-only stereo frame leaves behind.
d.prev_decode_only_middle = true;
d.ch[1].first_frame_after_reset = false;
d.decode(None, LostFlag::Lost, true, 20, &mut [0.0; 2 * 960]).unwrap();
assert!(d.ch[1].first_frame_after_reset, "a lost frame codes no mid-only flag, so the side channel restarts");
}
}
crates/audio/src/opus/decode/silk/resampler.rs
+205
View File
@@ -0,0 +1,205 @@
//! SILK output resampler (RFC 6716 §4.2.9: non-normative filter, normative delay budget).
//!
//! A windowed-sinc polyphase interpolator whose group delay matches the delay allocation of
//! RFC 6716 Table 54 (and the integer delays used when no rate conversion is needed), so that the
//! SILK output lines up with the CELT layer in Hybrid mode.
#[derive(Clone)]
pub struct Resampler {
fin: u32,
fout: u32,
/// Pure-delay mode (equal rates): delay line.
copy_delay: usize,
/// Polyphase taps per output phase.
phases: Vec<(i64, Vec<f32>)>,
period: u64,
/// Total delay in units of 1/fout input samples.
delay_q: i64,
half: i64,
future: i64,
hist: Vec<f32>,
/// Absolute input index of `hist[0]`.
hist_start: i64,
n_in: i64,
n_out: u64,
}
fn gcd(a: u32, b: u32) -> u32 {
if b == 0 { a } else { gcd(b, a % b) }
}
/// Least-squares fractional-delay interpolator: taps at offsets `-past+1 ..= future` (in input
/// samples) approximating a delay of `frac` samples over `[0, wp]`, with zero response weighted
/// over `[ws, pi]` when `ws < pi` (decimation).
fn design_fractional(past: i64, future: i64, frac: f64, wp: f64, ws: f64) -> Vec<f64> {
let offs: Vec<f64> = ((-past + 1)..=future).map(|m| m as f64).collect();
let n = offs.len();
// \int_a^b cos(w x) dw
let ic = |x: f64, a: f64, b: f64| if x.abs() < 1e-12 { b - a } else { ((b * x).sin() - (a * x).sin()) / x };
let stop_w = 30.0;
const LOW_W: f64 = 0.05;
let mut q = vec![vec![0f64; n + 1]; n];
for i in 0..n {
for k in 0..n {
let d = offs[i] - offs[k];
let mut v = ic(d, 0.0, 0.6 * wp) + LOW_W * ic(d, 0.6 * wp, wp);
if ws < std::f64::consts::PI {
v += stop_w * ic(d, ws, std::f64::consts::PI);
}
q[i][k] = v + if i == k { 1e-9 } else { 0.0 };
}
q[i][n] = ic(offs[i] - frac, 0.0, 0.6 * wp) + LOW_W * ic(offs[i] - frac, 0.6 * wp, wp);
}
// Gaussian elimination with partial pivoting.
for c in 0..n {
let p = (c..n).max_by(|&a, &b| q[a][c].abs().total_cmp(&q[b][c].abs())).unwrap_or(c);
q.swap(c, p);
let piv = q[c][c];
for r in 0..n {
if r != c {
let f = q[r][c] / piv;
if f != 0.0 {
for k in c..=n {
q[r][k] -= f * q[c][k];
}
}
}
}
}
(0..n).map(|i| q[i][n] / q[i][i]).collect()
}
impl Resampler {
pub fn new(fin: u32, fout: u32) -> Resampler {
let mut r = Resampler {
fin,
fout,
copy_delay: 0,
phases: Vec::new(),
period: 1,
delay_q: 0,
half: 0,
future: 0,
hist: Vec::new(),
hist_start: 0,
n_in: 0,
n_out: 0,
};
if fin == fout {
r.copy_delay = match fin {
8000 => 4,
12000 => 9,
_ => 12,
};
r.hist = vec![0.0; r.copy_delay];
return r;
}
// Delay in milliseconds per internal bandwidth (RFC 6716 Table 54).
let delay_ms = match fin {
8000 => 0.538,
12000 => 0.692,
_ => 0.706,
};
let d_in = delay_ms * fin as f64 / 1000.0; // input samples
// Future taps are limited by the delay budget; past taps are free.
let future = (d_in.floor() as i64).max(1);
let past = 24i64;
r.half = past;
r.future = future;
r.delay_q = (d_in * fout as f64).round() as i64;
let g = gcd(fin, fout);
r.period = (fout / g) as u64;
let pi = std::f64::consts::PI;
// Passband edge and (for decimation) stopband edge in radians per input sample.
let (wp, ws) = if fout > fin { (0.94 * pi, pi) } else { (0.88 * pi * fout as f64 / fin as f64, pi * fout as f64 / fin as f64) };
for j in 0..r.period as i64 {
let num = j * fin as i64 - r.delay_q;
let base = num.div_euclid(fout as i64);
let frac = num.rem_euclid(fout as i64) as f64 / fout as f64;
let taps = design_fractional(past, future, frac, wp, ws);
r.phases.push((base, taps.iter().map(|&t| t as f32).collect()));
}
r.hist = vec![0.0; (past + future + 4) as usize];
r.hist_start = -(past + future + 4);
r
}
pub fn rates(&self) -> (u32, u32) {
(self.fin, self.fout)
}
/// Resamples `input` (16-bit scale) and appends the output samples.
pub fn process(&mut self, input: &[i16], out: &mut Vec<f32>) {
if self.fin == self.fout {
for &v in input {
self.hist.push(v as f32);
}
let n = self.hist.len() - self.copy_delay;
out.extend_from_slice(&self.hist[..n]);
self.hist.drain(..n);
return;
}
self.hist.extend(input.iter().map(|&v| v as f32));
self.n_in += input.len() as i64;
let total_out = (self.n_in as u64 * self.fout as u64) / self.fin as u64;
while self.n_out < total_out {
let j = self.n_out;
let cycle = (j / self.period) as i64;
let (base0, ref taps) = self.phases[(j % self.period) as usize];
let base = base0 + cycle * self.fin as i64 * self.period as i64 / self.fout as i64;
let first = base - self.half + 1;
let mut acc = 0f32;
for (m, &t) in taps.iter().enumerate() {
let idx = first + m as i64 - self.hist_start;
if idx >= 0 && (idx as usize) < self.hist.len() {
acc += self.hist[idx as usize] * t;
}
}
out.push(acc);
self.n_out += 1;
}
// Trim history, keeping enough for the next taps.
let keep_from = self.n_in - (self.half + self.future + 4);
if keep_from > self.hist_start {
let drop = (keep_from - self.hist_start) as usize;
self.hist.drain(..drop.min(self.hist.len()));
self.hist_start = keep_from;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn upsample_sine_and_delay() {
for &(fin, fout) in &[(16000u32, 48000u32), (8000, 48000), (12000, 48000), (16000, 24000), (16000, 8000), (12000, 16000)] {
let mut r = Resampler::new(fin, fout);
let f = 440.0;
let n = fin as usize / 10;
let x: Vec<i16> = (0..n).map(|i| ((2.0 * std::f64::consts::PI * f * i as f64 / fin as f64).sin() * 10000.0) as i16).collect();
let mut y = Vec::new();
for c in x.chunks(fin as usize / 50) {
r.process(c, &mut y);
}
assert_eq!(y.len(), n * fout as usize / fin as usize);
// Compare against the ideal delayed sine.
let delay_ms = match fin {
8000 => 0.538,
12000 => 0.692,
_ => 0.706,
};
let mut err = 0f64;
let mut sig = 0f64;
for (j, &v) in y.iter().enumerate().skip(fout as usize / 100) {
let t = j as f64 / fout as f64 - delay_ms / 1000.0;
let ideal = (2.0 * std::f64::consts::PI * f * t).sin() * 10000.0;
err += (v as f64 - ideal).powi(2);
sig += ideal * ideal;
}
let snr = 10.0 * (sig / err).log10();
assert!(snr > 35.0, "{fin}->{fout}: {snr} dB");
}
}
}
crates/audio/src/opus/decode/silk/tables.rs
+540
View File
@@ -0,0 +1,540 @@
//! SILK probability tables and codebooks, transcribed from RFC 6716 §4.2 (Tables 4–53).
//!
//! PDFs are written as in the RFC (`{...}/256`) and converted at compile time to the inverse-CDF
//! form consumed by the range decoder.
/// Converts a PDF (summing to 256) into an inverse CDF (`256 - cdf[i+1]`).
pub const fn icdf<const N: usize>(pdf: &[u8; N]) -> [u8; N] {
let mut out = [0u8; N];
let mut acc: u32 = 0;
let mut i = 0;
while i < N {
acc += pdf[i] as u32;
out[i] = (256 - acc) as u8;
i += 1;
}
out
}
pub const STEREO_PRED_JOINT: [u8; 25] = icdf(&[7, 2, 1, 1, 1, 10, 24, 8, 1, 1, 3, 23, 92, 23, 3, 1, 1, 8, 24, 10, 1, 1, 1, 2, 7]);
pub const UNIFORM3: [u8; 3] = icdf(&[85, 86, 85]);
pub const UNIFORM5: [u8; 5] = icdf(&[51, 51, 52, 51, 51]);
pub const STEREO_WEIGHTS_Q13: [i32; 16] = [-13732, -10050, -8266, -7526, -6500, -5000, -2950, -820, 820, 2950, 5000, 6500, 7526, 8266, 10050, 13732];
pub const MID_ONLY: [u8; 2] = icdf(&[192, 64]);
pub const TYPE_INACTIVE: [u8; 2] = icdf(&[26, 230]);
pub const TYPE_ACTIVE: [u8; 4] = icdf(&[24, 74, 148, 10]);
pub const GAIN_MSB: [&[u8]; 3] =
[&icdf::<8>(&[32, 112, 68, 29, 12, 1, 1, 1]), &icdf::<8>(&[2, 17, 45, 60, 62, 47, 19, 4]), &icdf::<8>(&[1, 3, 26, 71, 94, 50, 9, 2])];
pub const UNIFORM8: [u8; 8] = icdf(&[32, 32, 32, 32, 32, 32, 32, 32]);
pub const DELTA_GAIN: [u8; 41] =
icdf(&[6, 5, 11, 31, 132, 21, 8, 4, 3, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]);
pub const NLSF_STAGE1: [&[u8]; 4] = [
&icdf::<32>(&[44, 34, 30, 19, 21, 12, 11, 3, 3, 2, 16, 2, 2, 1, 5, 2, 1, 3, 3, 1, 1, 2, 2, 2, 3, 1, 9, 9, 2, 7, 2, 1]),
&icdf::<32>(&[1, 10, 1, 8, 3, 8, 8, 14, 13, 14, 1, 14, 12, 13, 11, 11, 12, 11, 10, 10, 11, 8, 9, 8, 7, 8, 1, 1, 6, 1, 6, 5]),
&icdf::<32>(&[31, 21, 3, 17, 1, 8, 17, 4, 1, 18, 16, 4, 2, 3, 1, 10, 1, 3, 16, 11, 16, 2, 2, 3, 2, 11, 1, 4, 9, 8, 7, 3]),
&icdf::<32>(&[1, 4, 16, 5, 18, 11, 5, 14, 15, 1, 3, 12, 13, 14, 14, 6, 14, 12, 2, 6, 1, 12, 12, 11, 10, 3, 10, 5, 1, 1, 1, 3]),
];
pub const NLSF_STAGE2: [&[u8]; 16] = [
&icdf::<9>(&[1, 1, 1, 15, 224, 11, 1, 1, 1]),
&icdf::<9>(&[1, 1, 2, 34, 183, 32, 1, 1, 1]),
&icdf::<9>(&[1, 1, 4, 42, 149, 55, 2, 1, 1]),
&icdf::<9>(&[1, 1, 8, 52, 123, 61, 8, 1, 1]),
&icdf::<9>(&[1, 3, 16, 53, 101, 74, 6, 1, 1]),
&icdf::<9>(&[1, 3, 17, 55, 90, 73, 15, 1, 1]),
&icdf::<9>(&[1, 7, 24, 53, 74, 67, 26, 3, 1]),
&icdf::<9>(&[1, 1, 18, 63, 78, 58, 30, 6, 1]),
&icdf::<9>(&[1, 1, 1, 9, 232, 9, 1, 1, 1]),
&icdf::<9>(&[1, 1, 2, 28, 186, 35, 1, 1, 1]),
&icdf::<9>(&[1, 1, 3, 42, 152, 53, 2, 1, 1]),
&icdf::<9>(&[1, 1, 10, 49, 126, 65, 2, 1, 1]),
&icdf::<9>(&[1, 4, 19, 48, 100, 77, 5, 1, 1]),
&icdf::<9>(&[1, 1, 14, 54, 100, 72, 12, 1, 1]),
&icdf::<9>(&[1, 1, 15, 61, 87, 61, 25, 4, 1]),
&icdf::<9>(&[1, 7, 21, 50, 77, 81, 17, 1, 1]),
];
pub const NLSF_SEL_NBMB: [[u8; 10]; 32] = [
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 3, 1, 2, 2, 1, 2, 1, 1, 1],
[2, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 2, 2, 2, 2, 1, 2, 1, 1, 1],
[2, 3, 3, 3, 3, 2, 2, 2, 2, 2],
[0, 5, 3, 3, 2, 2, 2, 2, 1, 1],
[0, 2, 2, 2, 2, 2, 2, 2, 2, 1],
[2, 3, 6, 4, 4, 4, 5, 4, 5, 5],
[2, 4, 5, 5, 4, 5, 4, 6, 4, 4],
[2, 4, 4, 7, 4, 5, 4, 5, 5, 4],
[4, 3, 3, 3, 2, 3, 2, 2, 2, 2],
[1, 5, 5, 6, 4, 5, 4, 5, 5, 5],
[2, 7, 4, 6, 5, 5, 5, 5, 5, 5],
[2, 7, 5, 5, 5, 5, 5, 6, 5, 4],
[3, 3, 5, 4, 4, 5, 4, 5, 4, 4],
[2, 3, 3, 5, 5, 4, 4, 4, 4, 4],
[2, 4, 4, 6, 4, 5, 4, 5, 5, 5],
[2, 5, 4, 6, 5, 5, 5, 4, 5, 4],
[2, 7, 4, 5, 4, 5, 4, 5, 5, 5],
[2, 5, 4, 6, 7, 6, 5, 6, 5, 4],
[3, 6, 7, 4, 6, 5, 5, 6, 4, 5],
[2, 7, 6, 4, 4, 4, 5, 4, 5, 5],
[4, 5, 5, 4, 6, 6, 5, 6, 5, 4],
[2, 5, 5, 6, 5, 6, 4, 6, 4, 4],
[4, 5, 5, 5, 3, 7, 4, 5, 5, 4],
[2, 3, 4, 5, 5, 6, 4, 5, 5, 4],
[2, 3, 2, 3, 3, 4, 2, 3, 3, 3],
[1, 1, 2, 2, 2, 2, 2, 3, 2, 2],
[4, 5, 5, 6, 6, 6, 5, 6, 4, 5],
[3, 5, 5, 4, 4, 4, 4, 3, 3, 2],
[2, 5, 3, 7, 5, 5, 4, 4, 5, 4],
[4, 4, 5, 4, 5, 6, 5, 6, 5, 4],
];
pub const NLSF_SEL_WB: [[u8; 16]; 32] = [
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[2, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 1, 1, 1, 0, 3],
[2, 5, 5, 3, 7, 4, 4, 5, 2, 5, 4, 5, 5, 4, 3, 3],
[0, 2, 1, 2, 2, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1],
[0, 6, 5, 4, 6, 4, 7, 5, 4, 4, 4, 5, 5, 4, 4, 3],
[0, 3, 5, 5, 4, 3, 3, 5, 3, 3, 3, 3, 3, 3, 2, 4],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 2, 6, 3, 7, 2, 5, 3, 4, 5, 5, 4, 3, 3, 2, 3],
[0, 6, 2, 6, 6, 4, 5, 4, 6, 5, 4, 4, 5, 3, 3, 3],
[2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[2, 2, 3, 4, 5, 3, 3, 3, 3, 3, 3, 3, 2, 2, 1, 3],
[2, 2, 3, 3, 4, 3, 3, 3, 3, 3, 3, 3, 3, 2, 1, 3],
[3, 4, 4, 4, 6, 4, 4, 5, 3, 5, 4, 4, 5, 4, 3, 4],
[0, 6, 4, 5, 4, 7, 5, 2, 6, 5, 7, 4, 4, 3, 5, 3],
[0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0],
[1, 6, 5, 7, 5, 4, 5, 3, 4, 5, 4, 4, 4, 3, 3, 4],
[1, 3, 3, 4, 4, 3, 3, 5, 2, 3, 3, 5, 5, 5, 3, 4],
[2, 3, 3, 2, 2, 2, 3, 2, 1, 2, 1, 2, 1, 1, 1, 4],
[0, 2, 3, 5, 3, 3, 2, 2, 2, 1, 1, 0, 0, 0, 0, 0],
[3, 4, 3, 5, 3, 3, 2, 2, 1, 1, 1, 1, 1, 2, 2, 4],
[2, 6, 3, 7, 7, 4, 5, 4, 5, 3, 5, 3, 3, 2, 3, 3],
[2, 3, 5, 6, 6, 3, 5, 3, 4, 4, 3, 3, 3, 3, 2, 4],
[1, 3, 3, 4, 4, 4, 4, 3, 5, 5, 5, 3, 1, 1, 1, 1],
[2, 5, 3, 6, 6, 4, 7, 4, 4, 5, 3, 4, 4, 3, 3, 3],
[0, 6, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 6, 6, 3, 5, 2, 5, 5, 3, 4, 4, 7, 7, 4, 4, 4],
[3, 3, 7, 3, 5, 4, 3, 3, 3, 2, 2, 3, 3, 3, 2, 3],
[0, 0, 1, 0, 0, 0, 2, 1, 2, 1, 1, 2, 2, 2, 1, 1],
[0, 3, 2, 5, 3, 3, 2, 3, 2, 1, 0, 0, 1, 0, 0, 1],
[3, 5, 5, 4, 7, 5, 3, 3, 2, 3, 2, 2, 1, 0, 1, 0],
[2, 3, 5, 3, 4, 3, 3, 3, 2, 1, 2, 6, 4, 0, 0, 0],
];
pub const NLSF_EXT: [u8; 7] = icdf(&[156, 60, 24, 9, 4, 2, 1]);
pub const NLSF_PRED_NBMB: [[i32; 9]; 2] = [[179, 138, 140, 148, 151, 149, 153, 151, 163], [116, 67, 82, 59, 92, 72, 100, 89, 92]];
pub const NLSF_PRED_WB: [[i32; 15]; 2] = [
[175, 148, 160, 176, 178, 173, 174, 164, 177, 174, 196, 182, 198, 192, 182],
[68, 62, 66, 60, 72, 117, 85, 90, 118, 136, 151, 142, 160, 142, 155],
];
pub const NLSF_PSEL_NBMB: [[u8; 9]; 32] = [
[0, 1, 0, 0, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 0, 0, 0, 0, 1, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0],
[1, 0, 1, 1, 0, 0, 0, 1, 0],
[0, 1, 1, 0, 0, 1, 1, 0, 0],
[0, 0, 1, 1, 0, 1, 0, 1, 1],
[0, 0, 1, 1, 0, 0, 1, 1, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 1, 1, 1, 1, 1, 0],
[0, 1, 0, 1, 1, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 1, 1, 1, 0],
[1, 0, 1, 1, 0, 1, 1, 1, 1],
[0, 1, 1, 1, 1, 1, 0, 1, 0],
[0, 0, 1, 1, 0, 1, 0, 1, 0],
[0, 0, 1, 1, 1, 0, 1, 1, 1],
[0, 1, 1, 0, 0, 1, 1, 1, 0],
[0, 0, 0, 1, 1, 1, 0, 1, 0],
[0, 1, 1, 0, 0, 1, 0, 1, 0],
[0, 1, 1, 0, 0, 0, 1, 1, 0],
[0, 0, 0, 0, 0, 1, 1, 1, 1],
[0, 0, 1, 1, 0, 0, 0, 1, 1],
[0, 0, 0, 1, 0, 1, 1, 1, 1],
[0, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 0, 1, 1, 0, 1, 0],
[1, 0, 0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 1, 0, 1, 0, 1],
[1, 0, 1, 1, 0, 1, 1, 1, 1],
];
pub const NLSF_PSEL_WB: [[u8; 15]; 32] = [
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0],
[0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0],
[0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0],
[0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1],
[0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0],
[0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0],
[0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1],
[0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0],
[0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0],
[0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0],
[0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0],
[0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1],
[0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0],
[0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0],
];
pub const NLSF_CB_NBMB: [[i32; 10]; 32] = [
[12, 35, 60, 83, 108, 132, 157, 180, 206, 228],
[15, 32, 55, 77, 101, 125, 151, 175, 201, 225],
[19, 42, 66, 89, 114, 137, 162, 184, 209, 230],
[12, 25, 50, 72, 97, 120, 147, 172, 200, 223],
[26, 44, 69, 90, 114, 135, 159, 180, 205, 225],
[13, 22, 53, 80, 106, 130, 156, 180, 205, 228],
[15, 25, 44, 64, 90, 115, 142, 168, 196, 222],
[19, 24, 62, 82, 100, 120, 145, 168, 190, 214],
[22, 31, 50, 79, 103, 120, 151, 170, 203, 227],
[21, 29, 45, 65, 106, 124, 150, 171, 196, 224],
[30, 49, 75, 97, 121, 142, 165, 186, 209, 229],
[19, 25, 52, 70, 93, 116, 143, 166, 192, 219],
[26, 34, 62, 75, 97, 118, 145, 167, 194, 217],
[25, 33, 56, 70, 91, 113, 143, 165, 196, 223],
[21, 34, 51, 72, 97, 117, 145, 171, 196, 222],
[20, 29, 50, 67, 90, 117, 144, 168, 197, 221],
[22, 31, 48, 66, 95, 117, 146, 168, 196, 222],
[24, 33, 51, 77, 116, 134, 158, 180, 200, 224],
[21, 28, 70, 87, 106, 124, 149, 170, 194, 217],
[26, 33, 53, 64, 83, 117, 152, 173, 204, 225],
[27, 34, 65, 95, 108, 129, 155, 174, 210, 225],
[20, 26, 72, 99, 113, 131, 154, 176, 200, 219],
[34, 43, 61, 78, 93, 114, 155, 177, 205, 229],
[23, 29, 54, 97, 124, 138, 163, 179, 209, 229],
[30, 38, 56, 89, 118, 129, 158, 178, 200, 231],
[21, 29, 49, 63, 85, 111, 142, 163, 193, 222],
[27, 48, 77, 103, 133, 158, 179, 196, 215, 232],
[29, 47, 74, 99, 124, 151, 176, 198, 220, 237],
[33, 42, 61, 76, 93, 121, 155, 174, 207, 225],
[29, 53, 87, 112, 136, 154, 170, 188, 208, 227],
[24, 30, 52, 84, 131, 150, 166, 186, 203, 229],
[37, 48, 64, 84, 104, 118, 156, 177, 201, 230],
];
pub const NLSF_CB_WB: [[i32; 16]; 32] = [
[7, 23, 38, 54, 69, 85, 100, 116, 131, 147, 162, 178, 193, 208, 223, 239],
[13, 25, 41, 55, 69, 83, 98, 112, 127, 142, 157, 171, 187, 203, 220, 236],
[15, 21, 34, 51, 61, 78, 92, 106, 126, 136, 152, 167, 185, 205, 225, 240],
[10, 21, 36, 50, 63, 79, 95, 110, 126, 141, 157, 173, 189, 205, 221, 237],
[17, 20, 37, 51, 59, 78, 89, 107, 123, 134, 150, 164, 184, 205, 224, 240],
[10, 15, 32, 51, 67, 81, 96, 112, 129, 142, 158, 173, 189, 204, 220, 236],
[8, 21, 37, 51, 65, 79, 98, 113, 126, 138, 155, 168, 179, 192, 209, 218],
[12, 15, 34, 55, 63, 78, 87, 108, 118, 131, 148, 167, 185, 203, 219, 236],
[16, 19, 32, 36, 56, 79, 91, 108, 118, 136, 154, 171, 186, 204, 220, 237],
[11, 28, 43, 58, 74, 89, 105, 120, 135, 150, 165, 180, 196, 211, 226, 241],
[6, 16, 33, 46, 60, 75, 92, 107, 123, 137, 156, 169, 185, 199, 214, 225],
[11, 19, 30, 44, 57, 74, 89, 105, 121, 135, 152, 169, 186, 202, 218, 234],
[12, 19, 29, 46, 57, 71, 88, 100, 120, 132, 148, 165, 182, 199, 216, 233],
[17, 23, 35, 46, 56, 77, 92, 106, 123, 134, 152, 167, 185, 204, 222, 237],
[14, 17, 45, 53, 63, 75, 89, 107, 115, 132, 151, 171, 188, 206, 221, 240],
[9, 16, 29, 40, 56, 71, 88, 103, 119, 137, 154, 171, 189, 205, 222, 237],
[16, 19, 36, 48, 57, 76, 87, 105, 118, 132, 150, 167, 185, 202, 218, 236],
[12, 17, 29, 54, 71, 81, 94, 104, 126, 136, 149, 164, 182, 201, 221, 237],
[15, 28, 47, 62, 79, 97, 115, 129, 142, 155, 168, 180, 194, 208, 223, 238],
[8, 14, 30, 45, 62, 78, 94, 111, 127, 143, 159, 175, 192, 207, 223, 239],
[17, 30, 49, 62, 79, 92, 107, 119, 132, 145, 160, 174, 190, 204, 220, 235],
[14, 19, 36, 45, 61, 76, 91, 108, 121, 138, 154, 172, 189, 205, 222, 238],
[12, 18, 31, 45, 60, 76, 91, 107, 123, 138, 154, 171, 187, 204, 221, 236],
[13, 17, 31, 43, 53, 70, 83, 103, 114, 131, 149, 167, 185, 203, 220, 237],
[17, 22, 35, 42, 58, 78, 93, 110, 125, 139, 155, 170, 188, 206, 224, 240],
[8, 15, 34, 50, 67, 83, 99, 115, 131, 146, 162, 178, 193, 209, 224, 239],
[13, 16, 41, 66, 73, 86, 95, 111, 128, 137, 150, 163, 183, 206, 225, 241],
[17, 25, 37, 52, 63, 75, 92, 102, 119, 132, 144, 160, 175, 191, 212, 231],
[19, 31, 49, 65, 83, 100, 117, 133, 147, 161, 174, 187, 200, 213, 227, 242],
[18, 31, 52, 68, 88, 103, 117, 126, 138, 149, 163, 177, 192, 207, 223, 239],
[16, 29, 47, 61, 76, 90, 106, 119, 133, 147, 161, 176, 193, 209, 224, 240],
[15, 21, 35, 50, 61, 73, 86, 97, 110, 119, 129, 141, 175, 198, 218, 237],
];
pub const NLSF_MIN_SPACING_NBMB: [i32; 11] = [250, 3, 6, 3, 3, 3, 4, 3, 3, 3, 461];
pub const NLSF_MIN_SPACING_WB: [i32; 17] = [100, 3, 40, 3, 3, 3, 5, 14, 14, 10, 11, 3, 8, 9, 7, 3, 347];
pub const NLSF_INTERP: [u8; 5] = icdf(&[13, 22, 29, 11, 181]);
pub const LSF_ORDER_NBMB: [usize; 10] = [0, 9, 6, 3, 4, 5, 8, 1, 2, 7];
pub const LSF_ORDER_WB: [usize; 16] = [0, 15, 8, 7, 4, 11, 12, 3, 2, 13, 10, 5, 6, 9, 14, 1];
pub const COS_Q12: [i32; 129] = [
4096, 4095, 4091, 4085, 4076, 4065, 4052, 4036, 4017, 3997, 3973, 3948, 3920, 3889, 3857, 3822, 3784, 3745, 3703, 3659, 3613, 3564, 3513, 3461,
3406, 3349, 3290, 3229, 3166, 3102, 3035, 2967, 2896, 2824, 2751, 2676, 2599, 2520, 2440, 2359, 2276, 2191, 2106, 2019, 1931, 1842, 1751, 1660,
1568, 1474, 1380, 1285, 1189, 1093, 995, 897, 799, 700, 601, 501, 401, 301, 201, 101, 0, -101, -201, -301, -401, -501, -601, -700, -799, -897,
-995, -1093, -1189, -1285, -1380, -1474, -1568, -1660, -1751, -1842, -1931, -2019, -2106, -2191, -2276, -2359, -2440, -2520, -2599, -2676, -2751,
-2824, -2896, -2967, -3035, -3102, -3166, -3229, -3290, -3349, -3406, -3461, -3513, -3564, -3613, -3659, -3703, -3745, -3784, -3822, -3857,
-3889, -3920, -3948, -3973, -3997, -4017, -4036, -4052, -4065, -4076, -4085, -4091, -4095, -4096,
];
pub const PITCH_HIGH: [u8; 32] = icdf(&[3, 3, 6, 11, 21, 30, 32, 19, 11, 10, 12, 13, 13, 12, 11, 9, 8, 7, 6, 4, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1]);
pub const PITCH_LOW: [&[u8]; 3] =
[&icdf::<4>(&[64, 64, 64, 64]), &icdf::<6>(&[43, 42, 43, 43, 42, 43]), &icdf::<8>(&[32, 32, 32, 32, 32, 32, 32, 32])];
pub const PITCH_DELTA: [u8; 21] = icdf(&[46, 2, 2, 3, 4, 6, 10, 15, 26, 38, 30, 22, 15, 10, 7, 6, 4, 4, 2, 2, 2]);
pub const PITCH_CONTOUR: [&[u8]; 4] = [
&icdf::<3>(&[143, 50, 63]),
&icdf::<11>(&[68, 12, 21, 17, 19, 22, 30, 24, 17, 16, 10]),
&icdf::<12>(&[91, 46, 39, 19, 14, 12, 8, 7, 6, 5, 5, 4]),
&icdf::<34>(&[33, 22, 18, 16, 15, 14, 14, 13, 13, 10, 9, 9, 8, 6, 6, 6, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1]),
];
pub const CB_NB_10: [[i32; 2]; 3] = [[0, 0], [1, 0], [0, 1]];
pub const CB_NB_20: [[i32; 4]; 11] = [
[0, 0, 0, 0],
[2, 1, 0, -1],
[-1, 0, 1, 2],
[-1, 0, 0, 1],
[-1, 0, 0, 0],
[0, 0, 0, 1],
[0, 0, 1, 1],
[1, 1, 0, 0],
[1, 0, 0, 0],
[0, 0, 0, -1],
[1, 0, 0, -1],
];
pub const CB_WB_10: [[i32; 2]; 12] = [[0, 0], [0, 1], [1, 0], [-1, 1], [1, -1], [-1, 2], [2, -1], [-2, 2], [2, -2], [-2, 3], [3, -2], [-3, 3]];
pub const CB_WB_20: [[i32; 4]; 34] = [
[0, 0, 0, 0],
[0, 0, 1, 1],
[1, 1, 0, 0],
[-1, 0, 0, 0],
[0, 0, 0, 1],
[1, 0, 0, 0],
[-1, 0, 0, 1],
[0, 0, 0, -1],
[-1, 0, 1, 2],
[1, 0, 0, -1],
[-2, -1, 1, 2],
[2, 1, 0, -1],
[-2, 0, 0, 2],
[-2, 0, 1, 3],
[2, 1, -1, -2],
[-3, -1, 1, 3],
[2, 0, 0, -2],
[3, 1, 0, -2],
[-3, -1, 2, 4],
[-4, -1, 1, 4],
[3, 1, -1, -3],
[-4, -1, 2, 5],
[4, 2, -1, -3],
[4, 1, -1, -4],
[-5, -1, 2, 6],
[5, 2, -1, -4],
[-6, -2, 2, 6],
[-5, -2, 2, 5],
[6, 2, -1, -5],
[-7, -2, 3, 8],
[6, 2, -2, -6],
[5, 2, -2, -5],
[8, 3, -2, -7],
[-9, -3, 3, 9],
];
pub const LTP_PERIODICITY: [u8; 3] = icdf(&[77, 80, 99]);
pub const LTP_FILTER: [&[u8]; 3] = [
&icdf::<8>(&[185, 15, 13, 13, 9, 9, 6, 6]),
&icdf::<16>(&[57, 34, 21, 20, 15, 13, 12, 13, 10, 10, 9, 10, 9, 8, 7, 8]),
&icdf::<32>(&[15, 16, 14, 12, 12, 12, 11, 11, 11, 10, 9, 9, 9, 9, 8, 8, 8, 8, 7, 7, 6, 6, 5, 4, 5, 4, 4, 4, 3, 4, 3, 2]),
];
pub const LTP_TAPS_0: [[i32; 5]; 8] = [
[4, 6, 24, 7, 5],
[0, 0, 2, 0, 0],
[12, 28, 41, 13, -4],
[-9, 15, 42, 25, 14],
[1, -2, 62, 41, -9],
[-10, 37, 65, -4, 3],
[-6, 4, 66, 7, -8],
[16, 14, 38, -3, 33],
];
pub const LTP_TAPS_1: [[i32; 5]; 16] = [
[13, 22, 39, 23, 12],
[-1, 36, 64, 27, -6],
[-7, 10, 55, 43, 17],
[1, 1, 8, 1, 1],
[6, -11, 74, 53, -9],
[-12, 55, 76, -12, 8],
[-3, 3, 93, 27, -4],
[26, 39, 59, 3, -8],
[2, 0, 77, 11, 9],
[-8, 22, 44, -6, 7],
[40, 9, 26, 3, 9],
[-7, 20, 101, -7, 4],
[3, -8, 42, 26, 0],
[-15, 33, 68, 2, 23],
[-2, 55, 46, -2, 15],
[3, -1, 21, 16, 41],
];
pub const LTP_TAPS_2: [[i32; 5]; 32] = [
[-6, 27, 61, 39, 5],
[-11, 42, 88, 4, 1],
[-2, 60, 65, 6, -4],
[-1, -5, 73, 56, 1],
[-9, 19, 94, 29, -9],
[0, 12, 99, 6, 4],
[8, -19, 102, 46, -13],
[3, 2, 13, 3, 2],
[9, -21, 84, 72, -18],
[-11, 46, 104, -22, 8],
[18, 38, 48, 23, 0],
[-16, 70, 83, -21, 11],
[5, -11, 117, 22, -8],
[-6, 23, 117, -12, 3],
[3, -8, 95, 28, 4],
[-10, 15, 77, 60, -15],
[-1, 4, 124, 2, -4],
[3, 38, 84, 24, -25],
[2, 13, 42, 13, 31],
[21, -4, 56, 46, -1],
[-1, 35, 79, -13, 19],
[-7, 65, 88, -9, -14],
[20, 4, 81, 49, -29],
[20, 0, 75, 3, -17],
[5, -9, 44, 92, -8],
[1, -3, 22, 69, 31],
[-6, 95, 41, -12, 5],
[39, 67, 16, -4, 1],
[0, -6, 120, 55, -36],
[-13, 44, 122, 4, -24],
[81, 5, 11, 3, 7],
[2, 0, 9, 10, 88],
];
pub const LTP_SCALE: [u8; 3] = icdf(&[128, 64, 64]);
pub const UNIFORM4: [u8; 4] = icdf(&[64, 64, 64, 64]);
pub const RATE_LEVEL: [&[u8]; 2] = [&icdf::<9>(&[15, 51, 12, 46, 45, 13, 33, 27, 14]), &icdf::<9>(&[33, 30, 36, 17, 34, 49, 18, 21, 18])];
pub const PULSE_COUNT: [&[u8]; 11] = [
&icdf::<18>(&[131, 74, 25, 8, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
&icdf::<18>(&[58, 93, 60, 23, 7, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
&icdf::<18>(&[43, 51, 46, 33, 24, 16, 11, 8, 6, 3, 3, 3, 2, 1, 1, 2, 1, 2]),
&icdf::<18>(&[17, 52, 71, 57, 31, 12, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
&icdf::<18>(&[6, 21, 41, 53, 49, 35, 21, 11, 6, 3, 2, 2, 1, 1, 1, 1, 1, 1]),
&icdf::<18>(&[7, 14, 22, 28, 29, 28, 25, 20, 17, 13, 11, 9, 7, 5, 4, 4, 3, 10]),
&icdf::<18>(&[2, 5, 14, 29, 42, 46, 41, 31, 19, 11, 6, 3, 2, 1, 1, 1, 1, 1]),
&icdf::<18>(&[1, 2, 4, 10, 19, 29, 35, 37, 34, 28, 20, 14, 8, 5, 4, 2, 2, 2]),
&icdf::<18>(&[1, 2, 2, 5, 9, 14, 20, 24, 27, 28, 26, 23, 20, 15, 11, 8, 6, 15]),
&icdf::<18>(&[1, 1, 1, 6, 27, 58, 56, 39, 25, 14, 10, 6, 3, 3, 2, 1, 1, 2]),
&icdf::<18>(&[2, 1, 6, 27, 58, 56, 39, 25, 14, 10, 6, 3, 3, 2, 1, 1, 2, 0]),
];
pub const SHELL_16: [&[u8]; 16] = [
&icdf::<2>(&[126, 130]),
&icdf::<3>(&[56, 142, 58]),
&icdf::<4>(&[25, 101, 104, 26]),
&icdf::<5>(&[12, 60, 108, 64, 12]),
&icdf::<6>(&[7, 35, 84, 87, 37, 6]),
&icdf::<7>(&[4, 20, 59, 86, 63, 21, 3]),
&icdf::<8>(&[3, 12, 38, 72, 75, 42, 12, 2]),
&icdf::<9>(&[2, 8, 25, 54, 73, 59, 27, 7, 1]),
&icdf::<10>(&[2, 5, 17, 39, 63, 65, 42, 18, 4, 1]),
&icdf::<11>(&[1, 4, 12, 28, 49, 63, 54, 30, 11, 3, 1]),
&icdf::<12>(&[1, 4, 8, 20, 37, 55, 57, 41, 22, 8, 2, 1]),
&icdf::<13>(&[1, 3, 7, 15, 28, 44, 53, 48, 33, 16, 6, 1, 1]),
&icdf::<14>(&[1, 2, 6, 12, 21, 35, 47, 48, 40, 25, 12, 5, 1, 1]),
&icdf::<15>(&[1, 1, 4, 10, 17, 27, 37, 47, 43, 33, 21, 9, 4, 1, 1]),
&icdf::<16>(&[1, 1, 1, 8, 14, 22, 33, 40, 43, 38, 28, 16, 8, 1, 1, 1]),
&icdf::<17>(&[1, 1, 1, 1, 13, 18, 27, 36, 41, 41, 34, 24, 14, 1, 1, 1, 1]),
];
pub const SHELL_8: [&[u8]; 16] = [
&icdf::<2>(&[127, 129]),
&icdf::<3>(&[53, 149, 54]),
&icdf::<4>(&[22, 105, 106, 23]),
&icdf::<5>(&[11, 61, 111, 63, 10]),
&icdf::<6>(&[6, 35, 86, 88, 36, 5]),
&icdf::<7>(&[4, 20, 59, 87, 62, 21, 3]),
&icdf::<8>(&[3, 13, 40, 71, 73, 41, 13, 2]),
&icdf::<9>(&[3, 9, 27, 53, 70, 56, 28, 9, 1]),
&icdf::<10>(&[3, 8, 19, 37, 57, 61, 44, 20, 6, 1]),
&icdf::<11>(&[3, 7, 15, 28, 44, 54, 49, 33, 17, 5, 1]),
&icdf::<12>(&[1, 7, 13, 22, 34, 46, 48, 38, 28, 14, 4, 1]),
&icdf::<13>(&[1, 1, 11, 22, 27, 35, 42, 47, 33, 25, 10, 1, 1]),
&icdf::<14>(&[1, 1, 6, 14, 26, 37, 43, 43, 37, 26, 14, 6, 1, 1]),
&icdf::<15>(&[1, 1, 4, 10, 20, 31, 40, 42, 40, 31, 20, 10, 4, 1, 1]),
&icdf::<16>(&[1, 1, 3, 8, 16, 26, 35, 38, 38, 35, 26, 16, 8, 3, 1, 1]),
&icdf::<17>(&[1, 1, 2, 6, 12, 21, 30, 36, 38, 36, 30, 21, 12, 6, 2, 1, 1]),
];
pub const SHELL_4: [&[u8]; 16] = [
&icdf::<2>(&[127, 129]),
&icdf::<3>(&[49, 157, 50]),
&icdf::<4>(&[20, 107, 109, 20]),
&icdf::<5>(&[11, 60, 113, 62, 10]),
&icdf::<6>(&[7, 36, 84, 87, 36, 6]),
&icdf::<7>(&[6, 24, 57, 82, 60, 23, 4]),
&icdf::<8>(&[5, 18, 39, 64, 68, 42, 16, 4]),
&icdf::<9>(&[6, 14, 29, 47, 61, 52, 30, 14, 3]),
&icdf::<10>(&[1, 15, 23, 35, 51, 50, 40, 30, 10, 1]),
&icdf::<11>(&[1, 1, 21, 32, 42, 52, 46, 41, 18, 1, 1]),
&icdf::<12>(&[1, 6, 16, 27, 36, 42, 42, 36, 27, 16, 6, 1]),
&icdf::<13>(&[1, 5, 12, 21, 31, 38, 40, 38, 31, 21, 12, 5, 1]),
&icdf::<14>(&[1, 3, 9, 17, 26, 34, 38, 38, 34, 26, 17, 9, 3, 1]),
&icdf::<15>(&[1, 3, 7, 14, 22, 29, 34, 36, 34, 29, 22, 14, 7, 3, 1]),
&icdf::<16>(&[1, 2, 5, 11, 18, 25, 31, 35, 35, 31, 25, 18, 11, 5, 2, 1]),
&icdf::<17>(&[1, 1, 4, 9, 15, 21, 28, 32, 34, 32, 28, 21, 15, 9, 4, 1, 1]),
];
pub const SHELL_2: [&[u8]; 16] = [
&icdf::<2>(&[128, 128]),
&icdf::<3>(&[42, 172, 42]),
&icdf::<4>(&[21, 107, 107, 21]),
&icdf::<5>(&[12, 60, 112, 61, 11]),
&icdf::<6>(&[8, 34, 86, 86, 35, 7]),
&icdf::<7>(&[8, 23, 55, 90, 55, 20, 5]),
&icdf::<8>(&[5, 15, 38, 72, 72, 36, 15, 3]),
&icdf::<9>(&[6, 12, 27, 52, 77, 47, 20, 10, 5]),
&icdf::<10>(&[6, 19, 28, 35, 40, 40, 35, 28, 19, 6]),
&icdf::<11>(&[4, 14, 22, 31, 37, 40, 37, 31, 22, 14, 4]),
&icdf::<12>(&[3, 10, 18, 26, 33, 38, 38, 33, 26, 18, 10, 3]),
&icdf::<13>(&[2, 8, 13, 21, 29, 36, 38, 36, 29, 21, 13, 8, 2]),
&icdf::<14>(&[1, 5, 10, 17, 25, 32, 38, 38, 32, 25, 17, 10, 5, 1]),
&icdf::<15>(&[1, 4, 7, 13, 21, 29, 35, 36, 35, 29, 21, 13, 7, 4, 1]),
&icdf::<16>(&[1, 2, 5, 10, 17, 25, 32, 36, 36, 32, 25, 17, 10, 5, 2, 1]),
&icdf::<17>(&[1, 2, 4, 7, 13, 21, 28, 34, 36, 34, 28, 21, 13, 7, 4, 2, 1]),
];
pub const EXC_LSB: [u8; 2] = icdf(&[136, 120]);
pub const EXC_SIGN: [&[u8]; 42] = [
&icdf::<2>(&[2, 254]),
&icdf::<2>(&[207, 49]),
&icdf::<2>(&[189, 67]),
&icdf::<2>(&[179, 77]),
&icdf::<2>(&[174, 82]),
&icdf::<2>(&[163, 93]),
&icdf::<2>(&[157, 99]),
&icdf::<2>(&[58, 198]),
&icdf::<2>(&[245, 11]),
&icdf::<2>(&[238, 18]),
&icdf::<2>(&[232, 24]),
&icdf::<2>(&[225, 31]),
&icdf::<2>(&[220, 36]),
&icdf::<2>(&[211, 45]),
&icdf::<2>(&[1, 255]),
&icdf::<2>(&[210, 46]),
&icdf::<2>(&[190, 66]),
&icdf::<2>(&[178, 78]),
&icdf::<2>(&[169, 87]),
&icdf::<2>(&[162, 94]),
&icdf::<2>(&[152, 104]),
&icdf::<2>(&[48, 208]),
&icdf::<2>(&[242, 14]),
&icdf::<2>(&[235, 21]),
&icdf::<2>(&[224, 32]),
&icdf::<2>(&[214, 42]),
&icdf::<2>(&[205, 51]),
&icdf::<2>(&[190, 66]),
&icdf::<2>(&[1, 255]),
&icdf::<2>(&[162, 94]),
&icdf::<2>(&[152, 104]),
&icdf::<2>(&[147, 109]),
&icdf::<2>(&[144, 112]),
&icdf::<2>(&[141, 115]),
&icdf::<2>(&[138, 118]),
&icdf::<2>(&[8, 248]),
&icdf::<2>(&[203, 53]),
&icdf::<2>(&[187, 69]),
&icdf::<2>(&[176, 80]),
&icdf::<2>(&[168, 88]),
&icdf::<2>(&[161, 95]),
&icdf::<2>(&[154, 102]),
];
pub const LBRR_FLAGS: [&[u8]; 2] = [&icdf::<3>(&[53, 53, 150]), &icdf::<7>(&[41, 20, 29, 41, 15, 28, 82])];
crates/audio/src/opus/mod.rs
+44
View File
@@ -0,0 +1,44 @@
//! Clean-room pure-Rust Opus (RFC 6716, updated by RFC 8251; RFC 7845 `OpusHead`).
//!
//! Decoding (feature `opus-decode`): [`Decoder`] (multistream, surround), [`StreamDecoder`] (one
//! elementary stream), [`OpusHead`] and the packet parser [`packet`]. From FilmCraft's
//! `filmcraft-opus` (storytold/filmcraft@8fda2f2).
//!
//! Public items keep their names, so FilmCraft can re-export this module.
// Index loops and long argument lists follow the specification's pseudo-code (FilmCraft allows
// these lints workspace-wide).
#![allow(clippy::needless_range_loop, clippy::too_many_arguments)]
mod decode;
pub use decode::{Decoder, Error, Mode, OpusHead, Packet, Result, SAMPLE_RATES, StreamDecoder, Toc, packet};
/// Coded audio bandwidth (RFC 6716 §2, Table 1).
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum Bandwidth {
/// 4 kHz (8 kHz sampling).
Narrowband,
/// 6 kHz (12 kHz sampling).
Mediumband,
/// 8 kHz (16 kHz sampling).
Wideband,
/// 12 kHz (24 kHz sampling).
SuperWideband,
/// 20 kHz (48 kHz sampling).
Fullband,
}
/// Channel order for Vorbis-ordered surround (mapping family 1, RFC 7845 §5.1.1.2): output channel
/// → decoded channel, giving the WAV/SMPTE order (L R C LFE back… side…) the rest of craft-audio
/// uses. From FilmCraft's `filmcraft-codecs`.
pub fn wav_order(channels: usize) -> Option<&'static [usize]> {
Some(match channels {
3 => &[0, 2, 1],
5 => &[0, 2, 1, 3, 4],
6 => &[0, 2, 1, 5, 3, 4],
7 => &[0, 2, 1, 6, 5, 3, 4],
8 => &[0, 2, 1, 7, 5, 6, 3, 4],
_ => return None,
})
}
crates/audio/src/read/mod.rs
+6 -6
View File
@@ -183,12 +183,12 @@ enum Inner<'a> {
/// A file's audio, decoded block by block: the streaming counterpart of [`decode`], with seeking
/// and track choice.
///
/// Native formats (WAV, AIFF, AAC and PCM in MP4/MOV and Matroska/WebM, and raw ADTS streams)
/// read their sample data from the source packet by packet. The container's packet index is held
/// in memory (tens of bytes per packet), and opening a Matroska file or a raw ADTS stream reads
/// through the whole file to build it; WAV and AIFF need no index. Formats symphonia decodes (MP1/MP2/MP3, FLAC, ALAC,
/// Vorbis, CAF, ADPCM, A-law, µ-law) read the whole source into memory first; their decoded audio
/// still streams.
/// Native formats (WAV, AIFF, AAC, Opus and PCM in MP4/MOV, Matroska/WebM and Ogg, and raw ADTS
/// streams) read their sample data from the source packet by packet. The container's packet index
/// is held in memory (tens of bytes per packet), and opening an Ogg or Matroska file or a raw ADTS
/// stream reads through the whole file to build it; WAV and AIFF need no index. Formats symphonia
/// decodes (MP1/MP2/MP3, FLAC, ALAC, Vorbis, CAF, ADPCM, A-law, µ-law) read the whole source into
/// memory first; their decoded audio still streams.
///
/// ```no_run
/// # fn demo() -> craft_audio::Result<()> {
crates/audio/src/read/native.rs
+90 -10
View File
@@ -1,11 +1,12 @@
//! The native decode path: craft-media's demuxers (ISO BMFF / QuickTime, Matroska / WebM), raw
//! ADTS streams, and WAV / AIFF PCM, feeding [`TrackDecoder`].
//! [`Native::open`] answers `None` for a stream with no native codec (MP1/MP2/MP3, FLAC, ALAC,
//! Vorbis, Opus, ...) or a container these demuxers do not read; the caller then uses symphonia.
//! Vorbis, ...) or a container these demuxers do not read; the caller then uses symphonia.
//!
//! Every container is a [`Packets`] source over a [`Src`], read packet by packet, so a decode
//! streams. Timing follows the containers: MP4 edit lists (AAC priming) and Matroska `CodecDelay` and
//! per-block `DiscardPadding` are applied, so a file decodes to exactly its presentation length.
//! streams. Timing follows the containers: MP4 edit lists (AAC priming, Opus pre-skip), Matroska
//! `CodecDelay` and per-block `DiscardPadding`, and Ogg Opus pre-skip and end trimming (RFC 7845
//! §4) are applied, so a file decodes to exactly its presentation length.
use super::aiff::{self, AiffParse};
use super::id3v2_len;
@@ -58,9 +59,18 @@ enum Packets<'a> {
first_ns: i64,
/// How a packet's place on the decoded timeline is found after a seek.
frames: MkvFrames,
src: Src<'a>,
/// A packet read ahead by a seek, returned next.
pending: Option<Packet>,
},
Ogg {
file: Box<ogg::OggFile>,
src: Src<'a>,
stream: usize,
next: usize,
/// Output frame of each packet's first decoded sample (the pre-skip negative).
starts: Vec<i64>,
},
/// Frames of a raw stream: `(offset, length, decoded frames)`.
Raw { frames: Vec<(u64, u32, u32)>, src: Src<'a>, next: usize },
/// Interleaved PCM in `start..start + frames * frame_bytes` (WAV, AIFF).
@@ -79,6 +89,13 @@ impl Packets<'_> {
file.read_sample(src, *track, i).map(whole).map_err(mp4_error)
})
}
Packets::Ogg { file, src, stream, next, .. } => {
let i = *next;
(i < file.streams.get(*stream).map_or(0, |s| s.packets.len())).then(|| {
*next += 1;
file.read_packet(src, *stream, i).map(whole).map_err(ogg_error)
})
}
Packets::Raw { frames, src, next } => {
let &(at, len, _) = frames.get(*next)?;
*next += 1;
@@ -131,6 +148,12 @@ impl Packets<'_> {
*next = i;
Ok(s)
}
Packets::Ogg { next, starts, .. } => {
let base = starts.first().copied().unwrap_or(0);
let (i, s) = at_or_before(starts.iter().map(|&s| u64::try_from(s.saturating_sub(base)).unwrap_or(0)), frame);
*next = i;
Ok(s)
}
Packets::Raw { frames, next, .. } => {
let mut acc = 0u64;
let starts = frames.iter().map(|&(_, _, n)| {
@@ -146,7 +169,7 @@ impl Packets<'_> {
*next_frame = frame.min(*frames);
Ok(*next_frame)
}
Packets::Mkv { demux, track, rate, first_ns, frames, pending } => {
Packets::Mkv { demux, track, rate, first_ns, frames, pending, src } => {
let ns = i64::try_from(rescale(frame, u64::from(*rate), 1_000_000_000)).unwrap_or(i64::MAX).saturating_add(*first_ns);
let point = demux.seek(*track, ns).map_err(mkv_error)?;
match mkv_next(demux, *track, *rate) {
@@ -154,7 +177,7 @@ impl Packets<'_> {
*pending = Some(p);
// Timestamps are rounded to the file's timestamp scale (often 1 ms), so the
// packet's index places it exactly where packets have a known length.
let by_index = point.sample.and_then(|k| frames.start(k));
let by_index = point.sample.and_then(|k| frames.start(src, k));
Ok(by_index.unwrap_or_else(|| {
rescale_round(u64::try_from(pts_ns.saturating_sub(*first_ns)).unwrap_or(0), 1_000_000_000, u64::from(*rate))
}))
@@ -173,16 +196,35 @@ enum MkvFrames {
Constant(u64),
/// PCM: the stored sizes of the track's packets, and the bytes per frame.
Pcm { sizes: Vec<u32>, frame_bytes: u64 },
/// Opus: each packet's length is in its first bytes (RFC 6716 §3.1); the packets' places and
/// sizes, and their starts once read.
Opus { samples: Vec<(u64, u32)>, starts: Option<Vec<u64>> },
/// From the timestamps.
Timestamps,
}
impl MkvFrames {
/// The decoded-timeline start of packet `k`, when the packets say.
fn start(&self, k: usize) -> Option<u64> {
fn start(&mut self, src: &Src<'_>, k: usize) -> Option<u64> {
match self {
MkvFrames::Constant(n) => Some(k as u64 * *n),
MkvFrames::Pcm { sizes, frame_bytes } => Some(sizes.iter().take(k).map(|&s| u64::from(s)).sum::<u64>() / *frame_bytes),
MkvFrames::Opus { samples, starts } => {
if starts.is_none() {
// A read that fails leaves the starts unknown (the caller then places the
// packet by its timestamp) and is tried again on the next seek.
let mut acc = 0u64;
let mut found = Vec::with_capacity(samples.len());
for &(at, size) in samples.iter() {
let b = src.bytes(at, size.min(2) as usize)?;
let prefix = [b.first().copied().unwrap_or(0), b.get(1).copied().unwrap_or(0)];
found.push(acc);
acc += u64::from(ogg::opus_packet_samples(prefix, size).unwrap_or(0));
}
*starts = Some(found);
}
starts.as_ref()?.get(k).copied()
}
MkvFrames::Timestamps => None,
}
}
@@ -220,6 +262,13 @@ fn mkv_error(e: matroska::Error) -> AudioError {
}
}
fn ogg_error(e: ogg::Error) -> AudioError {
match e {
ogg::Error::Io(e) => io_error(e),
e => AudioError::Malformed(e.to_string()),
}
}
/// A track the native path decodes: its packets, its decoder and its container timing.
pub(crate) struct Native<'a> {
packets: Packets<'a>,
@@ -227,7 +276,7 @@ pub(crate) struct Native<'a> {
info: AudioInfo,
/// Whether `info.frames` came from the container (else it is 0 until decoded).
frames_known: bool,
/// Decoded frames at the start that are not presented (AAC priming).
/// Decoded frames at the start that are not presented (AAC priming, Opus pre-skip).
skip: u64,
/// Presented frames, when the container says.
limit: Option<u64>,
@@ -267,6 +316,7 @@ impl<'a> Native<'a> {
match sniffed.or(detected) {
Some(Container::Wav) if wav::is_wav(&head) => single(open_wav(src)),
Some(Container::Aiff) if aiff::is_aiff(&head) => single(open_aiff(src)),
Some(Container::Ogg) if ogg::sniff(body) => open_ogg(src, track),
Some(c @ (Container::Mp4 | Container::QuickTime)) if head.get(4..8) == Some(b"ftyp") => open_mp4(src, c, track),
Some(c @ (Container::Matroska | Container::WebM)) if body.starts_with(&[0x1A, 0x45, 0xDF, 0xA3]) => open_mkv(src, c, track),
Some(Container::Adts) | None => single(open_adts(src, at)),
@@ -640,7 +690,7 @@ fn open_mp4<'a>(src: &Src<'a>, container: Container, track: Option<usize>) -> Op
// Probe the decoder for the output rate before the timing is worked out.
let probe = TrackDecoder::new(&params, &lookahead.iter().map(Vec::as_slice).collect::<Vec<_>>()).ok()?;
let rate = probe.sample_rate();
// A leading media edit (AAC priming) is skipped, and the edit's duration is the
// A leading media edit (AAC priming, Opus pre-skip) is skipped, and the edit's duration is the
// presentation length. Empty edits and multi-segment lists play the media as it is.
// A leading empty edit (a delay before the track starts, ffmpeg's `-itsoffset` and muxers
// aligning audio to video) does not change which media is presented: the media edit after it
@@ -721,9 +771,10 @@ fn open_mkv<'a>(src: &Src<'a>, container: Container, track: Option<usize>) -> Op
let frame_bytes = u64::from(pcm.bits.div_ceil(8)) * u64::from(params.channels.unwrap_or(1).max(1));
MkvFrames::Pcm { sizes: t.samples.iter().map(|s| s.size).collect(), frame_bytes: frame_bytes.max(1) }
}
_ if params.codec == CodecId::Opus => MkvFrames::Opus { samples: t.samples.iter().map(|s| (s.offset, s.size)).collect(), starts: None },
_ => MkvFrames::Timestamps,
};
// `CodecDelay` (AAC priming) is decoded audio that is not presented.
// `CodecDelay` (AAC priming, Opus pre-skip) is decoded audio that is not presented.
let skip = rescale_round(t.codec_delay_ns, 1_000_000_000, u64::from(rate));
// The presentation ends with the last frame (timestamps, with `CodecDelay` already taken off,
// are in ticks of `timebase` seconds); known only when the last frame has a duration.
@@ -755,7 +806,7 @@ fn open_mkv<'a>(src: &Src<'a>, container: Container, track: Option<usize>) -> Op
_ => 0,
};
demux.seek(track, i64::MIN).ok()?;
let packets = Packets::Mkv { demux: Box::new(demux), track, rate, first_ns, frames: frames_of, pending: None };
let packets = Packets::Mkv { demux: Box::new(demux), track, rate, first_ns, frames: frames_of, src: src.clone(), pending: None };
native(container, packets, &params, &lookahead, skip, frames, frames).map(Ok)
}
@@ -790,6 +841,35 @@ fn tagged_last_duration(tag_ns: u64, last_pts: i64, delay_ticks: i64, packet_tic
(duration > 0 && duration <= packet_ticks).then_some(duration)
}
/// An Ogg file's Opus stream (RFC 7845: pre-skip and end trimming applied).
fn open_ogg<'a>(src: &Src<'a>, track: Option<usize>) -> Option<Result<Native<'a>>> {
let file = match demuxed(ogg::open(src), ogg_failure)? {
Ok(file) => file,
Err(e) => return Some(Err(e)),
};
let stream = match audio_track(file.streams.iter().map(|s| s.codec.codec_id().is_audio()), track)? {
Ok(s) => s,
Err(e) => return Some(Err(e)),
};
let s = file.streams.get(stream)?;
if s.codec != ogg::Codec::Opus {
return None;
}
let params = s.codec_params();
let head = match crate::opus::OpusHead::parse(&params.extradata) {
Ok(head) => head,
// A feature this decoder lacks is left to the fallback; a broken header is reported.
Err(crate::opus::Error::Unsupported(_)) => return None,
Err(e) => return Some(Err(e.into())),
};
let timing = ogg::OpusTiming::of(s, u32::from(head.pre_skip));
let limit = u64::try_from(timing.total).ok();
let skip = u64::from(timing.pre_skip);
let starts = timing.starts.clone();
let packets = Packets::Ogg { file: Box::new(file), src: src.clone(), stream, next: 0, starts };
native(Container::Ogg, packets, &params, &[], skip, limit, limit).map(Ok)
}
/// The audio tracks of `src` the native path reads, described for a track list; `None` when it
/// does not read the container.
pub(crate) fn tracks(src: &Src<'_>, detected: Option<Container>) -> Option<Result<Vec<CodecParams>>> {
crates/audio/src/track.rs
+111 -16
View File
@@ -4,6 +4,8 @@
#[cfg(feature = "aac-decode")]
use crate::aac;
#[cfg(feature = "opus-decode")]
use crate::opus;
use crate::pcm::{PcmKind, deinterleave};
use crate::{AudioBuffer, AudioError, ChannelLayout, CodecId, CodecParams, Result, SampleFormat};
@@ -20,6 +22,11 @@ enum Inner {
/// The format ADTS packets must keep, when their headers gave the configuration.
adts_format: Option<AdtsFormat>,
},
#[cfg(feature = "opus-decode")]
Opus {
dec: Box<opus::Decoder>,
order: Option<&'static [usize]>,
},
Pcm {
kind: PcmKind,
channels: usize,
@@ -31,9 +38,9 @@ enum Inner {
///
/// It takes the track's [`CodecParams`] (from `isobmff::Track::codec_params`,
/// `matroska::Track::codec_params`, `ogg::Stream::codec_params`, or built by hand for another
/// container) and decodes AAC (LC; HE-AAC as its upsampled LC core) and PCM.
/// Container timing (AAC priming, end trimming) is the caller's: every decoded sample is
/// returned.
/// container) and decodes AAC (LC; HE-AAC as its upsampled LC core), Opus and PCM.
/// Container timing (AAC priming, Opus pre-skip, end trimming) is the caller's: every decoded
/// sample is returned.
///
/// ```no_run
/// # fn demo(params: &craft_audio::CodecParams, packets: &[Vec<u8>]) -> craft_audio::Result<()> {
@@ -79,15 +86,19 @@ impl TrackDecoder {
///
/// Streams without codec configuration are accepted where the packets carry it: AAC with
/// empty `extradata` takes its configuration from the ADTS headers of the lookahead (MPEG-TS,
/// raw `.aac`), the one most of them share, and ADTS headers are stripped from every packet. Other missing configuration is
/// [`AudioError::Malformed`].
/// raw `.aac`), the one most of them share, and ADTS headers are stripped from every packet;
/// Opus with empty `extradata` and 1 or 2 `channels` (RTP, MPEG-TS) needs no `OpusHead`. Other
/// missing configuration is [`AudioError::Malformed`].
///
/// Codecs without a native packet decoder yet are [`AudioError::Unsupported`];
/// `craft_audio::decode` decodes MP3, FLAC, ALAC and Vorbis through symphonia when the
/// Codecs without a native packet decoder yet (MP3, FLAC, ALAC, Vorbis) are
/// [`AudioError::Unsupported`]; `craft_audio::decode` decodes them through symphonia when the
/// `symphonia` feature is on.
pub fn new(params: &CodecParams, lookahead: &[&[u8]]) -> Result<TrackDecoder> {
let rate = params.sample_rate.unwrap_or(0);
let channels = usize::from(params.channels.unwrap_or(0));
// Only AAC reads the lookahead; Opus needs none.
#[cfg(not(feature = "aac-decode"))]
let _ = lookahead;
let (inner, codec, sample_rate, layout, sample_format) = match params.codec {
#[cfg(feature = "aac-decode")]
CodecId::Aac | CodecId::HeAac => {
@@ -121,6 +132,22 @@ impl TrackDecoder {
let order = layout.is_ordered().then(|| aac::wav_order(cfg)).flatten();
(Inner::Aac { dec: Box::new(dec), up, order, adts_format }, codec, out, layout, SampleFormat::Compressed)
}
#[cfg(feature = "opus-decode")]
CodecId::Opus => {
// Opus over RTP or in MPEG-TS has no OpusHead: mono and stereo need none (RFC 7845
// mapping family 0), so one is made from the channel count.
let head = match (params.extradata.is_empty(), channels) {
(false, _) => opus::OpusHead::parse(&params.extradata)?,
(true, 1 | 2) => opus::OpusHead::parse(&default_opus_head(channels as u8, rate))?,
(true, _) => {
return Err(AudioError::Malformed("Opus track without an OpusHead (extradata) needs 1 or 2 channels".into()));
}
};
let layout = opus_layout(&head);
let order = if head.mapping_family == 1 { opus::wav_order(head.channels) } else { None };
let dec = opus::Decoder::from_head(head, 48_000)?;
(Inner::Opus { dec: Box::new(dec), order }, CodecId::Opus, 48_000, layout, SampleFormat::Compressed)
}
CodecId::Pcm => {
let pcm = params.pcm.ok_or_else(|| AudioError::Malformed("PCM track without a sample format".into()))?;
let (kind, sample_format) = pcm_kind(pcm.bits, pcm.float, pcm.big_endian, !pcm.signed)
@@ -190,6 +217,11 @@ impl TrackDecoder {
}
reorder(out, *order)
}
#[cfg(feature = "opus-decode")]
Inner::Opus { dec, order } => {
let out = dec.decode(Some(packet))?;
reorder(out, *order)
}
Inner::Pcm { kind, channels } => {
let frames = (packet.len() / (kind.bytes() * *channels).max(1)) as u64;
deinterleave(packet, *kind, *channels, frames)?
@@ -205,14 +237,23 @@ impl TrackDecoder {
}
/// What stands in for a lost or undecodable packet (`packet` when its bytes are at hand):
/// silence as long as the packet would have been (from its frame
/// header, the codec's frame length, or the last packet). Empty when the length is unknown.
/// Opus's loss concealment, or silence, as long as the packet would have been (from its TOC or
/// frame header, the codec's frame length, or the last packet). Empty when the length is
/// unknown.
/// Fails with [`AudioError::TooLarge`](crate::AudioError::TooLarge) when that silence would pass
/// the decode limit (2^30 samples, as [`decode`](Self::decode) has) or cannot be allocated,
/// rather than return less than the packet's length.
pub fn conceal(&mut self, packet: Option<&[u8]>) -> Result<AudioBuffer> {
let data = packet.unwrap_or_default();
let frames = match &mut self.inner {
#[cfg(feature = "opus-decode")]
Inner::Opus { dec, order } => {
let order = *order;
if let Ok(out) = dec.conceal(packet) {
return Ok(AudioBuffer { sample_rate: self.sample_rate, channels: reorder(out, order) });
}
None
}
#[cfg(feature = "aac-decode")]
Inner::Aac { up, .. } => {
let per_unit = aac::FRAME_LEN * if up.is_some() { 2 } else { 1 };
@@ -232,27 +273,31 @@ impl TrackDecoder {
/// The frames every packet decodes to, for codecs whose packets all have one length: AAC
/// (1024, 2048 for HE-AAC; a packet of several ADTS frames decodes to that many times this);
/// `None` for PCM.
/// `None` for Opus and PCM.
pub fn frames_per_packet(&self) -> Option<usize> {
match &self.inner {
#[cfg(feature = "aac-decode")]
Inner::Aac { up, .. } => Some(aac::FRAME_LEN * if up.is_some() { 2 } else { 1 }),
#[cfg(feature = "opus-decode")]
Inner::Opus { .. } => None,
Inner::Pcm { .. } => None,
}
}
/// Frames to decode before a seek target and drop, so the decoder has settled when the target
/// is reached: two AAC frames, none for PCM.
/// is reached: two AAC frames, 80 ms of Opus (RFC 7845 §4.6), none for PCM.
pub fn preroll(&self) -> usize {
match &self.inner {
#[cfg(feature = "aac-decode")]
Inner::Aac { up, .. } => 2 * aac::FRAME_LEN * if up.is_some() { 2 } else { 1 },
#[cfg(feature = "opus-decode")]
Inner::Opus { .. } => 3840,
Inner::Pcm { .. } => 0,
}
}
/// Forget the decoder state, for a seek: the next packet decodes as if it were the first
/// (codecs with overlap, such as AAC, need [`TrackDecoder::preroll`] frames to
/// (codecs with overlap, such as AAC and Opus, need [`TrackDecoder::preroll`] frames to
/// settle).
pub fn reset(&mut self) {
// the packet before a seek says nothing about the length of a packet lost after it
@@ -265,6 +310,8 @@ impl TrackDecoder {
up.reset();
}
}
#[cfg(feature = "opus-decode")]
Inner::Opus { dec, .. } => dec.reset(),
Inner::Pcm { .. } => {}
}
}
@@ -273,6 +320,7 @@ impl TrackDecoder {
fn unsupported(codec: CodecId) -> AudioError {
let feature = match codec {
CodecId::Aac | CodecId::HeAac => Some("aac-decode"),
CodecId::Opus => Some("opus-decode"),
_ => None,
};
match feature {
@@ -287,7 +335,7 @@ fn unsupported(codec: CodecId) -> AudioError {
/// `channels` reordered by `order` (output channel → decoded channel); unchanged when the counts
/// differ.
#[cfg(feature = "aac-decode")]
#[cfg(any(feature = "aac-decode", feature = "opus-decode"))]
fn reorder(mut channels: Vec<Vec<f32>>, order: Option<&[usize]>) -> Vec<Vec<f32>> {
match order {
Some(order) if order.len() == channels.len() => order.iter().map(|&c| channels.get_mut(c).map(std::mem::take).unwrap_or_default()).collect(),
@@ -386,6 +434,16 @@ fn adts_units(frames: Vec<(aac::AdtsHeader, &[u8])>, format: Option<AdtsFormat>)
Ok(frames.into_iter().map(|(_, unit)| unit).collect())
}
/// A mapping-family-0 `OpusHead` (RFC 7845 §5.1) for `channels` (1 or 2) and no pre-skip.
#[cfg(feature = "opus-decode")]
fn default_opus_head(channels: u8, input_rate: u32) -> Vec<u8> {
let mut h = b"OpusHead".to_vec();
h.extend_from_slice(&[1, channels, 0, 0]);
h.extend_from_slice(&(if input_rate > 0 { input_rate } else { 48_000 }).to_le_bytes());
h.extend_from_slice(&[0, 0, 0]);
h
}
/// The sample layout and stored format of PCM samples, `None` for a depth we do not read.
pub(crate) fn pcm_kind(bits: u16, float: bool, big_endian: bool, unsigned: bool) -> Option<(PcmKind, SampleFormat)> {
let bytes = usize::from(bits.div_ceil(8));
@@ -422,6 +480,23 @@ pub(crate) fn aac_layout(channel_config: u8, channels: usize) -> ChannelLayout {
}
}
#[cfg(feature = "opus-decode")]
/// The speakers of an Opus stream (RFC 7845 §5.1.1): Vorbis order for mapping family 1, reordered
/// to WAV order by [`opus::wav_order`]; other families have no speaker assignment.
pub(crate) fn opus_layout(head: &opus::OpusHead) -> ChannelLayout {
match (head.mapping_family, head.channels) {
(0 | 1, 1) => ChannelLayout::MONO,
(0 | 1, 2) => ChannelLayout::STEREO,
(1, 3) => ChannelLayout::SURROUND_3_0,
(1, 4) => ChannelLayout::QUAD,
(1, 5) => ChannelLayout::SURROUND_5_0,
(1, 6) => ChannelLayout::SURROUND_5_1,
(1, 7) => ChannelLayout::SURROUND_6_1,
(1, 8) => ChannelLayout::SURROUND_7_1,
(_, n) => ChannelLayout::unordered(crate::layout::count(n)),
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -443,6 +518,26 @@ mod tests {
assert!(matches!(dec.conceal(None), Err(AudioError::TooLarge(_))));
}
#[cfg(feature = "opus-decode")]
#[test]
fn opus_conceals_a_damaged_packet_at_its_own_length() {
let params = CodecParams::new(CodecId::Opus).with_sample_rate(48_000).with_channels(2);
let mut dec = TrackDecoder::new(&params, &[]).unwrap();
// CELT fullband, 20 ms (config 31); SILK narrowband, 60 ms (config 3); CELT, 10 ms (config 30).
let celt_20ms = [31 << 3, 0xFF, 0xFE, 0x01];
let silk_60ms = [3 << 3, 0x12, 0x34];
let celt_10ms = [30 << 3, 0x12, 0x34];
assert_eq!(dec.decode(&celt_20ms).unwrap().frames(), 960);
assert_eq!(dec.conceal(Some(&silk_60ms)).unwrap().frames(), 2880);
// A packet lost whole is as long as the last one.
assert_eq!(dec.conceal(None).unwrap().frames(), 2880);
// The first packet after a reset has no last packet to go by.
dec.reset();
assert_eq!(dec.conceal(Some(&celt_10ms)).unwrap().frames(), 480);
// Bytes whose framing does not parse fall back to the last length.
assert_eq!(dec.conceal(Some(&[(31 << 3) | 3])).unwrap().frames(), 480);
}
/// One ADTS frame (stereo, 48 kHz, or channel configuration 0 with a PCE) of silence.
#[cfg(feature = "aac-decode")]
fn adts_frame(channel_config: u8) -> Vec<u8> {
@@ -714,12 +809,12 @@ mod tests {
#[test]
fn codecs_without_a_packet_decoder_are_unsupported() {
for codec in [CodecId::Flac, CodecId::Vorbis, CodecId::Alac, CodecId::Mp3, CodecId::Opus, CodecId::H264] {
for codec in [CodecId::Flac, CodecId::Vorbis, CodecId::Alac, CodecId::Mp3, CodecId::H264] {
assert!(matches!(TrackDecoder::new(&CodecParams::new(codec), &[]), Err(AudioError::Unsupported(_))), "{codec:?}");
}
// A codec whose feature is off names the feature.
#[cfg(not(feature = "aac-decode"))]
assert!(matches!(TrackDecoder::new(&CodecParams::new(CodecId::Aac), &[]), Err(AudioError::Unsupported(m)) if m.contains("aac-decode")));
#[cfg(not(feature = "opus-decode"))]
assert!(matches!(TrackDecoder::new(&CodecParams::new(CodecId::Opus), &[]), Err(AudioError::Unsupported(m)) if m.contains("opus-decode")));
// PCM needs its sample format.
assert!(TrackDecoder::new(&CodecParams::new(CodecId::Pcm).with_channels(2).with_sample_rate(48_000), &[]).is_err());
}
crates/audio/tests/codecs/main.rs
+1
View File
@@ -9,5 +9,6 @@ mod testkit;
mod aac;
mod concealment;
mod opus;
mod routing_oracle;
mod track_decoder;
crates/audio/tests/codecs/opus/common.rs
+278
View File
@@ -0,0 +1,278 @@
#![allow(dead_code)]
use std::path::PathBuf;
/// Plain SNR (dB) of `test` against `reference` over the common length.
pub fn snr_db(reference: &[f32], test: &[f32]) -> f64 {
let n = reference.len().min(test.len());
let mut s = 0f64;
let mut e = 0f64;
for i in 0..n {
s += (reference[i] as f64).powi(2);
e += (reference[i] as f64 - test[i] as f64).powi(2);
}
if e == 0.0 { 200.0 } else { 10.0 * (s.max(1e-20) / e).log10() }
}
const NBANDS: usize = 21;
const BANDS: [usize; NBANDS + 1] = [0, 2, 4, 6, 8, 10, 12, 14, 16, 20, 24, 28, 32, 40, 48, 56, 68, 80, 96, 120, 156, 200];
const WIN: usize = 480;
const STEP: usize = 120;
const NFREQ: usize = 240;
/// Calibration: white noise at 48 dB SNR maps to quality 0 (RFC 6716 §6.1).
const CAL: f64 = 0.144;
fn spectra(x: &[f32], channels: usize, ch: usize, rate: u32) -> Vec<[f64; NFREQ]> {
// Work at 48 kHz bin spacing (100 Hz per bin); lower rates use proportionally fewer bins.
let scale = 48000 / rate as usize;
let win = WIN / scale;
let step = STEP / scale;
let n = x.len() / channels;
let nframes = if n >= win { (n - win) / step + 1 } else { 0 };
let w: Vec<f64> = (0..win).map(|i| 0.5 - 0.5 * (2.0 * std::f64::consts::PI * (i as f64 + 0.5) / win as f64).cos()).collect();
let nb = NFREQ / scale;
let mut out = Vec::with_capacity(nframes);
let tw = twiddles(win);
let mut buf = vec![(0f64, 0f64); win];
let mut spec = vec![(0f64, 0f64); win];
let mut scratch = vec![(0f64, 0f64); win];
for f in 0..nframes {
for i in 0..win {
buf[i] = (x[(f * step + i) * channels + ch] as f64 * w[i], 0.0);
}
fft(&buf, 1, &mut spec, &tw, &mut scratch);
let mut ps = [0f64; NFREQ];
for (k, p) in ps.iter_mut().enumerate().take(nb) {
*p = spec[k].0 * spec[k].0 + spec[k].1 * spec[k].1;
}
out.push(ps);
}
out
}
/// `e^(-2πi·j/n)` for `j` in `0..n`: the twiddle factors of every sub-transform of a size-`n` FFT.
fn twiddles(n: usize) -> Vec<(f64, f64)> {
(0..n)
.map(|j| {
let ang = -2.0 * std::f64::consts::PI * j as f64 / n as f64;
(ang.cos(), ang.sin())
})
.collect()
}
/// Recursive mixed-radix DFT (sizes with factors 2, 3, 5) of `x[0], x[xs], x[2·xs], …` into
/// `out`. `tw` is `twiddles` of the top-level size, which `out.len()` divides; `scratch` holds at
/// least `out.len()` values.
fn fft(x: &[(f64, f64)], xs: usize, out: &mut [(f64, f64)], tw: &[(f64, f64)], scratch: &mut [(f64, f64)]) {
let n = out.len();
if n == 1 {
out[0] = x[0];
return;
}
let p = [2usize, 3, 5].into_iter().find(|p| n.is_multiple_of(*p)).unwrap_or(n);
let m = n / p;
let stride = tw.len() / n;
for r in 0..p {
fft(&x[r * xs..], xs * p, &mut out[r * m..(r + 1) * m], tw, scratch);
}
for (k, o) in scratch[..n].iter_mut().enumerate() {
let j = k % m;
let mut acc = (0f64, 0f64);
// e = r·k mod n, kept below n without a division per term.
let mut e = 0;
for r in 0..p {
let v = out[r * m + j];
let (c, s) = tw[e * stride];
acc.0 += v.0 * c - v.1 * s;
acc.1 += v.0 * s + v.1 * c;
e += k;
if e >= n {
e -= n;
}
}
*o = acc;
}
out.copy_from_slice(&scratch[..n]);
}
/// Opus quality metric in the spirit of `opus_compare` (RFC 6716 §6.1): a per-band,
/// masking-weighted spectral distortion between reference and test, mapped so that 100 means
/// identical output and 0 is the pass threshold (white noise at 48 dB SNR). Phase-insensitive
/// (compares power spectra), as the RFC requires tolerance to resampler phase differences.
pub fn opus_quality(reference: &[f32], test: &[f32], channels: usize, rate: u32) -> f64 {
let n = reference.len().min(test.len());
let (reference, test) = (&reference[..n], &test[..n]);
let scale = 48000 / rate as usize;
let mut total_err = 0f64;
let mut count = 0usize;
for ch in 0..channels {
let xr = spectra(reference, channels, ch, rate);
let xt = spectra(test, channels, ch, rate);
let nf = xr.len();
// Band energies of the reference with temporal and frequency spreading (masking).
let mut mask = vec![[0f64; NBANDS]; nf];
for f in 0..nf {
for b in 0..NBANDS {
let (lo, hi) = (BANDS[b] / scale, (BANDS[b + 1] / scale).max(BANDS[b] / scale + 1));
let e: f64 = xr[f][lo..hi.min(NFREQ / scale)].iter().sum::<f64>() / (hi - lo) as f64;
mask[f][b] = e;
}
}
for f in 0..nf {
for b in 1..NBANDS {
mask[f][b] += 0.1 * mask[f][b - 1];
}
for b in (0..NBANDS - 1).rev() {
mask[f][b] += 0.03 * mask[f][b + 1];
}
}
for f in 1..nf {
for b in 0..NBANDS {
mask[f][b] += 0.5 * mask[f - 1][b];
}
}
for f in (0..nf.saturating_sub(1)).rev() {
for b in 0..NBANDS {
mask[f][b] += 0.1 * mask[f + 1][b];
}
}
for f in 0..nf {
let mut ef = 0f64;
let mut nb = 0;
for b in 0..NBANDS {
let (lo, hi) = (BANDS[b] / scale, BANDS[b + 1] / scale);
if hi <= lo {
continue;
}
let m = mask[f][b] + 1e-9 * (WIN * WIN) as f64 / (scale * scale) as f64;
let mut eb = 0f64;
for k in lo..hi {
let d = xt[f][k].sqrt() - xr[f][k].sqrt();
eb += d * d;
}
eb /= (hi - lo) as f64 * m;
ef += eb;
nb += 1;
}
total_err += (ef / nb.max(1) as f64).powi(2);
count += 1;
}
}
let err = (total_err / count.max(1) as f64).sqrt() / CAL;
100.0 * (1.0 - 0.5 * (1.0 + err).ln() / 1.13f64.ln())
}
/// ffmpeg (see `crate::testkit::oracle`); `None` after printing `SKIPPED`.
pub fn ffmpeg() -> Option<PathBuf> {
crate::testkit::ffmpeg_or_skip("opus oracle")
}
pub fn fixtures() -> PathBuf {
crate::testkit::fixtures_dir("opus/oracle")
}
/// Minimal Ogg demuxer: returns (packets, last granule position).
pub fn ogg_packets(data: &[u8]) -> (Vec<Vec<u8>>, i64) {
let mut packets = Vec::new();
let mut cur = Vec::new();
let mut pos = 0;
let mut granule = 0i64;
while pos + 27 <= data.len() && &data[pos..pos + 4] == b"OggS" {
let g = i64::from_le_bytes(data[pos + 6..pos + 14].try_into().unwrap());
let nseg = data[pos + 26] as usize;
let table = &data[pos + 27..pos + 27 + nseg];
let mut body = pos + 27 + nseg;
for &l in table {
cur.extend_from_slice(&data[body..body + l as usize]);
body += l as usize;
if l < 255 {
packets.push(std::mem::take(&mut cur));
}
}
if g >= 0 {
granule = g;
}
pos = body;
}
(packets, granule)
}
pub struct Rng(pub u64);
impl Rng {
pub fn next(&mut self) -> f32 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
((self.0 >> 40) as f32 / (1u64 << 24) as f32) * 2.0 - 1.0
}
}
/// Logarithmic sine sweep 40 Hz -> 20 kHz at 48 kHz.
pub fn sweep(n: usize, amp: f32) -> Vec<f32> {
let (f0, f1) = (40f64, 20000f64);
let t_total = n as f64 / 48000.0;
let k = (f1 / f0).ln();
(0..n)
.map(|i| {
let t = i as f64 / 48000.0;
((2.0 * std::f64::consts::PI * f0 * t_total / k * ((t / t_total * k).exp() - 1.0)).sin() as f32) * amp
})
.collect()
}
/// Music-like: harmonic chords with vibrato changing every 0.5 s, decaying noise bursts.
pub fn music(n: usize, seed: u64) -> Vec<f32> {
let mut r = Rng(seed);
let notes = [220.0f32, 277.2, 329.6, 440.0, 349.2, 523.3, 392.0, 659.3];
(0..n)
.map(|i| {
let t = i as f32 / 48000.0;
let chord = (t * 2.0) as usize;
let mut s = 0.0;
for k in 0..3 {
let f = notes[(chord * 3 + k * 2) % notes.len()] * (1.0 + 0.003 * (2.0 * std::f32::consts::PI * 5.0 * t).sin());
for h in 1..6 {
s += (2.0 * std::f32::consts::PI * f * h as f32 * t).sin() * 0.08 / h as f32;
}
}
let env = (-(t * 2.0).fract() * 3.0).exp();
s * (0.5 + 0.5 * env) + 0.02 * r.next() * env
})
.collect()
}
/// Speech-like: glottal pulses through formant resonators with a syllabic envelope.
pub fn speech(n: usize, seed: u64) -> Vec<f32> {
let mut r = Rng(seed);
let fs = 48000.0f32;
let formants = [(700.0f32, 130.0f32), (1220.0, 70.0), (2600.0, 160.0)];
let mut states = [[0f32; 2]; 3];
let mut phase = 0f32;
let mut out = Vec::with_capacity(n);
for i in 0..n {
let t = i as f32 / fs;
let f0 = 120.0 + 30.0 * (2.0 * std::f32::consts::PI * 0.7 * t).sin();
phase += f0 / fs;
let pulse = if phase >= 1.0 {
phase -= 1.0;
1.0
} else {
0.0
};
let syll = (2.0 * std::f32::consts::PI * 3.0 * t).sin().max(0.0);
let voiced = if (t * 1.3).fract() < 0.8 { syll } else { 0.0 };
let fric = if (t * 1.3).fract() > 0.85 { 0.05 * r.next() } else { 0.0 };
let mut s = 0f32;
for (j, &(f, bw)) in formants.iter().enumerate() {
let rr = (-std::f32::consts::PI * bw / fs).exp();
let a1 = 2.0 * rr * (2.0 * std::f32::consts::PI * f / fs).cos();
let a2 = -rr * rr;
let y = pulse * voiced + a1 * states[j][0] + a2 * states[j][1];
states[j][1] = states[j][0];
states[j][0] = y;
s += y;
}
out.push(s * 0.05 + fric);
}
out
}
crates/audio/tests/codecs/opus/mod.rs
+6
View File
@@ -0,0 +1,6 @@
//! FilmCraft's `filmcraft-opus` integration tests.
mod common;
mod oracle;
mod robustness;
mod vectors;
crates/audio/tests/codecs/opus/oracle.rs
+246
View File
@@ -0,0 +1,246 @@
//! ffmpeg/libopus oracle tests (skipped when ffmpeg with libopus is unavailable).
//!
//! Synthetic signals are encoded with `ffmpeg -c:a libopus` across bitrates, frame sizes,
//! applications, channel layouts and VBR/CBR, decoded by libopus (through ffmpeg, as an external
//! oracle only) to f32, and compared with our decoder: sample-accurate length after pre-skip and
//! end trimming, plus SNR. CELT-only streams are expected to match to floating-point precision;
//! streams containing SILK differ by the (non-normative) SILK resampler, so they are held to a
//! lower SNR bound and the spectral quality metric.
use super::common;
use std::process::Command;
use craft_audio::opus::{Decoder, Mode, OpusHead, Packet};
fn has_libopus(ff: &std::path::Path) -> bool {
Command::new(ff).args(["-hide_banner", "-encoders"]).output().map(|o| String::from_utf8_lossy(&o.stdout).contains("libopus")).unwrap_or(false)
}
struct Case {
name: &'static str,
channels: usize,
signal: fn(usize, u64) -> Vec<f32>,
args: &'static [&'static str],
min_snr_celt: f64,
}
fn sweep_sig(n: usize, _: u64) -> Vec<f32> {
common::sweep(n, 0.5)
}
fn run(case: &Case, rate: u32) -> (f64, f64, Vec<Mode>) {
let ff = common::ffmpeg().unwrap();
let dir = common::fixtures();
let secs = 3;
let n = 48000 * secs;
let mut pcm = vec![0f32; n * case.channels];
for c in 0..case.channels {
let s = (case.signal)(n, 7 + c as u64);
for i in 0..n {
// Decorrelate channels a little (delay + gain) so stereo coding is exercised.
let j = i.saturating_sub(c * 37);
pcm[i * case.channels + c] = s[j] * (1.0 - 0.1 * c as f32);
}
}
let raw = dir.join(format!("{}.f32", case.name));
let opus = dir.join(format!("{}.opus", case.name));
let dec = dir.join(format!("{}.dec.f32", case.name));
std::fs::write(&raw, pcm.iter().flat_map(|v| v.to_le_bytes()).collect::<Vec<u8>>()).unwrap();
let layout = match case.channels {
1 => "mono",
2 => "stereo",
6 => "5.1",
_ => unreachable!(),
};
let mut cmd = Command::new(&ff);
cmd.args(["-hide_banner", "-loglevel", "error", "-y", "-f", "f32le", "-ar", "48000", "-ch_layout", layout, "-i"]).arg(&raw);
cmd.args(["-c:a", "libopus"]).args(case.args).arg(&opus);
assert!(cmd.status().unwrap().success(), "ffmpeg encode failed for {}", case.name);
let st = Command::new(&ff)
.args(["-hide_banner", "-loglevel", "error", "-y", "-c:a", "libopus", "-i"])
.arg(&opus)
.args(["-f", "f32le", "-ar", &rate.to_string()])
.arg(&dec)
.status()
.unwrap();
assert!(st.success());
let reference: Vec<f32> = std::fs::read(&dec).unwrap().as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes(*c)).collect();
let (packets, granule) = common::ogg_packets(&std::fs::read(&opus).unwrap());
let mut d = Decoder::from_head(OpusHead::parse(&packets[0]).unwrap(), rate).unwrap();
assert_eq!(d.channels(), case.channels);
d.set_trim_pre_skip(true);
let mut planar = vec![Vec::new(); case.channels];
let mut modes = Vec::new();
for p in &packets[2..] {
if let Ok(pk) = Packet::parse(p)
&& !modes.contains(&pk.toc.mode)
{
modes.push(pk.toc.mode);
}
let out = d.decode(Some(p)).unwrap();
for (c, o) in out.into_iter().enumerate() {
planar[c].extend(o);
}
}
// End trimming from the final granule position.
let total = ((granule as usize).saturating_sub(d.head().pre_skip as usize)) * rate as usize / 48000;
for p in &mut planar {
assert!(p.len() >= total, "{}: decoded {} < granule length {}", case.name, p.len(), total);
p.truncate(total);
}
// Our output is in Vorbis channel order; ffmpeg reorders 5.1 to FL FR FC LFE BL BR.
let order: Vec<usize> = if case.channels == 6 { vec![0, 2, 1, 5, 3, 4] } else { (0..case.channels).collect() };
let ours: Vec<f32> = (0..total).flat_map(|i| order.iter().map(|&c| planar[c][i]).collect::<Vec<_>>()).collect();
if std::env::var("DIAG").is_ok_and(|v| v == case.name) {
let blk = 12000 * case.channels;
for (b, (r, o)) in reference.chunks(blk).zip(ours.chunks(blk)).enumerate() {
let er: f32 = r.iter().map(|v| v * v).sum();
let eo: f32 = o.iter().map(|v| v * v).sum();
eprintln!("{:5.2}s ref {:9.3e} ours {:9.3e} snr {:6.1}", b as f32 * 0.25, er, eo, common::snr_db(r, o));
}
}
assert_eq!(
ours.len(),
reference.len(),
"{}: length mismatch vs libopus (samples/ch {} vs {})",
case.name,
total,
reference.len() / case.channels
);
let snr = common::snr_db(&reference, &ours);
let q = if case.channels <= 2 { common::opus_quality(&reference, &ours, case.channels, rate) } else { f64::NAN };
let _ = std::fs::remove_file(&raw);
let _ = std::fs::remove_file(&dec);
let _ = std::fs::remove_file(&opus);
(snr, q, modes)
}
#[test]
fn libopus_oracle() {
let Some(ff) = common::ffmpeg() else {
eprintln!("ffmpeg not found; skipping");
return;
};
if !has_libopus(&ff) {
crate::common::skip::<()>("opus oracle: ffmpeg was built without libopus");
return;
}
let cases = [
Case { name: "speech_mono_6k_voip", channels: 1, signal: common::speech, args: &["-b:a", "6k", "-application", "voip"], min_snr_celt: 0.0 },
Case {
name: "speech_mono_12k_voip_60ms",
channels: 1,
signal: common::speech,
args: &["-b:a", "12k", "-application", "voip", "-frame_duration", "60"],
min_snr_celt: 0.0,
},
Case {
name: "speech_mono_16k_voip_40ms",
channels: 1,
signal: common::speech,
args: &["-b:a", "16k", "-application", "voip", "-frame_duration", "40"],
min_snr_celt: 0.0,
},
Case {
name: "speech_stereo_24k_voip",
channels: 2,
signal: common::speech,
args: &["-b:a", "24k", "-application", "voip"],
min_snr_celt: 0.0,
},
Case {
name: "music_stereo_32k_audio",
channels: 2,
signal: common::music,
args: &["-b:a", "32k", "-application", "audio"],
min_snr_celt: 0.0,
},
Case {
name: "music_stereo_64k_audio_10ms",
channels: 2,
signal: common::music,
args: &["-b:a", "64k", "-frame_duration", "10"],
min_snr_celt: 0.0,
},
Case {
name: "music_stereo_96k_lowdelay_2_5ms",
channels: 2,
signal: common::music,
args: &["-b:a", "96k", "-application", "lowdelay", "-frame_duration", "2.5"],
min_snr_celt: 50.0,
},
Case {
name: "sweep_stereo_128k_lowdelay_5ms",
channels: 2,
signal: sweep_sig,
args: &["-b:a", "128k", "-application", "lowdelay", "-frame_duration", "5"],
min_snr_celt: 50.0,
},
Case { name: "music_stereo_160k_cbr", channels: 2, signal: common::music, args: &["-b:a", "160k", "-vbr", "off"], min_snr_celt: 50.0 },
Case { name: "music_mono_256k_20ms", channels: 1, signal: common::music, args: &["-b:a", "256k"], min_snr_celt: 50.0 },
Case { name: "sweep_stereo_510k", channels: 2, signal: sweep_sig, args: &["-b:a", "510k"], min_snr_celt: 50.0 },
Case {
name: "music_stereo_48k_constrained",
channels: 2,
signal: common::music,
args: &["-b:a", "48k", "-vbr", "constrained", "-frame_duration", "40"],
min_snr_celt: 0.0,
},
Case { name: "music_51_256k", channels: 6, signal: common::music, args: &["-b:a", "256k", "-mapping_family", "1"], min_snr_celt: 0.0 },
Case {
name: "speech_51_96k",
channels: 6,
signal: common::speech,
args: &["-b:a", "96k", "-mapping_family", "1", "-application", "lowdelay"],
min_snr_celt: 50.0,
},
];
let mut failures = Vec::new();
for case in &cases {
let (snr, q, modes) = run(case, 48000);
let celt_only = modes.iter().all(|m| *m == Mode::CeltOnly);
let ok = if celt_only { snr >= case.min_snr_celt.max(50.0) } else { snr >= 10.0 && (q.is_nan() || q >= 0.0) };
eprintln!("{:36} modes {:?}: snr {snr:6.1} dB quality {q:6.1} {}", case.name, modes, if ok { "ok" } else { "FAIL" });
if !ok {
failures.push(case.name);
}
}
assert!(failures.is_empty(), "{failures:?}");
}
/// Decoding at 8/12/16/24 kHz against libopus at 48 kHz resampled by ffmpeg. The comparison
/// includes ffmpeg's resampler, so only coarse agreement (quality metric >= 0) is required.
#[test]
fn libopus_oracle_output_rates() {
let Some(ff) = common::ffmpeg() else {
return;
};
if !has_libopus(&ff) {
crate::common::skip::<()>("opus oracle: ffmpeg was built without libopus");
return;
}
let cases = [
Case { name: "rates_music_stereo_64k", channels: 2, signal: common::music, args: &["-b:a", "64k"], min_snr_celt: 0.0 },
Case {
name: "rates_speech_mono_16k",
channels: 1,
signal: common::speech,
args: &["-b:a", "16k", "-application", "voip"],
min_snr_celt: 0.0,
},
];
let mut failures = Vec::new();
for case in &cases {
for rate in [8000u32, 12000, 16000, 24000] {
let (snr, q, modes) = run(case, rate);
let ok = q >= 0.0;
eprintln!("{:28} @{rate:5} modes {:?}: snr {snr:6.1} dB quality {q:6.1} {}", case.name, modes, if ok { "ok" } else { "FAIL" });
if !ok {
failures.push(format!("{}@{rate}", case.name));
}
}
}
assert!(failures.is_empty(), "{failures:?}");
}
crates/audio/tests/codecs/opus/robustness.rs
+167
View File
@@ -0,0 +1,167 @@
//! Garbage, truncated and bit-flipped packets must never panic; concealment must produce
//! well-formed output.
use std::path::PathBuf;
use craft_audio::opus::{Decoder, OpusHead, StreamDecoder};
struct Rng(u64);
impl Rng {
fn next(&mut self) -> u32 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
(self.0 >> 16) as u32
}
}
fn check_finite(out: &[f32]) {
assert!(out.iter().all(|v| v.is_finite()), "non-finite output");
}
#[test]
fn random_garbage_never_panics() {
let mut rng = Rng(0x1234_5678_9abc_def0);
for &rate in &[48000u32, 24000, 16000, 12000, 8000] {
for channels in 1..=2 {
let mut dec = StreamDecoder::new(rate, channels).unwrap();
let mut out = Vec::new();
for i in 0..3000 {
let len = (rng.next() % 400) as usize;
let mut pkt: Vec<u8> = (0..len).map(|_| rng.next() as u8).collect();
if let Some(t) = pkt.first_mut() {
// Cycle through every TOC configuration.
*t = ((i % 32) as u8) << 3 | (*t & 7);
}
out.clear();
let _ = dec.decode_interleaved(Some(&pkt), &mut out);
check_finite(&out);
if i % 17 == 0 {
out.clear();
let _ = dec.decode_interleaved(None, &mut out);
check_finite(&out);
}
}
}
}
}
#[test]
fn multistream_garbage_never_panics() {
let mut h = b"OpusHead".to_vec();
h.extend([1, 6, 0x38, 0x01, 0x80, 0xBB, 0, 0, 0, 0, 1, 4, 2, 0, 4, 1, 2, 3, 5]);
let mut dec = Decoder::new(&h).unwrap();
let mut rng = Rng(42);
for _ in 0..2000 {
let len = (rng.next() % 600) as usize;
let pkt: Vec<u8> = (0..len).map(|_| rng.next() as u8).collect();
if let Ok(out) = dec.decode(Some(&pkt)) {
assert_eq!(out.len(), 6);
for c in &out {
check_finite(c);
}
}
}
let out = dec.decode(None).unwrap();
assert_eq!(out.len(), 6);
}
#[test]
fn concealment_before_and_after_packets() {
let mut dec = Decoder::from_head(OpusHead::simple(2).unwrap(), 48000).unwrap();
// Loss before any packet: silence of a default duration.
let out = dec.decode(None).unwrap();
assert_eq!(out.len(), 2);
assert!(out[0].iter().all(|&v| v == 0.0));
}
/// The RFC 8251 vectors, or `None` after printing `SKIPPED` (a failure with
/// `CRAFT_MEDIA_REQUIRE_ORACLES=1`). `scripts/fetch-opus-vectors.sh` fetches them.
fn vectors_dir() -> Option<PathBuf> {
let d = crate::common::fixture_subdir("opus").join("vectors");
if d.join("testvector01.bit").exists() {
return Some(d);
}
crate::common::skip(&format!("the Opus RFC 8251 vectors are not in {} (scripts/fetch-opus-vectors.sh fetches them)", d.display()))
}
/// Real packets (from the conformance vectors) truncated and bit-flipped, interleaved with losses.
#[test]
fn corrupted_real_packets() {
let Some(dir) = vectors_dir() else {
return;
};
let mut rng = Rng(7);
for v in [1, 2, 5, 12] {
let data = std::fs::read(dir.join(format!("testvector{v:02}.bit"))).unwrap();
let mut dec = StreamDecoder::new(48000, 2).unwrap();
let mut out = Vec::new();
let mut p = 0;
let mut n = 0;
while p + 8 <= data.len() && n < 1500 {
let len = u32::from_be_bytes(data[p..p + 4].try_into().unwrap()) as usize;
p += 8;
let mut pkt = data[p..(p + len).min(data.len())].to_vec();
p += len;
n += 1;
match rng.next() % 4 {
0 => {
let keep = (rng.next() as usize) % (pkt.len() + 1);
pkt.truncate(keep);
}
1 if !pkt.is_empty() => {
for _ in 0..3 {
let i = rng.next() as usize % pkt.len();
pkt[i] ^= 1 << (rng.next() % 8);
}
}
2 => pkt.clear(),
_ => {}
}
out.clear();
let _ = dec.decode_interleaved(if pkt.is_empty() { None } else { Some(&pkt) }, &mut out);
check_finite(&out);
}
}
}
/// An Ogg page holding `packet` whole (RFC 3533), with its CRC.
fn ogg_page(packet: &[u8], flags: u8, granule: u64, seq: u32) -> Vec<u8> {
let mut p = b"OggS\0".to_vec();
p.push(flags);
p.extend_from_slice(&granule.to_le_bytes());
p.extend_from_slice(&1u32.to_le_bytes());
p.extend_from_slice(&seq.to_le_bytes());
p.extend_from_slice(&[0; 4]);
let mut lacing = vec![255u8; packet.len() / 255];
lacing.push((packet.len() % 255) as u8);
p.push(lacing.len() as u8);
p.extend_from_slice(&lacing);
p.extend_from_slice(packet);
let mut crc = 0u32;
for &b in &p {
crc ^= u32::from(b) << 24;
for _ in 0..8 {
crc = if crc & 0x8000_0000 != 0 { (crc << 1) ^ 0x04C1_1DB7 } else { crc << 1 };
}
}
p[22..26].copy_from_slice(&crc.to_le_bytes());
p
}
#[test]
fn ogg_with_a_malformed_opus_head_reports_it() {
// Mapping family 0 with three channels: RFC 7845 §5.1.1 allows at most two.
let mut head = b"OpusHead\x01\x03".to_vec();
head.extend_from_slice(&312u16.to_le_bytes());
head.extend_from_slice(&48_000u32.to_le_bytes());
head.extend_from_slice(&[0, 0, 0]);
let mut file = ogg_page(&head, 2, 0, 0);
file.extend(ogg_page(b"OpusTags\0\0\0\0\0\0\0\0", 0, 0, 1));
// One 20 ms silent CELT frame.
file.extend(ogg_page(&[0xF8, 0xFF, 0xFE], 4, 960, 2));
match craft_audio::decode(&file, Some("opus")) {
Err(craft_audio::AudioError::Malformed(m)) => assert!(m.contains("family 0"), "{m}"),
other => panic!("expected a malformed-header error, got {:?}", other.map(|(info, _)| info)),
}
}
crates/audio/tests/codecs/opus/vectors.rs
+165
View File
@@ -0,0 +1,165 @@
//! RFC 6716 / RFC 8251 conformance vectors.
//!
//! The official vectors (opus_testvectors-rfc8251) are test data downloaded to
//! `target/fixtures/opus/vectors/` by `scripts/fetch-opus-vectors.sh` (never committed). Without
//! them the tests print `SKIPPED`, or fail with `CRAFT_MEDIA_REQUIRE_ORACLES=1` (as in CI).
//! For each vector we check (1) the final range-coder state after every packet against the value
//! stored in the `.bit` file (bit-exact parsing), and (2) the decoded PCM against the reference
//! `.dec` output using our implementation of the opus_compare-style quality metric
//! (see `common::opus_quality`), for stereo (`.dec`) and mono (`m.dec`) output at 48 kHz.
use super::common;
use std::path::PathBuf;
use craft_audio::opus::StreamDecoder;
/// The RFC 8251 vectors, or `None` after printing `SKIPPED` (a failure with
/// `CRAFT_MEDIA_REQUIRE_ORACLES=1`). `scripts/fetch-opus-vectors.sh` fetches them.
fn vectors_dir() -> Option<PathBuf> {
let d = crate::common::fixture_subdir("opus").join("vectors");
if d.join("testvector01.bit").exists() {
return Some(d);
}
crate::common::skip(&format!("the Opus RFC 8251 vectors are not in {} (scripts/fetch-opus-vectors.sh fetches them)", d.display()))
}
fn read_bit(path: &PathBuf) -> Vec<(Vec<u8>, u32)> {
let data = std::fs::read(path).unwrap();
let mut out = Vec::new();
let mut p = 0;
while p + 8 <= data.len() {
let len = u32::from_be_bytes(data[p..p + 4].try_into().unwrap()) as usize;
let rng = u32::from_be_bytes(data[p + 4..p + 8].try_into().unwrap());
p += 8;
if p + len > data.len() {
break;
}
out.push((data[p..p + len].to_vec(), rng));
p += len;
}
out
}
fn read_pcm(path: &PathBuf) -> Vec<f32> {
std::fs::read(path).unwrap().as_chunks::<2>().0.iter().map(|c| i16::from_le_bytes([c[0], c[1]]) as f32 / 32768.0).collect()
}
/// Decodes a vector; returns (interleaved output, index of first range mismatch).
fn decode_vector(packets: &[(Vec<u8>, u32)], rate: u32, channels: usize, no_inv: bool) -> (Vec<f32>, Option<usize>) {
let mut dec = StreamDecoder::new(rate, channels).unwrap();
if no_inv {
dec.set_phase_inversion_disabled(true);
}
let mut out = Vec::new();
let mut mismatch = None;
for (i, (pkt, rng)) in packets.iter().enumerate() {
let r = if pkt.is_empty() { dec.decode_interleaved(None, &mut out) } else { dec.decode_interleaved(Some(pkt), &mut out) };
if r.is_err() {
mismatch.get_or_insert(i);
continue;
}
if !pkt.is_empty() && dec.final_range() != *rng {
mismatch.get_or_insert(i);
}
}
(out, mismatch)
}
#[test]
fn rfc8251_test_vectors() {
let Some(dir) = vectors_dir() else { return };
let mut failures = Vec::new();
for v in 1..=12 {
let name = format!("testvector{v:02}");
let packets = read_bit(&dir.join(format!("{name}.bit")));
for (channels, suffix) in [(2usize, ""), (2usize, "m"), (1usize, "m")] {
let (out, mismatch) = decode_vector(&packets, 48000, channels, suffix == "m");
let mut reference = read_pcm(&dir.join(format!("{name}{suffix}.dec")));
if channels == 1 {
// Mono output is compared against the downmix of the phase-inversion-free reference.
reference = reference.as_chunks::<2>().0.iter().map(|c| 0.5 * (c[0] + c[1])).collect();
}
let q = common::opus_quality(&reference, &out, channels, 48000);
let snr = common::snr_db(&reference, &out);
let pass = mismatch.is_none() && q >= 0.0;
eprintln!(
"{name} {}: range {} quality {q:6.1} snr {snr:6.1} dB len {} / ref {} -> {}",
match (channels, suffix) {
(2, "") => "stereo ",
(2, _) => "stereo-m",
_ => "mono ",
},
mismatch.map_or("ok".to_string(), |i| format!("MISMATCH@{i}")),
out.len(),
reference.len(),
if pass { "PASS" } else { "FAIL" }
);
if !pass {
failures.push(format!("{name}{suffix} ({channels} ch)"));
}
}
}
assert!(failures.is_empty(), "failing vectors: {failures:?}");
}
/// Sanity check of the quality metric calibration: additive white noise at 48 dB SNR sits at the
/// pass threshold (quality ~0), at 70 dB it is clearly above it.
#[test]
fn quality_metric_calibration() {
let Some(dir) = vectors_dir() else {
return;
};
let reference = read_pcm(&dir.join("testvector01.dec"));
let reference = &reference[..reference.len().min(48000 * 2 * 20)];
let p: f64 = reference.iter().map(|v| (*v as f64).powi(2)).sum::<f64>() / reference.len() as f64;
for (snr, lo, hi) in [(48.0f64, -15.0, 15.0), (70.0, 60.0, 101.0)] {
let amp = (p / 10f64.powf(snr / 10.0) * 3.0).sqrt();
let mut seed = 1u32;
let noisy: Vec<f32> = reference
.iter()
.map(|v| {
seed = seed.wrapping_mul(1664525).wrapping_add(1013904223);
v + (((seed >> 8) as f64 / (1u64 << 24) as f64 * 2.0 - 1.0) * amp) as f32
})
.collect();
let q = common::opus_quality(reference, &noisy, 2, 48000);
eprintln!("white noise at {snr} dB SNR: quality {q:.1}");
assert!(q > lo && q < hi, "{snr} dB -> {q}");
}
}
/// Diagnostic (`VEC=n cargo test ... -- --ignored`): per-second SNR/quality with packet modes.
#[test]
#[ignore]
fn vector_diag() {
let Some(dir) = vectors_dir() else {
return;
};
let v: usize = std::env::var("VEC").ok().and_then(|s| s.parse().ok()).unwrap_or(2);
let name = format!("testvector{v:02}");
let packets = read_bit(&dir.join(format!("{name}.bit")));
let mut dec = StreamDecoder::new(48000, 2).unwrap();
let mut out = Vec::new();
let mut lines = Vec::new();
for (pkt, _) in &packets {
let start = out.len() / 2;
if pkt.is_empty() {
dec.decode_interleaved(None, &mut out).unwrap();
lines.push((start, "LOST".to_string()));
} else {
let p = craft_audio::opus::Packet::parse(pkt).unwrap();
dec.decode_interleaved(Some(pkt), &mut out).unwrap();
lines.push((start, format!("{:?} {:?} st{} n{}", p.toc.mode, p.toc.bandwidth, p.toc.stereo as u8, p.frames.len())));
}
}
let reference = read_pcm(&dir.join(format!("{name}.dec")));
let sec = 48000 * 2;
for s in 0..reference.len() / sec {
let r = &reference[s * sec..(s + 1) * sec];
let o = &out[s * sec..(s + 1) * sec];
let mut kinds: Vec<&String> = lines.iter().filter(|(st, _)| *st >= s * 48000 && *st < (s + 1) * 48000).map(|(_, c)| c).collect();
kinds.dedup();
eprintln!("{s:3}s snr {:6.1} q {:6.1} {:?}", common::snr_db(r, o), common::opus_quality(r, o, 2, 48000), kinds);
}
}
crates/audio/tests/codecs/routing_oracle.rs
+59 -5
View File
@@ -1,8 +1,8 @@
//! `craft_audio::decode` routes each container and codec below to the home-grown demuxers and
//! decoders, or to symphonia (FLAC): each pair decodes, sample-aligned, to its presentation length
//! (edit lists, Matroska `CodecDelay` and `DiscardPadding` applied), within an SNR
//! of ffmpeg's decode; truncated and corrupted copies never panic. Fixtures are written by ffmpeg
//! (skipped without it).
//! (edit lists, Matroska `CodecDelay` and `DiscardPadding`, Ogg Opus pre-skip and end trimming
//! applied), within an SNR of ffmpeg's decode; truncated and corrupted copies never panic. Fixtures
//! are written by ffmpeg (skipped without it).
use craft_audio::{ChannelLayout, CodecId, Container, decode, probe};
use std::path::{Path, PathBuf};
@@ -97,6 +97,58 @@ const CASES: &[Case] = &[
frames: Some(240_000),
min_snr: 120.0,
},
Case {
name: "opus.opus",
surround: false,
mono: false,
rate: 48_000,
args: &["-c:a", "libopus", "-b:a", "128k"],
container: Container::Ogg,
codec: CodecId::Opus,
layout: ChannelLayout::STEREO,
channels: 2,
frames: Some(240_000),
min_snr: 120.0,
},
Case {
name: "opus-5.1.opus",
surround: true,
mono: false,
rate: 48_000,
args: &["-c:a", "libopus", "-b:a", "256k", "-mapping_family", "1"],
container: Container::Ogg,
codec: CodecId::Opus,
layout: ChannelLayout::SURROUND_5_1,
channels: 6,
frames: Some(240_000),
min_snr: 120.0,
},
Case {
name: "opus.webm",
surround: false,
mono: false,
rate: 48_000,
args: &["-c:a", "libopus", "-b:a", "96k"],
container: Container::WebM,
codec: CodecId::Opus,
layout: ChannelLayout::STEREO,
channels: 2,
frames: Some(240_000),
min_snr: 120.0,
},
Case {
name: "opus.mp4",
surround: false,
mono: false,
rate: 48_000,
args: &["-c:a", "libopus", "-b:a", "96k"],
container: Container::Mp4,
codec: CodecId::Opus,
layout: ChannelLayout::STEREO,
channels: 2,
frames: Some(240_000),
min_snr: 120.0,
},
Case {
name: "pcm-s24be.mov",
surround: false,
@@ -293,7 +345,9 @@ fn streaming_and_seeking_match_the_whole_file_decode() {
assert_eq!(after[0].len(), whole.frames() - target, "{}: frames after a seek to {target}", c.name);
let expected: Vec<Vec<f32>> = whole.channels.iter().map(|ch| ch[target..].to_vec()).collect();
let s = snr_planar(&after, &expected);
assert!(s == f64::INFINITY, "{}: SNR {s:.1} dB after a seek to {target}", c.name);
// Opus rebuilds its predictor state from the pre-roll.
let bound = if c.codec == CodecId::Opus { c.min_snr.min(25.0) } else { f64::INFINITY };
assert!(s >= bound, "{}: SNR {s:.1} dB after a seek to {target}", c.name);
}
}
}
@@ -305,7 +359,7 @@ fn track_decoder_on_demuxed_packets_matches_decode() {
use craft_audio::TrackDecoder;
use craft_media::isobmff;
let ff = require_ffmpeg!();
for name in ["aac.m4a", "aac-5.1.m4a", "pcm-s24be.mov"] {
for name in ["aac.m4a", "aac-5.1.m4a", "opus.mp4", "pcm-s24be.mov"] {
let c = CASES.iter().find(|c| c.name == name).unwrap();
let Some(path) = fixture(&ff, c) else { continue };
let bytes = std::fs::read(&path).unwrap();
scripts/fetch-opus-vectors.sh
Executable
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Fetch the Opus RFC 8251 conformance vectors (opus_testvectors-rfc8251, 75 MB) into DIR
# (default target/fixtures/opus/vectors), checked against their SHA-256. craft-audio's Opus
# conformance test reads them from there; they are test data, never committed.
#
# The archive is fetched in parallel byte ranges, and a range whose connection slows to a crawl is
# dropped and fetched again: opus-codec.org throttles some connections to a few KB/s.
set -euo pipefail
URL=https://opus-codec.org/static/testvectors/opus_testvectors-rfc8251.tar.gz
SHA256=6b26a22f9ba87b2b836906a9bb7afec5f8e54d49553b1200382520ee6fedfa55
DEST=${1:-target/fixtures/opus/vectors}
if [ -f "$DEST/testvector01.bit" ] && [ -f "$DEST/testvector12m.dec" ]; then
echo "Opus vectors already in $DEST"
exit 0
fi
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
len=$(curl -fsSIL --retry 10 --retry-delay 2 --retry-all-errors --connect-timeout 20 --max-time 60 "$URL" | tr -d '\r' | awk 'tolower($1) == "content-length:" { n = $2 } END { print n }')
[ -n "$len" ] || { echo "could not read the archive size" >&2; exit 1; }
# Small ranges, eight at a time: a range whose connection drops below 50 KB/s for 15 s is
# abandoned and fetched again, which costs little for a 2 MB range.
chunk=2000000
ranges=()
for ((start = 0; start < len; start += chunk)); do
end=$((start + chunk - 1))
((end < len)) || end=$((len - 1))
ranges+=("$start-$end")
done
fetch() { # fetch RANGE PART
curl -fsS --retry 20 --retry-delay 1 --retry-all-errors --speed-limit 50000 --speed-time 15 -r "$1" -o "$2" "$URL"
}
export -f fetch
export URL
for i in "${!ranges[@]}"; do
printf '%s %s/part.%04d\n' "${ranges[$i]}" "$tmp" "$i"
done | xargs -P 8 -n 2 bash -c 'fetch "$0" "$1"' || true
# Fetch again any range that came back short, one at a time.
for attempt in 1 2 3 4 5; do
short=0
for i in "${!ranges[@]}"; do
range=${ranges[$i]}
want=$((${range#*-} - ${range%-*} + 1))
part=$(printf '%s/part.%04d' "$tmp" "$i")
if [ "$(wc -c < "$part" 2> /dev/null || echo 0)" -ne "$want" ]; then
short=1
fetch "$range" "$part" || true
fi
done
[ "$short" -eq 0 ] && break
done
cat "$tmp"/part.* > "$tmp/vectors.tar.gz"
if command -v sha256sum > /dev/null; then
echo "$SHA256 $tmp/vectors.tar.gz" | sha256sum -c -
else
echo "$SHA256 $tmp/vectors.tar.gz" | shasum -a 256 -c -
fi
mkdir -p "$DEST"
tar xzf "$tmp/vectors.tar.gz" -C "$DEST" --strip-components=1
echo "Opus vectors in $DEST"