image-codecs: decode AVIF pixels with the pure-Rust AV1 decoder (#778) (#1601)

* image-codecs: decode AVIF pixels with craft-video's pure-Rust AV1 decoder

New optional `avif-decode` feature (native targets) reads still AVIF
files: the HEIF/ISOBMFF item graph, grids, clap/irot/imir, alpha
auxiliary images (including premultiplied), 8/10/12-bit 4:0:0, 4:2:0,
4:2:2 and 4:4:4, nclx/ICC colour, Exif and XMP, and PQ/HLG to linear
F32 through cicp::linearize as the HEIF path does.

The container walk follows LightCraft #3017 by @emmertex. Every size is
checked against the caller's Limits before the AV1 decoder runs, the
decoder's memory budget is capped by max_alloc, and the decode is
wrapped in catch_unwind so a decoder panic becomes an error.

Fixtures are tiny ffmpeg/libaom files; the 4:2:0 planes match libdav1d
bit for bit. Adds a decode_avif fuzz target.

Part of storytold/craft-libs#778.

* image-codecs, video: address review on the AVIF decoder

- clap: width and height are unsigned fractions, the offsets have signed
  numerators, so 0x80000000/0x40000000 is a width of 2, not -2.
- An item with clap, irot or imir more than once (repeated ipma records)
  is refused before any canvas work, instead of rotating 261120 times.
- Matrix coefficients 10 (BT.2020 constant luminance) is refused rather
  than converted with the non-constant-luminance equations.
- An alpha plane whose own transforms differ from the colour's is refused
  rather than composed in the wrong place; none, or the colour's, still
  decode.
- craft-video: Decoder::set_shown_budget lowers the per-chunk byte cap on
  shown pictures (never above 1 GiB), and the AVIF decoder sets it to
  max_alloc, so a payload of many shown pictures is refused before they
  are built.
This commit is contained in:
FredApps authored and GitHub committed 2026-11-15 23:25:25 +01:00
1 parent 9b5b517b42
commit 59b4f1e8c2
16 files changed
+1488 -39

No files matched your search

+4
View File
@@ -15,6 +15,9 @@ corpus = []
default = []
avif = ["image/avif", "dep:ravif", "dep:rav1e", "dep:avif-serialize"]
heif = ["dep:craft-heif"]
# AVIF decode: craft-video's clean-room pure-Rust AV1 decoder. Native targets only (on wasm32 a
# decoder panic aborts instead of becoming an error), so wasm32 builds leave AVIF unreadable.
avif-decode = ["dep:craft-video", "craft-video/av1-decode"]
[dependencies]
crc32fast = "1.5"
@@ -41,6 +44,7 @@ hayro-jpeg2000 = { version = "=0.3.5", default-features = false, features = ["st
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
rayon = { workspace = true }
craft-video = { workspace = true, optional = true }
exr = { version = "1.73", default-features = false, features = ["rayon"] }
[dev-dependencies]
+19 -8
View File
@@ -1,9 +1,10 @@
# craft-image-codecs
Codecs for flat raster image formats, written in pure Rust. This crate defines its own `Image`
type and shares its baseline and extended JPEG decoders through `craft-jpeg`, and its optional
HEIF decoder through `craft-heif`. It builds for `wasm32-unknown-unknown` and does no filesystem
I/O: decoders take `&[u8]` and encoders return `Vec<u8>`.
type and shares its baseline and extended JPEG decoders through `craft-jpeg`, its optional HEIF
decoder through `craft-heif`, and its optional AVIF decoder through `craft-video`'s AV1 decoder.
It builds for `wasm32-unknown-unknown` and does no filesystem I/O: decoders take `&[u8]` and
encoders return `Vec<u8>`.
It follows the "Avoiding GIMP's hole" rules in `docs/architecture.md` §1.1:
@@ -57,7 +58,7 @@ craft-image-codecs = { git = "https://github.com/storytold/craft-libs", rev = "<
| DICOM | yes | **no** | U8, U16 | Gray, RGB | no | no | no | no | no | no | n/a | built in |
| Amiga IFF (ILBM/PBM) | yes | yes (24-bit ILBM) | U8 | RGB, RGBA | yes (1-bit mask plane on write) | no | no | no | no | no | no | built in |
| JPEG-LS (`.jls`) | yes (any NEAR; one scan per component) | yes (lossless) | U8, U16 | Gray, RGB | no | no | no | no | no | no | no | built in |
| AVIF | **no** | only with feature `avif` | U8 | RGB, RGBA | yes | no | no | no | no | no | yes | `image` + `ravif` |
| AVIF | with feature `avif-decode` (native only) | only with feature `avif` | U8, U16 (10/12-bit decodes to U16; PQ/HLG to linear F32) | RGB, RGBA | yes (auxiliary alpha) | read (`colr` prof/rICC) | read | read | no | no | yes | `craft-video` AV1 decoder; `image` + `ravif` to write |
| HEIF/HEIC | with feature `heif` | **no** | U8, U16 (10/12-bit decodes to U16) | RGB, RGBA | yes (auxiliary alpha) | yes (`colr` prof) | yes | yes | no | no | n/a | `heic-rs` |
"Native" means the data is stored and read back without conversion. Anything else is converted by
@@ -178,10 +179,20 @@ embedded table (and a PNG's transparent entry) so `photocraft-io` can open them
decodes to RGB. Image sequences, overlays and identity derivations, multilayer HEVC and some
4:2:2/4:4:4 streams return `CodecError::Unsupported` or `Malformed`, never wrong pixels. Writing
needs an HEVC encoder and every mature one is C, so HEIF is listed in `ASYMMETRIC_EXCEPTIONS`.
* **AVIF.** Encoding uses `ravif`, which is pure Rust. Decoding
needs `dav1d`, which is C. AVIF is therefore read-unsupported, and write support is gated
behind the non-default `avif` feature. In a default build it is neither readable nor writable,
so the symmetric guarantee holds. It is listed in `ASYMMETRIC_EXCEPTIONS`.
* **AVIF.** Encoding uses `ravif`, which is pure Rust, behind the non-default `avif` feature.
Decoding (`codecs::avif_decode`, behind the non-default `avif-decode` feature) reads the HEIF
boxes itself and decodes the AV1 pictures with `craft-video`'s clean-room pure-Rust AV1 decoder:
one `av01` image or a `grid`, 8/10/12-bit, 4:2:0/4:2:2/4:4:4/monochrome, the `nclx` colour (or
the sequence header's) and an ICC profile, an `auxl` alpha plane (`prem` too), the Exif and XMP
items, and `clap`/`irot`/`imir`, which are applied unless `keep_orientation` is set, as for HEIF.
PQ and HLG open as linear light in F32, as HEIF does. Image sequences (animated AVIF), depth and
gain-map auxiliary images and layered progressive images aren't read. It is bounded like the
other decoders: the `Limits` are checked against the declared size before any picture decodes,
against every picture and every output buffer, `max_alloc` caps the AV1 decoder's picture store,
box, item and extent counts are capped, item data can't exceed the file, and a decoder panic is
`CodecError::Malformed`. wasm32 builds leave AVIF unreadable, since a panic there aborts. In a
default build AVIF is neither readable nor writable, so the symmetric guarantee holds; a build
with only one of the two features is listed in `ASYMMETRIC_EXCEPTIONS`.
* **Lossy WebP.** Lossless (VP8L, via `image-webp`) is the default. `webp_lossless: false`
writes a lossy file with our own clean-room VP8 key-frame encoder (`codecs::vp8`, from
RFC 6386; `webp_quality` 0–100 on libwebp's scale sets the quantizer). Alpha travels losslessly
+8 -1
View File
@@ -14,7 +14,7 @@ cargo-fuzz = true
[dependencies]
libfuzzer-sys = "0.4"
craft-image-codecs = { path = "..", features = ["heif"] }
craft-image-codecs = { path = "..", features = ["heif", "avif-decode"] }
# Called directly too: the codec turns its panics into errors (see decode_heif.rs).
heic-rs = "=0.1.4"
@@ -76,3 +76,10 @@ path = "fuzz_targets/decode_webp.rs"
test = false
doc = false
bench = false
[[bin]]
name = "decode_avif"
path = "fuzz_targets/decode_avif.rs"
test = false
doc = false
bench = false
@@ -0,0 +1,14 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use craft_image_codecs::{decode_as_with, probe_size, DecodeOptions, Format, Limits};
// The AVIF box reader and craft-video's AV1 decoder parse untrusted files: run with `-timeout=10`
// so a hang is reported, not just a panic. The codec turns an AV1 decoder panic into an error, so
// only the box reader's own panics would reach the fuzzer. Both orientation paths are covered.
fuzz_target!(|data: &[u8]| {
let keep_orientation = data.first().is_some_and(|b| b & 1 == 1);
let limits = Limits { max_width: 4096, max_height: 4096, max_pixels: 1 << 22, max_alloc: 256 << 20 };
let _ = probe_size(data);
let _ = decode_as_with(Format::Avif, data, &DecodeOptions { limits, keep_orientation });
});
+1 -1
View File
@@ -99,7 +99,7 @@ pub fn linear_to_pq(linear: f32) -> f32 {
/// Relative luminance weights of the RGB primaries `code` (BT.2020's for anything else, the
/// space BT.2100 defines HLG in).
fn luma_weights(code: u16) -> [f32; 3] {
pub(crate) fn luma_weights(code: u16) -> [f32; 3] {
match code {
1 => [0.2126, 0.7152, 0.0722],
12 => [0.2290, 0.6917, 0.0793],
+2
View File
@@ -1,4 +1,6 @@
pub(crate) mod avif;
#[cfg(all(feature = "avif-decode", not(target_arch = "wasm32")))]
pub(crate) mod avif_decode;
pub(crate) mod dds;
pub(crate) mod dds_bc;
pub(crate) mod deep_exr;
File diff suppressed because it is too large. Load diff
+9 -4
View File
@@ -26,7 +26,8 @@ pub enum Format {
OpenExr,
/// Radiance RGBE.
Hdr,
/// Encode-only behind the `avif` feature; see [`ASYMMETRIC_EXCEPTIONS`].
/// Encoded behind the `avif` feature and decoded behind `avif-decode` (native targets); see
/// [`ASYMMETRIC_EXCEPTIONS`] for a build with only one of them.
Avif,
/// HEIF/HEIC (HEVC-coded): read-only; see [`ASYMMETRIC_EXCEPTIONS`].
Heif,
@@ -91,8 +92,10 @@ pub const ASYMMETRIC_EXCEPTIONS: &[(Format, &str)] = &[
),
(
Format::Avif,
"AVIF encode uses ravif (pure Rust) but decoding requires dav1d (C); read stays unsupported \
until a pure-Rust AV1 decoder is viable. Only enabled with the non-default `avif` feature.",
"AVIF encode (ravif and rav1e, pure Rust) is behind the non-default `avif` feature and AVIF \
decode (craft-video's clean-room pure-Rust AV1 decoder) behind the non-default \
`avif-decode` feature, native targets only (on wasm32 a decoder panic would abort). A build \
with only one of them reads or writes AVIF, not both.",
),
(Format::Jxl, "Lossless JPEG XL export uses the pure-Rust zune-jpegxl encoder; JPEG XL reading is tracked separately and is not yet available."),
(
@@ -304,7 +307,9 @@ pub fn caps(format: Format) -> FormatCaps {
Format::Qoi => base,
Format::OpenExr => FormatCaps { depths: &[S::F16, S::F32], layouts: RGB_GRAY, ..base },
Format::Hdr => FormatCaps { depths: &[S::F32], layouts: &[L::Rgb], alpha: false, lossy: true, ..base },
Format::Avif => FormatCaps { read: false, write: cfg!(feature = "avif"), lossy: true, ..base },
Format::Avif => {
FormatCaps { read: cfg!(all(feature = "avif-decode", not(target_arch = "wasm32"))), write: cfg!(feature = "avif"), lossy: true, ..base }
}
Format::Heif => {
FormatCaps { read: cfg!(feature = "heif"), write: false, depths: &[S::U8, S::U16], icc: true, exif: true, xmp: true, lossy: true, ..base }
}
+17 -7
View File
@@ -110,7 +110,11 @@ pub fn decode_as_with(format: Format, bytes: &[u8], opts: &DecodeOptions) -> Res
Format::Heif => heif::decode(bytes, l, opts.keep_orientation),
Format::Jp2 => jp2::decode(bytes, l),
Format::Jxl => Err(CodecError::unsupported(format, "JPEG XL reading is not yet available; export is supported")),
Format::Gif | Format::Bmp | Format::Tga | Format::Ico | Format::Qoi | Format::Hdr | Format::Avif => via_image::decode(format, bytes, l),
#[cfg(all(feature = "avif-decode", not(target_arch = "wasm32")))]
Format::Avif => codecs::avif_decode::decode(bytes, l, opts.keep_orientation),
#[cfg(not(all(feature = "avif-decode", not(target_arch = "wasm32"))))]
Format::Avif => via_image::decode(format, bytes, l),
Format::Gif | Format::Bmp | Format::Tga | Format::Ico | Format::Qoi | Format::Hdr => via_image::decode(format, bytes, l),
}?;
finish_format(format, bytes, img, opts)
}
@@ -167,10 +171,11 @@ fn finish_format(format: Format, bytes: &[u8], mut img: Image, opts: &DecodeOpti
}
// Turn the pixels upright, like Photoshop: a TIFF records it in the decoded page's own
// directory, the others in their EXIF block. The metadata is rewritten to Orientation = 1
// on the way. HEIF keeps it in its container, and its decoder has already applied it.
// on the way. HEIF and AVIF keep it in their container, and their decoders have already
// applied it.
let o = match format {
Format::Tiff => tiff::orientation(bytes, None),
Format::Heif => 1,
Format::Heif | Format::Avif => 1,
Format::Dpx => dpx::orientation(bytes),
Format::Cineon => dpx::cineon_orientation(bytes),
_ => img.meta.exif.as_deref().map_or(1, exif_orientation),
@@ -199,10 +204,11 @@ pub fn decode_png_rows<E: From<CodecError>>(
}
/// The upright pixel size [`decode`] would return, read from the file's header without decoding
/// pixels, for WebP, OpenEXR (flat) and the GIF/BMP/TGA/ICO/QOI/HDR family. It runs the same
/// pixels, for WebP, OpenEXR (flat), AVIF (with `avif-decode`) and the GIF/BMP/TGA/ICO/QOI/HDR family. It runs the same
/// header reads and size checks as the decoder (default [`Limits`]) and swaps the size for an
/// EXIF orientation of 5–8, so wherever [`decode`] succeeds the two agree. Other formats (and
/// deep EXR or AVIF) return [`CodecError::Unsupported`]: decode those to learn their size.
/// EXIF orientation of 5–8 (AVIF: its container's `clap` and `irot`), so wherever [`decode`]
/// succeeds the two agree. Other formats (and deep EXR) return [`CodecError::Unsupported`]: decode
/// those to learn their size.
pub fn probe_size(bytes: &[u8]) -> Result<(u32, u32), CodecError> {
let format = detect(bytes).ok_or(CodecError::UnknownFormat)?;
if !caps(format).read {
@@ -212,7 +218,11 @@ pub fn probe_size(bytes: &[u8]) -> Result<(u32, u32), CodecError> {
let (w, h, exif) = match format {
Format::WebP => webp::probe(bytes, l)?,
Format::OpenExr => exr::probe(bytes, l).map(|(w, h)| (w, h, None))?,
Format::Gif | Format::Bmp | Format::Tga | Format::Ico | Format::Qoi | Format::Hdr | Format::Avif => via_image::probe(format, bytes, l)?,
#[cfg(all(feature = "avif-decode", not(target_arch = "wasm32")))]
Format::Avif => codecs::avif_decode::probe(bytes, l).map(|(w, h)| (w, h, None))?,
#[cfg(not(all(feature = "avif-decode", not(target_arch = "wasm32"))))]
Format::Avif => via_image::probe(format, bytes, l)?,
Format::Gif | Format::Bmp | Format::Tga | Format::Ico | Format::Qoi | Format::Hdr => via_image::probe(format, bytes, l)?,
_ => return Err(CodecError::unsupported(format, "no header-only size probe")),
};
let o = exif.as_deref().map_or(1, exif_orientation);
+12
View File
@@ -0,0 +1,12 @@
# AVIF decode fixtures
Small AVIF files from an independent encoder (libaom through FFmpeg 7.1), for `src/codecs/avif_decode.rs`.
- `quad-source.rgb`: the 33×17 source, raw RGB24 four-colour quadrants (red, green, blue, near-white).
- `quad-420-bt709.avif`: 8-bit 4:2:0, limited range, BT.709 matrix, odd size (FFmpeg leaves the primaries and transfer unspecified):
`ffmpeg -f rawvideo -pix_fmt rgb24 -s 33x17 -i quad-source.rgb -c:v libaom-av1 -crf 12 -still-picture 1 -pix_fmt yuv420p -colorspace bt709 -color_primaries bt709 -color_trc iec61966-2-1 -color_range tv quad-420-bt709.avif`
- `quad-420-bt709.dav1d.yuv`: that file's AV1 picture as libdav1d decodes it, planar 8-bit 4:2:0 (Y, then Cb, then Cr):
`ffmpeg -c:v libdav1d -i quad-420-bt709.avif -f rawvideo -pix_fmt yuv420p quad-420-bt709.dav1d.yuv`
- `quad-444-gbr-lossless.avif`: lossless 8-bit 4:4:4 with the identity (GBR) matrix, full range:
`ffmpeg -f rawvideo -pix_fmt rgb24 -s 33x17 -i quad-source.rgb -c:v libaom-av1 -crf 0 -still-picture 1 -pix_fmt gbrp -colorspace rgb -color_range pc quad-444-gbr-lossless.avif`.
It decodes to `quad-source.rgb` exactly.
Binary file not shown.

After

Width:  |  Height:  |  Size: 369 B

@@ -0,0 +1 @@
OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳OOOOOOOOOOOOOOOOO鋳鋳鋳鋳鋳鋳鋳鋳HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙HHHHHHHHHHHHHHHHH゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙゙ppppppppbUUUUUUUbppppppppbUUUUUUUbppppppppbUUUUUUUbmmmmmmmmbUUUUUUUb����r_______艦ヘヘヘヘヘヘヘヲ===※ノノノノノノノノヲ===※ノノノノノノノノヲ===※ノノノノノノノノヲ===¢ララララララララ�6666666�ラララララララ�6666666�ラララララララ�6666666�ロロロロロロロ�6666666�セセセセセセセォHHHHHHghhhhhhhhv===」llllllllv===」llllllllv===」llllllllv===-
Binary file not shown.

After

Width:  |  Height:  |  Size: 345 B

@@ -0,0 +1 @@
ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(ж(И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<И<<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ь<Ьрррррррррррррррррррррррррррррррррррррррррррррррр
+8 -3
View File
@@ -70,10 +70,15 @@ fn default_build_formats_all_symmetric() {
}
#[test]
fn avif_never_readable() {
assert!(!caps(Format::Avif).read);
fn avif_is_readable_only_with_the_decode_feature() {
assert_eq!(caps(Format::Avif).read, cfg!(all(feature = "avif-decode", not(target_arch = "wasm32"))));
assert_eq!(caps(Format::Avif).write, cfg!(feature = "avif"));
assert!(matches!(decode_as(Format::Avif, b"\0\0\0\x1cftypavif"), Err(CodecError::Unsupported { .. })));
let header = decode_as(Format::Avif, b"\0\0\0\x1cftypavif");
if caps(Format::Avif).read {
assert!(matches!(header, Err(CodecError::Malformed { .. })), "{header:?}");
} else {
assert!(matches!(header, Err(CodecError::Unsupported { .. })), "{header:?}");
}
}
#[test]
+42 -15
View File
@@ -239,6 +239,9 @@ pub struct Decoder {
/// Bytes of the pictures the chunk being parsed has shown so far, and whether one was refused for size.
shown_bytes: usize,
shown_overflow: bool,
/// Most bytes of pictures one chunk may show: [`MAX_SHOWN_BYTES_PER_CHUNK`] unless a caller
/// lowered it with [`Decoder::set_shown_budget`].
shown_budget: usize,
lf_deltas: LoopFilterDeltas,
seg_features: SegmentationFeatures,
prev_gm_params: [[i32; 6]; 8],
@@ -301,8 +304,8 @@ fn shown_within_limit(n: usize) -> bool {
/// copies of one 4K reference (about 6.4 GB) from a few hundred bytes of show_existing_frame headers.
const MAX_SHOWN_BYTES_PER_CHUNK: usize = 1 << 30;
fn shown_bytes_within_limit(bytes: usize) -> bool {
bytes <= MAX_SHOWN_BYTES_PER_CHUNK
fn shown_bytes_within_limit(bytes: usize, budget: usize) -> bool {
bytes <= budget.min(MAX_SHOWN_BYTES_PER_CHUNK)
}
/// Bytes of the `u16` sample planes of a `w` × `h` picture.
@@ -313,14 +316,14 @@ fn picture_bytes(w: usize, h: usize, c: &crate::av1::decode::header::ColorConfig
}
/// Add a shown picture to a chunk's outputs, refusing the chunk (and dropping everything it showed)
/// before the list can grow past [`MAX_SHOWN_PER_CHUNK`].
fn push_shown(out: &mut Vec<(Output, usize, u32)>, shown_bytes: &mut usize, item: (Output, usize, u32)) -> Result<()> {
/// before the list can grow past [`MAX_SHOWN_PER_CHUNK`] or its pictures past `budget` bytes.
fn push_shown(out: &mut Vec<(Output, usize, u32)>, shown_bytes: &mut usize, budget: usize, item: (Output, usize, u32)) -> Result<()> {
if !shown_within_limit(out.len() + 1) {
out.clear();
return Err(Error::Invalid("too many pictures shown by one chunk"));
}
let bytes = shown_bytes.saturating_add(item.0.bytes());
if !shown_bytes_within_limit(bytes) {
if !shown_bytes_within_limit(bytes, budget) {
out.clear();
return Err(Error::Invalid("pictures shown by one chunk are too large"));
}
@@ -349,6 +352,14 @@ impl Decoder {
self.memory_budget = bytes;
}
/// Cap the bytes of the pictures one `decode` call may return together, below the built-in
/// 1 GiB (a larger value keeps that). The pictures are refused before they are built, so a
/// caller that keeps only one of them (an AVIF still) can hold a stream to its own memory
/// budget instead of collecting up to 256 pictures first.
pub fn set_shown_budget(&mut self, bytes: u64) {
self.shown_budget = usize::try_from(bytes).unwrap_or(usize::MAX).min(MAX_SHOWN_BYTES_PER_CHUNK);
}
/// A decoder using all available cores (with the `threads` feature).
pub fn new() -> Decoder {
Decoder::with_threads(default_threads())
@@ -369,6 +380,7 @@ impl Decoder {
ref_color: Default::default(),
shown_bytes: 0,
shown_overflow: false,
shown_budget: MAX_SHOWN_BYTES_PER_CHUNK,
lf_deltas: LoopFilterDeltas::defaults(),
seg_features: SegmentationFeatures::default(),
prev_gm_params: default_gm_params(),
@@ -564,20 +576,20 @@ impl Decoder {
let mut r = BitReader::new(payload);
let shown = self.frame_header(&mut r, temporal_id, spatial_id)?;
if let Some(o) = shown {
push_shown(out, &mut self.shown_bytes, (o, tu, spatial_id))?;
push_shown(out, &mut self.shown_bytes, self.shown_budget, (o, tu, spatial_id))?;
continue;
}
if obu_type == OBU_FRAME_T {
r.byte_align();
let rest = payload.get(r.byte_pos()..).ok_or(Error::Truncated)?;
if let Some(o) = self.tile_group(rest)? {
push_shown(out, &mut self.shown_bytes, (o, tu, spatial_id))?;
push_shown(out, &mut self.shown_bytes, self.shown_budget, (o, tu, spatial_id))?;
}
}
}
OBU_TILE_GROUP_T => {
if let Some(o) = self.tile_group(payload)? {
push_shown(out, &mut self.shown_bytes, (o, tu, spatial_id))?;
push_shown(out, &mut self.shown_bytes, self.shown_budget, (o, tu, spatial_id))?;
}
}
OBU_METADATA_T | OBU_TILE_LIST_T => {}
@@ -726,7 +738,7 @@ impl Decoder {
// A picture past the chunk's byte budget is never built (decoding the frame as a reference
// still happens, so the reference slots stay as the stream says), and the chunk is refused.
let bytes = picture_bytes(job.fh.upscaled_width as usize, job.fh.frame_height as usize, &job.seq.color);
if job.picture.is_some() && !shown_bytes_within_limit(self.shown_bytes.saturating_add(bytes)) {
if job.picture.is_some() && !shown_bytes_within_limit(self.shown_bytes.saturating_add(bytes), self.shown_budget) {
job.picture = None;
self.shown_overflow = true;
self.submit(job);
@@ -1307,18 +1319,33 @@ mod shown_limit_tests {
#[test]
fn push_shown_refuses_a_chunk_past_either_limit_and_drops_what_it_showed() {
let (mut out, mut bytes) = (Vec::new(), 0usize);
push_shown(&mut out, &mut bytes, shown(10)).unwrap();
push_shown(&mut out, &mut bytes, MAX_SHOWN_BYTES_PER_CHUNK, shown(10)).unwrap();
assert_eq!((out.len(), bytes), (1, 10));
// past the byte budget
assert!(push_shown(&mut out, &mut bytes, shown(MAX_SHOWN_BYTES_PER_CHUNK)).is_err());
assert!(push_shown(&mut out, &mut bytes, MAX_SHOWN_BYTES_PER_CHUNK, shown(MAX_SHOWN_BYTES_PER_CHUNK)).is_err());
assert!(out.is_empty(), "the refused chunk shows nothing");
// past the count limit
let (mut out, mut bytes) = (Vec::new(), 0usize);
for _ in 0..MAX_SHOWN_PER_CHUNK {
push_shown(&mut out, &mut bytes, shown(0)).unwrap();
push_shown(&mut out, &mut bytes, MAX_SHOWN_BYTES_PER_CHUNK, shown(0)).unwrap();
}
assert!(push_shown(&mut out, &mut bytes, shown(0)).is_err());
assert!(push_shown(&mut out, &mut bytes, MAX_SHOWN_BYTES_PER_CHUNK, shown(0)).is_err());
assert!(out.is_empty());
// past a caller's lower budget
let (mut out, mut bytes) = (Vec::new(), 0usize);
push_shown(&mut out, &mut bytes, 100, shown(60)).unwrap();
assert!(push_shown(&mut out, &mut bytes, 100, shown(60)).is_err());
assert!(out.is_empty());
}
#[test]
fn a_shown_budget_can_lower_the_limit_but_not_raise_it() {
let mut d = Decoder::with_threads(1);
assert_eq!(d.shown_budget, MAX_SHOWN_BYTES_PER_CHUNK);
d.set_shown_budget(1 << 20);
assert_eq!(d.shown_budget, 1 << 20);
d.set_shown_budget(u64::MAX);
assert_eq!(d.shown_budget, MAX_SHOWN_BYTES_PER_CHUNK);
}
#[test]
@@ -1333,8 +1360,8 @@ mod shown_limit_tests {
// 4K 4:2:0, u16 samples: about 25 MB; 256 of them is far past the limit, a few are fine
let one = picture_bytes(3840, 2160, &c);
assert_eq!(one, (3840 * 2160 + 2 * 1920 * 1080) * 2);
assert!(shown_bytes_within_limit(4 * one));
assert!(!shown_bytes_within_limit(MAX_SHOWN_PER_CHUNK * one));
assert!(shown_bytes_within_limit(4 * one, MAX_SHOWN_BYTES_PER_CHUNK));
assert!(!shown_bytes_within_limit(MAX_SHOWN_PER_CHUNK * one, MAX_SHOWN_BYTES_PER_CHUNK));
let mono = crate::av1::decode::header::ColorConfig { num_planes: 1, ..Default::default() };
assert_eq!(picture_bytes(10, 5, &mono), 100);
assert_eq!(picture_bytes(usize::MAX, 2, &c), usize::MAX);