craft-audio names its VMD decoder feature `vmd-decode`, and craft-video
names a codec's decoder `<codec>-decode` wherever the codec may also have
an encoder (mpeg2, mpeg4, vp8…). The video decoder now follows that:
`vmd-decode` gates the module and its VideoFrame conversion, and `vmd`
turns it on, so `vmd` keeps working and an encoder can be added later
without renaming.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`VideoFrame::try_from(&vmd::Decoder)` looks the picture's palette indices
up in the decoder's palette and gives 8-bit R, G, B planes (matrix 0). The
README and the frame module docs said every decoder's output converts by
moving its buffers; paletted video copies, as VMD keeps only indices.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preserves codec algorithm from AniCraft1f3e603b7ff53a3da364e41ec4da8d43dc83aab0 and2b7571363b4b647266df758f95b6e397021cfd4d; neutral frame type and explicit caller limits adapt the shared API.
mjpeg::Decoder called craft_jpeg::decode with no Adobe transform, so a
packet with numeric component ids and APP14 transform 0 (RGB as stored)
was converted as YCbCr and came out in the wrong colours. The decoder
now reads the transform with craft_jpeg::adobe_transform, as the JPEG
image readers do. A regression test injects APP14 transforms 0 and 1;
it fails without the fix.
Addresses the CodeRabbit review on #1692: odd-width raw RGB rows carry a pad byte,
and BI_BITFIELDS is only BGRX with the matching masks (new from_avi_bitfields).
* image-codecs: decode AVIF pixels with craft-video's pure-Rust AV1 decoder
New optional `avif-decode` feature (native targets) reads still AVIF
files: the HEIF/ISOBMFF item graph, grids, clap/irot/imir, alpha
auxiliary images (including premultiplied), 8/10/12-bit 4:0:0, 4:2:0,
4:2:2 and 4:4:4, nclx/ICC colour, Exif and XMP, and PQ/HLG to linear
F32 through cicp::linearize as the HEIF path does.
The container walk follows LightCraft #3017 by @emmertex. Every size is
checked against the caller's Limits before the AV1 decoder runs, the
decoder's memory budget is capped by max_alloc, and the decode is
wrapped in catch_unwind so a decoder panic becomes an error.
Fixtures are tiny ffmpeg/libaom files; the 4:2:0 planes match libdav1d
bit for bit. Adds a decode_avif fuzz target.
Part of storytold/craft-libs#778.
* image-codecs, video: address review on the AVIF decoder
- clap: width and height are unsigned fractions, the offsets have signed
numerators, so 0x80000000/0x40000000 is a width of 2, not -2.
- An item with clap, irot or imir more than once (repeated ipma records)
is refused before any canvas work, instead of rotating 261120 times.
- Matrix coefficients 10 (BT.2020 constant luminance) is refused rather
than converted with the non-constant-luminance equations.
- An alpha plane whose own transforms differ from the colour's is refused
rather than composed in the wrong place; none, or the colour's, still
decode.
- craft-video: Decoder::set_shown_budget lowers the per-chunk byte cap on
shown pictures (never above 1 GiB), and the AVIF decoder sets it to
max_alloc, so a payload of many shown pictures is refused before they
are built.
A frame-level wrapper over craft-jpeg: packets decode to gray or RGB
frames (Annex K Huffman tables put in for AVI MJPG packets that have
none) and frames encode to baseline JPEG packets. Checked against
ffmpeg as a black-box oracle in both directions.
Part of #1510.
Adds the ffv1 feature (RFC 9043): YCbCr (any 2^n subsampling, gray,
optional transparency) and RGB through the JPEG 2000 reversible colour
transform, 8 to 16 bits, any slice raster, slice and configuration
record CRCs. Not covered yet: the Golomb-Rice coder, versions 0 and 1,
inter frames.
Part of #1509
- media: `read_frame` reserves the frame's bytes fallibly. On a 32-bit
target a frame of over 2 GiB inside a larger source made `vec!` panic
with a capacity overflow; it is now an error.
- video: `Decoder::decode_palette` takes the palette of a frame that
has no rectangle in the picture, so the documented frame loop no
longer drops it. ffmpeg applies the palette of a frame whose
rectangle is empty; the pictures after it now agree.
- media: correct round 1's frame placement. The picture's position is
not one value for a file: it starts at the screen's corner and a
frame with data that is as large as the picture and not at the corner
moves it for the frames that follow, as ffmpeg was seen to do.
`picture_rect` now takes the frame's index; `origin()` is gone.
- audio: unknown frame flags are refused for an empty frame too.
- video: count the unpack buffer in the picture budget.
- docs: which sound frames are bounded by what, which cut-short blocks
are errors, and why sound needs both the flag and a rate.
- tests: ffmpeg's placement cases, the size of the LZ stop, multi-block
16-bit stereo frames, and an LZ test that compared half its picture.
- media: place video frames by the first frame as large as the picture,
not by the header's position fields, as ffmpeg was seen to do; a
frame outside the picture has no rectangle.
- media: a file has sound only with the header flag and a sample rate.
- video: stop LZ unpacking at what a picture can use, so a frame's cost
no longer goes by a length the file states.
- audio: a flags 1 sound frame plays every whole block it holds.
- docs: how sound lines up with pictures, what the buffer count is, and
which cut-short blocks are errors.
- tests: the frame-count cap, the 0xFF tag near the end of LZ data,
palette components over 6 bits, the error conversion and the new
identifiers.
Sierra VMD (Video and Music Data) is the video format of Sierra and
Coktel Vision games of the 1990s. Written clean-room from the public
format description at https://wiki.multimedia.cx/index.php/VMD, with
ffmpeg as a black-box oracle.
- craft-media: `vmd` feature, a demuxer for the 816-byte header and the
table of contents; `Container::Vmd`, `CodecId::VmdVideo` and
`CodecId::VmdAudio`.
- craft-video: `vmd` feature, a decoder for 8-bit paletted pictures
(rendering methods 1 to 3, LZ packing, palette changes).
- craft-audio: `vmd-decode` feature, a decoder for 8-bit PCM and 16-bit
DPCM sound, mono and stereo.
- craft-media-testkit: a VMD fixture writer, since ffmpeg reads VMD but
cannot write it.
Tests compare the decoders with ffmpeg on generated files, and on real
files when CRAFT_MEDIA_VMD_SAMPLES names a directory of them. 16-bit,
24-bit and Indeo 3 pictures, IMA ADPCM sound and the later header
revisions are not decoded.
- flush ends the GOP for average bitrate too: the budget of the pictures
the GOP did not get is taken back, so flushed short GOPs no longer
raise the bitrate.
- A lone picture flushed before a second sets the step gets dts = pts - 1,
so decoding times still increase if encoding goes on.
- Presentation times whose decoding time does not fit in an i64 are
refused with BadTimestamp before any state changes.
- Tests: the skip range test uses an f_code that can carry its vector, and
the reconstruction count is checked before comparing pictures.
B pictures between anchors (b_pictures 0..=4, default 2) with reordering:
pictures go in in display order, packets come out in decoding order with
dts one picture interval before the presentation times. Closed or open
GOPs; the leading B pictures of a closed GOP predict backward only.
B macroblocks are forward, backward or interpolated, from a search per
direction seeded with the P picture's vectors scaled to the distance, or
skipped (previous directions with the predictors, kept inside the
reference). Rate control adds TM5's B complexity (K_b 1.4); a constant
quantiser codes B pictures at 1.25 times it.
The start of MPEG-2 encoding in craft_video::mpeg2 (feature
mpeg2-encode): progressive frame pictures, closed I/P GOPs, 4:2:0 Main
Profile and 4:2:2 Profile, constant quantiser or single-pass average
bitrate with adaptive quantisation, half-sample motion search.
The encoder shares the decoder's code tables, IDCT, dequantisation and
motion compensation, so the decoder outputs exactly the encoder's
reconstruction. B pictures, a VBV guarantee, interlaced coding and DVD
or broadcast presets are later steps of #1133.
The mpeg2 feature is split into mpeg2-decode and mpeg2-encode; mpeg2
still turns on both.