Commit Graph
76 Commits
Author SHA1 Message Date
alehelandClaude Opus 5.5 e0440f9942 video: the VMD decoder is behind vmd-decode, with vmd turning it on
craft-audio names its VMD decoder feature `vmd-decode`, and craft-video
names a codec's decoder `<codec>-decode` wherever the codec may also have
an encoder (mpeg2, mpeg4, vp8…). The video decoder now follows that:
`vmd-decode` gates the module and its VideoFrame conversion, and `vmd`
turns it on, so `vmd` keeps working and an encoder can be added later
without renaming.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 09:39:25 +01:00
alehel 983a32c6e6 Merge remote-tracking branch 'origin/main' into n1460-merge
# Conflicts:
#	crates/video/src/lib.rs
2026-11-17 08:32:17 +01:00
alehelandClaude Opus 5.5 73e6b12930 video: a VMD decoder's picture converts into a VideoFrame
`VideoFrame::try_from(&vmd::Decoder)` looks the picture's palette indices
up in the decoder's palette and gives 8-bit R, G, B planes (matrix 0). The
README and the frame module docs said every decoder's output converts by
moving its buffers; paletted video copies, as VMD keeps only indices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 04:54:38 +01:00
Dylan M. Taylor 18277c006c test(video): reject short and overlong inflated Screen Video blocks 2026-11-16 22:35:35 -05:00
Dylan M. Taylor 3e0bb788a7 test(video): bound Screen Video output and hostile packet handling 2026-11-16 22:33:39 -05:00
Tanish Manku 8ad90cc70d test(video): retain synthetic partial-block Screen Video fixture 2026-11-16 22:33:12 -05:00
Tanish Manku 5ed5fb9de7 video: extract bounded Screen Video key-frame decoding from AniCraft
Preserves codec algorithm from AniCraft1f3e603b7ff53a3da364e41ec4da8d43dc83aab0 and2b7571363b4b647266df758f95b6e397021cfd4d; neutral frame type and explicit caller limits adapt the shared API.
2026-11-16 22:32:40 -05:00
alehel 00e0075184 Merge remote-tracking branch 'origin/main' into m1460-merge
# Conflicts:
#	crates/audio/Cargo.toml
#	crates/audio/README.md
#	crates/audio/src/lib.rs
#	crates/media-testkit/src/lib.rs
2026-11-17 04:06:58 +01:00
Aleksander Helgaker 274d562dee craft-video: MPEG-2 encoder, third step: CBR and VBR within the VBV buffer (#1133) (#1767) 2026-11-16 21:31:10 -05:00
Aleksander Helgaker 77f244cb96 craft-video: VideoFrame, a shared frame type for decoder output (#1231) (#1768) 2026-11-16 21:31:05 -05:00
el-j 5ea29f8f05 craft-video: Hap video decoder and encoder (#1519) (#1911) 2026-11-16 21:30:00 -05:00
Dylan M. Taylor 136c34ae4c ffv1: bound thin-frame border and alpha storage before allocation (#1917) 2026-11-16 14:51:27 -05:00
alehel d4ed32ac8f Merge remote-tracking branch 'origin/main' into pr1460-merge
# Conflicts:
#	crates/media/src/lib.rs
2026-11-16 18:16:58 +01:00
FredApps ddbe78231b craft-video: pass the packet's Adobe APP14 transform to the MJPEG decoder (#1847)
mjpeg::Decoder called craft_jpeg::decode with no Adobe transform, so a
packet with numeric component ids and APP14 transform 0 (RGB as stored)
was converted as YCbCr and came out in the wrong colours. The decoder
now reads the transform with craft_jpeg::adobe_transform, as the JPEG
image readers do. A regression test injects APP14 transforms 0 and 1;
it fails without the fix.
2026-11-16 07:10:48 -05:00
alehel 1676c70cd0 Merge remote-tracking branch 'origin/main' into claude/vmd-format-support-7746c5
# Conflicts:
#	crates/audio/src/lib.rs
#	crates/video/src/lib.rs
2026-11-16 12:43:58 +01:00
FredApps 341020b2c1 Merge pull request #1757 from el-j/feat/video-mjpeg
craft-video: Motion JPEG decoder and encoder (mjpeg)
2026-11-16 11:18:40 +01:00
Dylan M. Taylor 5c91598ccc Merge pull request #1807 from storytold/fix/av1-zero-quantizer
fix(video): preserve explicit rav1e zero quantizer
2026-11-15 23:44:40 -05:00
Dylan M. Taylor f450b5b591 fix(video): preserve explicit rav1e zero quantizer 2026-11-15 22:33:24 -05:00
FredApps f57db543da raw_video: QuickTime RGB24 always uses even-padded rows; only the last row's pad may be missing 2026-11-16 03:12:25 +01:00
el-j 252c959167 Merge remote-tracking branch 'origin/main' into feat/video-mjpeg 2026-11-16 02:22:43 +01:00
FredApps c3ad96bfd8 video: honour QuickTime RGB row padding; require BGRX masks for BI_BITFIELDS
Addresses the CodeRabbit review on #1692: odd-width raw RGB rows carry a pad byte,
and BI_BITFIELDS is only BGRX with the matching masks (new from_avi_bitfields).
2026-11-16 02:14:30 +01:00
FredApps 4216b943ed Merge remote-tracking branch 'origin/main' into w-codex/shared-raw-video 2026-11-16 00:40:36 +01:00
FredApps dd46f41b9a video: cover every raw layout, label and v210/planar path 2026-11-16 00:40:34 +01:00
Dylan M. Taylor 4c254242de Merge pull request #1142 from vangel76/perf/hevc-residual-decode
HEVC decoding about 4% fewer instructions on one thread
2026-11-15 18:31:24 -05:00
Dylan M. Taylor faa36fc2ea Merge pull request #1140 from vangel76/perf/av1-wavefront-3106
AV1 decoding about 1.7x as fast with many threads
2026-11-15 18:28:38 -05:00
FredApps 59b4f1e8c2 image-codecs: decode AVIF pixels with the pure-Rust AV1 decoder (#778) (#1601)
* image-codecs: decode AVIF pixels with craft-video's pure-Rust AV1 decoder

New optional `avif-decode` feature (native targets) reads still AVIF
files: the HEIF/ISOBMFF item graph, grids, clap/irot/imir, alpha
auxiliary images (including premultiplied), 8/10/12-bit 4:0:0, 4:2:0,
4:2:2 and 4:4:4, nclx/ICC colour, Exif and XMP, and PQ/HLG to linear
F32 through cicp::linearize as the HEIF path does.

The container walk follows LightCraft #3017 by @emmertex. Every size is
checked against the caller's Limits before the AV1 decoder runs, the
decoder's memory budget is capped by max_alloc, and the decode is
wrapped in catch_unwind so a decoder panic becomes an error.

Fixtures are tiny ffmpeg/libaom files; the 4:2:0 planes match libdav1d
bit for bit. Adds a decode_avif fuzz target.

Part of storytold/craft-libs#778.

* image-codecs, video: address review on the AVIF decoder

- clap: width and height are unsigned fractions, the offsets have signed
  numerators, so 0x80000000/0x40000000 is a width of 2, not -2.
- An item with clap, irot or imir more than once (repeated ipma records)
  is refused before any canvas work, instead of rotating 261120 times.
- Matrix coefficients 10 (BT.2020 constant luminance) is refused rather
  than converted with the non-constant-luminance equations.
- An alpha plane whose own transforms differ from the colour's is refused
  rather than composed in the wrong place; none, or the colour's, still
  decode.
- craft-video: Decoder::set_shown_budget lowers the per-chunk byte cap on
  shown pictures (never above 1 GiB), and the AVIF decoder sets it to
  max_alloc, so a payload of many shown pictures is refused before they
  are built.
2026-11-15 23:25:25 +01:00
el-j 0281d0951b craft-video: Motion JPEG decoder and encoder (mjpeg)
A frame-level wrapper over craft-jpeg: packets decode to gray or RGB
frames (Annex K Huffman tables put in for AVI MJPG packets that have
none) and frames encode to baseline JPEG packets. Checked against
ffmpeg as a black-box oracle in both directions.

Part of #1510.
2026-11-15 22:19:12 +01:00
alehel ad5ff4426a Merge remote-tracking branch 'origin/main' into claude/vmd-format-support-7746c5
# Conflicts:
#	crates/audio/src/lib.rs
2026-11-15 21:17:32 +01:00
Dylan M. Taylor b63ab73fa3 video: share uncompressed AVI and QuickTime sample decoding 2026-11-15 17:59:49 +00:00
Dylan M. Taylor d1f67cae22 Merge pull request #1578 from el-j/feat/video-ffv1
craft-video: FFV1 lossless video (version 3, range coder) decoder and encoder
2026-11-15 12:22:53 -05:00
vangel76andClaude Sonnet 5.5 e07edcee5d Merge upstream/main into perf/hevc-residual-decode
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 14:16:57 +01:00
vangel76andClaude Sonnet 5.5 472e064b74 Merge upstream/main into perf/av1-wavefront-3106
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 14:13:59 +01:00
el-j 5ab7a6035c craft-video: FFV1 lossless video, version 3 range-coder decoder and encoder
Adds the ffv1 feature (RFC 9043): YCbCr (any 2^n subsampling, gray,
optional transparency) and RGB through the JPEG 2000 reversible colour
transform, 8 to 16 bits, any slice raster, slice and configuration
record CRCs. Not covered yet: the Golomb-Rice coder, versions 0 and 1,
inter frames.

Part of #1509
2026-11-15 13:06:49 +01:00
Dylan M. Taylor 9d088c77ea Support buffered output from tiled AV1 encoder 2026-11-15 11:25:41 +00:00
alehel 911932d3a5 fix(media,video): VMD frames too large to load, palettes of frames without a rectangle
- media: `read_frame` reserves the frame's bytes fallibly. On a 32-bit
  target a frame of over 2 GiB inside a larger source made `vec!` panic
  with a capacity overflow; it is now an error.
- video: `Decoder::decode_palette` takes the palette of a frame that
  has no rectangle in the picture, so the documented frame loop no
  longer drops it. ffmpeg applies the palette of a frame whose
  rectangle is empty; the pictures after it now agree.
2026-11-15 07:54:22 +01:00
alehel 6e95f83178 Merge remote-tracking branch 'origin/main' into claude/vmd-format-support-7746c5
# Conflicts:
#	crates/video/Cargo.toml
#	crates/video/README.md
2026-11-15 07:51:49 +01:00
Dylan M. Taylor 92c0b49b5e feat(video): support tiled AV1 for large frames (#1453)
* feat(video): add tiled AV1 large-frame fallback

* feat(video): expose tiled AV1 speed preset

* fix(video): use clamped AV1 keyframe interval
2026-11-14 20:38:28 -05:00
alehel fcfb66d0c2 fix(media,video,audio): VMD review round 2
- media: correct round 1's frame placement. The picture's position is
  not one value for a file: it starts at the screen's corner and a
  frame with data that is as large as the picture and not at the corner
  moves it for the frames that follow, as ffmpeg was seen to do.
  `picture_rect` now takes the frame's index; `origin()` is gone.
- audio: unknown frame flags are refused for an empty frame too.
- video: count the unpack buffer in the picture budget.
- docs: which sound frames are bounded by what, which cut-short blocks
  are errors, and why sound needs both the flag and a rate.
- tests: ffmpeg's placement cases, the size of the LZ stop, multi-block
  16-bit stereo frames, and an LZ test that compared half its picture.
2026-11-15 01:50:28 +01:00
alehel 75badac3b6 fix(media,video,audio): VMD review round 1
- media: place video frames by the first frame as large as the picture,
  not by the header's position fields, as ffmpeg was seen to do; a
  frame outside the picture has no rectangle.
- media: a file has sound only with the header flag and a sample rate.
- video: stop LZ unpacking at what a picture can use, so a frame's cost
  no longer goes by a length the file states.
- audio: a flags 1 sound frame plays every whole block it holds.
- docs: how sound lines up with pictures, what the buffer count is, and
  which cut-short blocks are errors.
- tests: the frame-count cap, the 0xFF tag near the end of LZ data,
  palette components over 6 bits, the error conversion and the new
  identifiers.
2026-11-15 01:50:28 +01:00
alehel 6ac5f83bb2 feat(media,video,audio): add Sierra VMD demuxer and decoders
Sierra VMD (Video and Music Data) is the video format of Sierra and
Coktel Vision games of the 1990s. Written clean-room from the public
format description at https://wiki.multimedia.cx/index.php/VMD, with
ffmpeg as a black-box oracle.

- craft-media: `vmd` feature, a demuxer for the 816-byte header and the
  table of contents; `Container::Vmd`, `CodecId::VmdVideo` and
  `CodecId::VmdAudio`.
- craft-video: `vmd` feature, a decoder for 8-bit paletted pictures
  (rendering methods 1 to 3, LZ packing, palette changes).
- craft-audio: `vmd-decode` feature, a decoder for 8-bit PCM and 16-bit
  DPCM sound, mono and stereo.
- craft-media-testkit: a VMD fixture writer, since ffmpeg reads VMD but
  cannot write it.

Tests compare the decoders with ffmpeg on generated files, and on real
files when CRAFT_MEDIA_VMD_SAMPLES names a directory of them. 16-bit,
24-bit and Indeo 3 pictures, IMA ADPCM sound and the later header
revisions are not decoded.
2026-11-15 01:50:28 +01:00
Dylan M. Taylor 1a4cf6d0ba Merge pull request #1195 from alehel/craft-video/mpeg2-encode-b
craft-video: MPEG-2 encoder, second step: B pictures (#1133)
2026-11-14 19:39:11 -05:00
alehel c83e8ba250 craft-video: MPEG-2 encoder: flush and timestamp fixes from review (#1133)
- flush ends the GOP for average bitrate too: the budget of the pictures
  the GOP did not get is taken back, so flushed short GOPs no longer
  raise the bitrate.
- A lone picture flushed before a second sets the step gets dts = pts - 1,
  so decoding times still increase if encoding goes on.
- Presentation times whose decoding time does not fit in an i64 are
  refused with BadTimestamp before any state changes.
- Tests: the skip range test uses an f_code that can carry its vector, and
  the reconstruction count is checked before comparing pictures.
2026-11-15 00:07:33 +01:00
Dylan M. Taylor d6cd7836d3 feat(video,image-codecs,stream-codecs): add encode_with_key, encode_avif_hdr, and decode_ccitt_raw (#1369) 2026-11-14 16:15:44 -05:00
Dylan M. Taylor 166ad3e46a craft-video: preserve FilmCraft H.264 search starts 2026-11-13 14:19:12 +00:00
alehel c298ac8bd2 craft-video: MPEG-2 encoder, second step: B pictures (#1133)
B pictures between anchors (b_pictures 0..=4, default 2) with reordering:
pictures go in in display order, packets come out in decoding order with
dts one picture interval before the presentation times. Closed or open
GOPs; the leading B pictures of a closed GOP predict backward only.

B macroblocks are forward, backward or interpolated, from a search per
direction seeded with the P picture's vectors scaled to the distance, or
skipped (previous directions with the predictors, kept inside the
reference). Rate control adds TM5's B complexity (K_b 1.4); a constant
quantiser codes B pictures at 1.25 times it.
2026-11-13 13:38:33 +01:00
Dylan M. Taylor d9419a111a Merge pull request #1136 from alehel/craft-video/mpeg2-encode-ip
craft-video: MPEG-2 encoder, first step: I and P pictures (#1133)
2026-11-13 05:41:51 -05:00
Dylan M. Taylor ea24422ae9 Merge pull request #1154 from storytold/codex/778-h264-picnum
video: harden H.264 and configure frame limits
2026-11-13 03:49:15 -05:00
Dylan M. Taylor 9814469613 Bound extreme H264 POC cycle arithmetic 2026-11-13 07:24:13 +00:00
Dylan M. Taylor 56a1f2b7c4 Avoid premature saturation in H264 type 1 POC 2026-11-13 07:11:12 +00:00
alehel 1df8d98c85 craft-video: MPEG-2 encoder, first step: I and P pictures (#1133)
The start of MPEG-2 encoding in craft_video::mpeg2 (feature
mpeg2-encode): progressive frame pictures, closed I/P GOPs, 4:2:0 Main
Profile and 4:2:2 Profile, constant quantiser or single-pass average
bitrate with adaptive quantisation, half-sample motion search.

The encoder shares the decoder's code tables, IDCT, dequantisation and
motion compensation, so the decoder outputs exactly the encoder's
reconstruction. B pictures, a VBV guarantee, interlaced coding and DVD
or broadcast presets are later steps of #1133.

The mpeg2 feature is split into mpeg2-decode and mpeg2-encode; mpeg2
still turns on both.
2026-11-13 07:34:05 +01:00