Write AVI files: hdrl with avih and one strl (strh, strf, strn, indx) per
stream, odml/dmlh, an interleaved movi and idx1. Past the RIFF limit
(default 1 GiB) the file continues in OpenDML RIFF AVIX extensions with
ix## standard indexes and an indx super index per stream.
- VideoSpec::for_codec builds the stream from CodecParams; MPEG-4 Part 2
is tagged FMP4.
- Audio is timed by bytes (PCM-style) or one packet per chunk.
- Bad options and chunks are errors; finish() closes a readable file
after a failed write.
Tests read the files back through this crate's demuxer by every index,
and through ffmpeg (decode identical to the sources, seeks through the
index, ffprobe fields), in one RIFF and across AVIX extensions. The
MPEG-4 encoder tests play its packets from an AVI file.
The start of MPEG-2 encoding in craft_video::mpeg2 (feature
mpeg2-encode): progressive frame pictures, closed I/P GOPs, 4:2:0 Main
Profile and 4:2:2 Profile, constant quantiser or single-pass average
bitrate with adaptive quantisation, half-sample motion search.
The encoder shares the decoder's code tables, IDCT, dequantisation and
motion compensation, so the decoder outputs exactly the encoder's
reconstruction. B pictures, a VBV guarantee, interlaced coding and DVD
or broadcast presets are later steps of #1133.
The mpeg2 feature is split into mpeg2-decode and mpeg2-encode; mpeg2
still turns on both.
* craft-video: MPEG-4 Part 2 Simple profile encoder
Adds a clean-room MPEG-4 Visual (ISO/IEC 14496-2) encoder behind the new
`mpeg4` / `mpeg4-encode` features: the stream type Xvid and DivX write,
which craft-media already identifies (CodecId::Mpeg4Video) in MP4, Matroska,
AVI and MXF but which no crate could produce.
- I- and P-VOPs, 4:2:0 8-bit, H.263 quantisation, half-sample motion with a
predictor-seeded diamond search, skipped and intra macroblocks in P-VOPs,
intra refresh within the 132 codings of §7.4.5.
- Constant quantiser or 1-pass average bitrate.
- Headers in band (raw .m4v, AVI) or as extradata / CodecParams (MP4, Matroska).
Tests use ffmpeg/ffprobe as external decoders: every coefficient table entry
and escape mode 1/2 event decodes like the fixed-length escape, and encoded
streams decode without diagnostics, close to the encoder's reconstruction.
---------
Co-authored-by: alehel <git@ornumhelgaker.no>
Port of storytold/filmcraft#1952, reviewed and approved there before the
decoder moved to craft-libs. 4:2:2 streams that use no range-extension
coding tools (what cameras and libx265 write) now decode:
- chroma geometry from the SPS chroma format (ChromaFormat shifts) in the
picture store, intra, inter, transform tree, deblocking and SAO;
- Table 8-3 IntraPredModeC mapping for ChromaArrayType 2;
- stacked chroma transform blocks with their second cbf flags;
- chroma QP = Min(qPi, 51) for non-4:2:0 (Table 8-10 applies to 4:2:0 only);
- chroma motion vectors scaled per axis;
- a PPS that scales SAO offsets (a RExt tool) is refused as unsupported.
Tests: 22 libx265 4:2:2 fixtures (8- and 10-bit) compared sample-exactly
with ffmpeg, the Table 8-3 derivation, and the SAO offset scale check.
craft-video: FilmCraft's later H.264 encoder work (High 10, threads, sub-pel diamond) and its H.264 and ProRes tests
* craft-video: H.264 High 10 and High 10 Intra in the encoder
The encoder is generic over the sample type (u8 at 8 bits, u16 at 9-10
bits): QP' = QP + QpBdOffset for quantisation and lambdas, mb_qp_delta
range and wrap, chroma QP mapping, deblocking thresholds scaled to the bit
depth, clipping in prediction, interpolation and reconstruction. SPS bit
depths, profile_idc 110, constraint_set3 for High 10 Intra, avcC
high-profile fields, the High 10 cpbBrVclFactor for level choice. New
YuvFrame16 / Encoder::encode16, EncoderConfig::bit_depth / intra_only.
8-bit output is unchanged.
Merged onto craft-video's copy: its input stride checks, B-frame timestamp
check, configuration validation, level 6.2 refusal, motion search start
candidates and codec_params are kept.
Ported from storytold/filmcraft#1580 (commit d24ce15e).
* craft-video: the H.264 encoder honours threads: 1 and reuses MbInfo buffers; raw-YUV bench
- A pool is built for every thread count, including 1: rayon calls outside
install run on the global pool (all cores), so threads: 1 used to run on
every core. A pool that cannot be built is an InvalidConfig error.
- Up to eight macroblock-info buffers are recycled between pictures and
cleared on reuse, instead of allocating one per picture.
- examples/h264_h264enc_bench.rs encodes a raw YUV file and reports speed;
its argument checks run as tests.
Ported from storytold/filmcraft#3205 (merge 08f8075a).
* craft-video: H.264 encoder sub-pel refinement on a 4-point diamond; 16x8 and 8x16 only after 8x8 beat 16x16
Sub-pel refinement tries the four axis neighbours instead of all eight, and
the 16x8 / 8x16 partitions are searched only when 8x8 beat 16x16: fewer
SATD evaluations for about the same decisions. craft-video's own motion
search start candidates are kept.
Ported from storytold/filmcraft#3207 (merge 8db5334c).
* craft-video: FilmCraft's H.264 encoder tests; EncoderConfig::writer names the program in the SEI
The encoder had no integration tests here. FilmCraft's come along as the
h264_encode test target: ffmpeg/ffprobe oracle decodes (8-bit and High 10,
bit-exact against the reconstruction), pinned decisions, CRF level choice
and the settings SEI, thread counts and VBR overshoot.
EncoderConfig::writer is the program and version the identification SEI
names (craft-h264enc and this crate's version by default), so an
application's files keep naming it.
Two expectations differ from FilmCraft's copy: the pinned slice hashes
(this encoder also starts the 16x16 search from the skip, zero and
co-located vectors; with those left out and FilmCraft's writer name, it
gives FilmCraft's four hashes exactly) and a CRF that is not a number,
which this encoder refuses.
* craft-video: FilmCraft's ProRes tests (oracle decode and encode, round trip, fuzz, perf)
craft-video's ProRes copy came with only its hostile-input test. FilmCraft's
ffmpeg oracle decode and encode tests, round trips, mutation fuzz and the
ignored throughput tests come along; craft-video's own fuzz_hostile stays.
Frame::new returns a Result here, so the tests unwrap it.
---------
Co-authored-by: JordyUSA <75485565+JordyUSA@users.noreply.github.com>
Co-authored-by: vangel76 <vangel1976@googlemail.com>
* craft-video: budget HEVC queued output, drop a failed VP9 superframe's repeats, serial AV1 tiles past a metadata cap
Ported from storytold/deckcraft#1778 and #1841, which landed in DeckCraft's
decoder copies after craft-video took them.
* craft-video: H.264 picture order count no longer overflows on extreme stream values
Wrapping arithmetic through the POC type 0/1/2 paths, frame_num_offset and
the MMCO5 prev_ref_lsb update. Ported from storytold/deckcraft#1913.
---------
Co-authored-by: jayzed80 <jzinkant@gmail.com>
Moves effectcraft's effectcraft-vp8enc here: the lossy picture of a WebP image
or animation frame, written from RFC 6386. Rate-distortion choice between the
16x16 and B_PRED intra modes, per-frame token probability updates, loop filter
level from the quantizer.
`vp8` now turns on `vp8-decode` and `vp8-encode`, as `vp9` does; the decoder's
tests and examples require `vp8-decode`. The tests decode the encoder's frames
with this crate's decoder: with the loop filter off it reproduces the encoder's
reconstruction exactly.
* craft-video: AV1 decoder from FilmCraft (`av1-decode` feature)
FilmCraft's crates/av1 at 4c0d2ad7 becomes craft_video::av1::decode. The encoder moves to
av1/encode and is still exported from craft_video::av1. Feature av1 now turns on av1-decode and
av1-encode, so either half can be built without the other.
The ignored speed tests no longer read the thread CPU clock (FilmCraft used rustix for it).
craft_video::budget (the picture memory budget from DeckCraft's bitstream crate) is added for the
next commit. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its
codecs.
* craft-video: AV1 decoder hardening from DeckCraft's copy
- The picture store (eight reference slots, the new frame, frames in flight) is checked against
a memory budget before a frame is allocated; Decoder::set_memory_budget.
- Non-uniform tile spacing is refused past MAX_TILE_COLS columns or MAX_TILE_ROWS rows.
* craft-video: AV1 fuzz seeds are 64x64 so SVT-AV1 2.x can encode them
FilmCraft's seeds were 64x48; the SVT-AV1 2.3 on the Ubuntu runners refuses a source under 64
lines, and an encoder that exists but fails is a test failure, not a skip.
* craft-video: MPEG-2 and MPEG-1 video decoder from FilmCraft (`mpeg2`)
FilmCraft's crates/mpeg2v at 4bec22bd moves to craft_video::mpeg2 behind the
decoder-only feature `mpeg2`: sources, README and tests (unit, synthetic
streams checked by ffmpeg, the ffmpeg oracle fixtures, hostile sweeps).
Changes in the move: craft-media-testkit instead of filmcraft-testkit, one
test target (tests/mpeg2), crate paths, rustfmt at this repo's width.
New: a memory budget (Decoder::set_memory_budget, craft_video::budget) checked
when a sequence becomes active, and a mutation test over the synthetic
field-picture, 16x8 and dual-prime streams, which ffmpeg's encoder cannot
produce and the hostile sweep therefore never reached.
* craft-video mpeg2: write the restart check without a negated conjunction (clippy nonminimal_bool)
FilmCraft's crates/apv at 4bec22bd moves to craft_video::apv: the decoder
(craft_video::apv::decode, feature `apv-decode`), the encoder
(craft_video::apv::encode, feature `apv-encode`), both re-exported at the
module root; `apv` turns on both. Sources, README and tests (ffmpeg oracle,
alpha, mutation and hostile sweeps).
Changes in the move: craft_media::bitstream and craft-media-testkit instead of
FilmCraft's, one test target (tests/apv), crate paths.
New: DecodeOptions::memory_budget (craft_video::budget) refuses a frame whose
samples do not fit before anything is allocated (Error::MemoryBudget).
Fixed, each with a test that fails without it (debug-build overflows):
- EncoderConfig::new multiplied width, height and 30 in u64 without a check.
- Frame::new shifted by the bit depth unchecked (32 and above overflow).
- The public transform functions shifted by a bit depth outside 10..=16.
Three-way merge of the changes EffectCraft made to its av1enc and hevcenc
crates after craft-video took them:
- hevc: EncoderConfig carries colour_primaries, transfer_characteristics and
matrix_coeffs (H.273 code points, BT.709 by default) and the VUI writes them.
- av1: EncoderConfig carries color_primaries, transfer_characteristics and
matrix_coefficients and color_config writes them.
- av1: motion search keeps a visited set and stops a SAD once it cannot win,
SATD has fixed-size kernels, and a decision snapshot saves only the block's
BlockDecoded flags. Output is unchanged.
ffprobe tests check the colour description of both streams.
* craft-video: H.264 decoder from FilmCraft (`h264-decode` feature)
FilmCraft's crates/h264 at 4c0d2ad7 becomes craft_video::h264::decode. The encoder moves to
h264/encode and is still exported from craft_video::h264. Feature h264 now turns on h264-decode
and h264-encode, so either half can be built without the other.
Without the threads feature a reference row that was never published read through expect();
it now reads as a gray row. craft_video::budget (the picture memory budget from DeckCraft's
bitstream crate) is added for the next commit. craft-video is built at opt-level 3 in the dev
profile, as FilmCraft built its codecs.
* craft-video: H.264 decoder hardening from DeckCraft's copy
- The picture store (DPB, current picture, pictures in flight or queued for output) is checked
against a memory budget before a picture is allocated; Decoder::set_memory_budget.
- configure_avcc checks every entry before handling any: SPS NAL units in the first list, PPS in
the second. A picture slice in the record starts no picture.
- The inverse transforms clamp coefficients to the spec's r range instead of overflowing.
- Temporal direct motion vectors are scaled in i32 and clipped to the spec range.
- slice_qp_delta is added with a checked add.
- A picture job sets up its picture state inside the panic guard.
* craft-video: HEVC decoder from FilmCraft (`hevc-decode` feature)
FilmCraft's crates/hevc at 4c0d2ad7 becomes craft_video::hevc::decode. The encoder moves to
hevc/encode and is still exported from craft_video::hevc. Feature hevc now turns on hevc-decode
and hevc-encode, so either half can be built without the other.
craft_video::budget (the picture memory budget from DeckCraft's bitstream crate) is added for the
next commit. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its
codecs.
* craft-video: HEVC decoder hardening from DeckCraft's copy
- The picture store (DPB, current picture, pictures in flight) is checked against a memory
budget before a picture is allocated; Decoder::set_memory_budget.
- configure_hvcc checks every array before handling any NAL: only VPS, SPS, PPS and SEI arrays,
each NAL header agreeing with its array type. A picture slice in the record starts no picture.
- init_qp_minus26, slice_qp_delta and delta_chroma_log2_weight_denom are range checked before
the additions that overflowed on Exp-Golomb values near the i32 limits.
- A picture job sets up its picture state inside the panic guard.
* craft-video: VP9 decoder from FilmCraft (`vp9-decode` feature)
FilmCraft's crates/vp9 at 4c0d2ad7 becomes craft_video::vp9::decode. The encoder moves to
vp9/encode and is still exported from craft_video::vp9. Feature vp9 now turns on vp9-decode and
vp9-encode, so either half can be built without the other.
The decoder's tests find ffmpeg and the fixture directory through craft-media-testkit; a fixture
that cannot be generated fails under CRAFT_MEDIA_REQUIRE_ORACLES=1. craft-video is built at
opt-level 3 in the dev profile, as FilmCraft built its codecs.
* craft-video: VP9 decoder checks its picture store against a memory budget
DeckCraft's copy of the decoder refuses a frame size whose pictures (eight reference slots, the
new frame and those in flight) would not fit a byte budget. The budget type comes with it as
craft_video::budget; FilmCraft's 2^26 luma sample cap stays.
The eight-part craft-video series landed as one commit, because each part's branch contains the parts before it.
- #702 dnx: DNxHD/DNxHR decoder and encoder
- #703 vp8: VP8 decoder
- #704 prores: ProRes decoder and encoder
- #705 h264: H.264 encoder
- #706 hevc: HEVC encoder
- #707 vp9: VP9 encoder
- #708 av1: AV1 encoder
Dylan M. Taylor
alehel
Melchi Aaron Zedek
Justin Wilson
el-j