* craft-video: MPEG-4 Part 2 Simple profile encoder
Adds a clean-room MPEG-4 Visual (ISO/IEC 14496-2) encoder behind the new
`mpeg4` / `mpeg4-encode` features: the stream type Xvid and DivX write,
which craft-media already identifies (CodecId::Mpeg4Video) in MP4, Matroska,
AVI and MXF but which no crate could produce.
- I- and P-VOPs, 4:2:0 8-bit, H.263 quantisation, half-sample motion with a
predictor-seeded diamond search, skipped and intra macroblocks in P-VOPs,
intra refresh within the 132 codings of §7.4.5.
- Constant quantiser or 1-pass average bitrate.
- Headers in band (raw .m4v, AVI) or as extradata / CodecParams (MP4, Matroska).
Tests use ffmpeg/ffprobe as external decoders: every coefficient table entry
and escape mode 1/2 event decodes like the fixed-length escape, and encoded
streams decode without diagnostics, close to the encoder's reconstruction.
---------
Co-authored-by: alehel <git@ornumhelgaker.no>
Port of storytold/filmcraft#1952, reviewed and approved there before the
decoder moved to craft-libs. 4:2:2 streams that use no range-extension
coding tools (what cameras and libx265 write) now decode:
- chroma geometry from the SPS chroma format (ChromaFormat shifts) in the
picture store, intra, inter, transform tree, deblocking and SAO;
- Table 8-3 IntraPredModeC mapping for ChromaArrayType 2;
- stacked chroma transform blocks with their second cbf flags;
- chroma QP = Min(qPi, 51) for non-4:2:0 (Table 8-10 applies to 4:2:0 only);
- chroma motion vectors scaled per axis;
- a PPS that scales SAO offsets (a RExt tool) is refused as unsupported.
Tests: 22 libx265 4:2:2 fixtures (8- and 10-bit) compared sample-exactly
with ffmpeg, the Table 8-3 derivation, and the SAO offset scale check.
craft-video: FilmCraft's later H.264 encoder work (High 10, threads, sub-pel diamond) and its H.264 and ProRes tests
* craft-video: H.264 High 10 and High 10 Intra in the encoder
The encoder is generic over the sample type (u8 at 8 bits, u16 at 9-10
bits): QP' = QP + QpBdOffset for quantisation and lambdas, mb_qp_delta
range and wrap, chroma QP mapping, deblocking thresholds scaled to the bit
depth, clipping in prediction, interpolation and reconstruction. SPS bit
depths, profile_idc 110, constraint_set3 for High 10 Intra, avcC
high-profile fields, the High 10 cpbBrVclFactor for level choice. New
YuvFrame16 / Encoder::encode16, EncoderConfig::bit_depth / intra_only.
8-bit output is unchanged.
Merged onto craft-video's copy: its input stride checks, B-frame timestamp
check, configuration validation, level 6.2 refusal, motion search start
candidates and codec_params are kept.
Ported from storytold/filmcraft#1580 (commit d24ce15e).
* craft-video: the H.264 encoder honours threads: 1 and reuses MbInfo buffers; raw-YUV bench
- A pool is built for every thread count, including 1: rayon calls outside
install run on the global pool (all cores), so threads: 1 used to run on
every core. A pool that cannot be built is an InvalidConfig error.
- Up to eight macroblock-info buffers are recycled between pictures and
cleared on reuse, instead of allocating one per picture.
- examples/h264_h264enc_bench.rs encodes a raw YUV file and reports speed;
its argument checks run as tests.
Ported from storytold/filmcraft#3205 (merge 08f8075a).
* craft-video: H.264 encoder sub-pel refinement on a 4-point diamond; 16x8 and 8x16 only after 8x8 beat 16x16
Sub-pel refinement tries the four axis neighbours instead of all eight, and
the 16x8 / 8x16 partitions are searched only when 8x8 beat 16x16: fewer
SATD evaluations for about the same decisions. craft-video's own motion
search start candidates are kept.
Ported from storytold/filmcraft#3207 (merge 8db5334c).
* craft-video: FilmCraft's H.264 encoder tests; EncoderConfig::writer names the program in the SEI
The encoder had no integration tests here. FilmCraft's come along as the
h264_encode test target: ffmpeg/ffprobe oracle decodes (8-bit and High 10,
bit-exact against the reconstruction), pinned decisions, CRF level choice
and the settings SEI, thread counts and VBR overshoot.
EncoderConfig::writer is the program and version the identification SEI
names (craft-h264enc and this crate's version by default), so an
application's files keep naming it.
Two expectations differ from FilmCraft's copy: the pinned slice hashes
(this encoder also starts the 16x16 search from the skip, zero and
co-located vectors; with those left out and FilmCraft's writer name, it
gives FilmCraft's four hashes exactly) and a CRF that is not a number,
which this encoder refuses.
* craft-video: FilmCraft's ProRes tests (oracle decode and encode, round trip, fuzz, perf)
craft-video's ProRes copy came with only its hostile-input test. FilmCraft's
ffmpeg oracle decode and encode tests, round trips, mutation fuzz and the
ignored throughput tests come along; craft-video's own fuzz_hostile stays.
Frame::new returns a Result here, so the tests unwrap it.
---------
Co-authored-by: JordyUSA <75485565+JordyUSA@users.noreply.github.com>
Co-authored-by: vangel76 <vangel1976@googlemail.com>
* craft-video: budget HEVC queued output, drop a failed VP9 superframe's repeats, serial AV1 tiles past a metadata cap
Ported from storytold/deckcraft#1778 and #1841, which landed in DeckCraft's
decoder copies after craft-video took them.
* craft-video: H.264 picture order count no longer overflows on extreme stream values
Wrapping arithmetic through the POC type 0/1/2 paths, frame_num_offset and
the MMCO5 prev_ref_lsb update. Ported from storytold/deckcraft#1913.
---------
Co-authored-by: jayzed80 <jzinkant@gmail.com>
Moves effectcraft's effectcraft-vp8enc here: the lossy picture of a WebP image
or animation frame, written from RFC 6386. Rate-distortion choice between the
16x16 and B_PRED intra modes, per-frame token probability updates, loop filter
level from the quantizer.
`vp8` now turns on `vp8-decode` and `vp8-encode`, as `vp9` does; the decoder's
tests and examples require `vp8-decode`. The tests decode the encoder's frames
with this crate's decoder: with the loop filter off it reproduces the encoder's
reconstruction exactly.
* craft-video: AV1 decoder from FilmCraft (`av1-decode` feature)
FilmCraft's crates/av1 at 4c0d2ad7 becomes craft_video::av1::decode. The encoder moves to
av1/encode and is still exported from craft_video::av1. Feature av1 now turns on av1-decode and
av1-encode, so either half can be built without the other.
The ignored speed tests no longer read the thread CPU clock (FilmCraft used rustix for it).
craft_video::budget (the picture memory budget from DeckCraft's bitstream crate) is added for the
next commit. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its
codecs.
* craft-video: AV1 decoder hardening from DeckCraft's copy
- The picture store (eight reference slots, the new frame, frames in flight) is checked against
a memory budget before a frame is allocated; Decoder::set_memory_budget.
- Non-uniform tile spacing is refused past MAX_TILE_COLS columns or MAX_TILE_ROWS rows.
* craft-video: AV1 fuzz seeds are 64x64 so SVT-AV1 2.x can encode them
FilmCraft's seeds were 64x48; the SVT-AV1 2.3 on the Ubuntu runners refuses a source under 64
lines, and an encoder that exists but fails is a test failure, not a skip.
* craft-video: MPEG-2 and MPEG-1 video decoder from FilmCraft (`mpeg2`)
FilmCraft's crates/mpeg2v at 4bec22bd moves to craft_video::mpeg2 behind the
decoder-only feature `mpeg2`: sources, README and tests (unit, synthetic
streams checked by ffmpeg, the ffmpeg oracle fixtures, hostile sweeps).
Changes in the move: craft-media-testkit instead of filmcraft-testkit, one
test target (tests/mpeg2), crate paths, rustfmt at this repo's width.
New: a memory budget (Decoder::set_memory_budget, craft_video::budget) checked
when a sequence becomes active, and a mutation test over the synthetic
field-picture, 16x8 and dual-prime streams, which ffmpeg's encoder cannot
produce and the hostile sweep therefore never reached.
* craft-video mpeg2: write the restart check without a negated conjunction (clippy nonminimal_bool)
FilmCraft's crates/apv at 4bec22bd moves to craft_video::apv: the decoder
(craft_video::apv::decode, feature `apv-decode`), the encoder
(craft_video::apv::encode, feature `apv-encode`), both re-exported at the
module root; `apv` turns on both. Sources, README and tests (ffmpeg oracle,
alpha, mutation and hostile sweeps).
Changes in the move: craft_media::bitstream and craft-media-testkit instead of
FilmCraft's, one test target (tests/apv), crate paths.
New: DecodeOptions::memory_budget (craft_video::budget) refuses a frame whose
samples do not fit before anything is allocated (Error::MemoryBudget).
Fixed, each with a test that fails without it (debug-build overflows):
- EncoderConfig::new multiplied width, height and 30 in u64 without a check.
- Frame::new shifted by the bit depth unchecked (32 and above overflow).
- The public transform functions shifted by a bit depth outside 10..=16.
Three-way merge of the changes EffectCraft made to its av1enc and hevcenc
crates after craft-video took them:
- hevc: EncoderConfig carries colour_primaries, transfer_characteristics and
matrix_coeffs (H.273 code points, BT.709 by default) and the VUI writes them.
- av1: EncoderConfig carries color_primaries, transfer_characteristics and
matrix_coefficients and color_config writes them.
- av1: motion search keeps a visited set and stops a SAD once it cannot win,
SATD has fixed-size kernels, and a decision snapshot saves only the block's
BlockDecoded flags. Output is unchanged.
ffprobe tests check the colour description of both streams.
* craft-video: H.264 decoder from FilmCraft (`h264-decode` feature)
FilmCraft's crates/h264 at 4c0d2ad7 becomes craft_video::h264::decode. The encoder moves to
h264/encode and is still exported from craft_video::h264. Feature h264 now turns on h264-decode
and h264-encode, so either half can be built without the other.
Without the threads feature a reference row that was never published read through expect();
it now reads as a gray row. craft_video::budget (the picture memory budget from DeckCraft's
bitstream crate) is added for the next commit. craft-video is built at opt-level 3 in the dev
profile, as FilmCraft built its codecs.
* craft-video: H.264 decoder hardening from DeckCraft's copy
- The picture store (DPB, current picture, pictures in flight or queued for output) is checked
against a memory budget before a picture is allocated; Decoder::set_memory_budget.
- configure_avcc checks every entry before handling any: SPS NAL units in the first list, PPS in
the second. A picture slice in the record starts no picture.
- The inverse transforms clamp coefficients to the spec's r range instead of overflowing.
- Temporal direct motion vectors are scaled in i32 and clipped to the spec range.
- slice_qp_delta is added with a checked add.
- A picture job sets up its picture state inside the panic guard.
* craft-video: HEVC decoder from FilmCraft (`hevc-decode` feature)
FilmCraft's crates/hevc at 4c0d2ad7 becomes craft_video::hevc::decode. The encoder moves to
hevc/encode and is still exported from craft_video::hevc. Feature hevc now turns on hevc-decode
and hevc-encode, so either half can be built without the other.
craft_video::budget (the picture memory budget from DeckCraft's bitstream crate) is added for the
next commit. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its
codecs.
* craft-video: HEVC decoder hardening from DeckCraft's copy
- The picture store (DPB, current picture, pictures in flight) is checked against a memory
budget before a picture is allocated; Decoder::set_memory_budget.
- configure_hvcc checks every array before handling any NAL: only VPS, SPS, PPS and SEI arrays,
each NAL header agreeing with its array type. A picture slice in the record starts no picture.
- init_qp_minus26, slice_qp_delta and delta_chroma_log2_weight_denom are range checked before
the additions that overflowed on Exp-Golomb values near the i32 limits.
- A picture job sets up its picture state inside the panic guard.
* craft-video: VP9 decoder from FilmCraft (`vp9-decode` feature)
FilmCraft's crates/vp9 at 4c0d2ad7 becomes craft_video::vp9::decode. The encoder moves to
vp9/encode and is still exported from craft_video::vp9. Feature vp9 now turns on vp9-decode and
vp9-encode, so either half can be built without the other.
The decoder's tests find ffmpeg and the fixture directory through craft-media-testkit; a fixture
that cannot be generated fails under CRAFT_MEDIA_REQUIRE_ORACLES=1. craft-video is built at
opt-level 3 in the dev profile, as FilmCraft built its codecs.
* craft-video: VP9 decoder checks its picture store against a memory budget
DeckCraft's copy of the decoder refuses a frame size whose pictures (eight reference slots, the
new frame and those in flight) would not fit a byte budget. The budget type comes with it as
craft_video::budget; FilmCraft's 2^26 luma sample cap stays.
The eight-part craft-video series landed as one commit, because each part's branch contains the parts before it.
- #702 dnx: DNxHD/DNxHR decoder and encoder
- #703 vp8: VP8 decoder
- #704 prores: ProRes decoder and encoder
- #705 h264: H.264 encoder
- #706 hevc: HEVC encoder
- #707 vp9: VP9 encoder
- #708 av1: AV1 encoder
Aleksander Helgaker
Melchi Aaron Zedek
Justin Wilson
el-j