filmcraft/crates/prores

filmcraft-prores

Clean-room Apple ProRes decoder and encoder in pure Rust, written from the public SMPTE RDD 36 bitstream description. Layer L0: depends only on filmcraft-bitstream, thiserror and (optional, feature threads, default on) rayon. Builds for wasm32-unknown-unknown; no unsafe.

API

use filmcraft_prores::*;

// Decode one QuickTime sample (one ProRes frame).
let frame: Frame = decode_frame(&sample)?;                 // 10-bit 4:2:2 / 12-bit 4:4:4
let f10 = decode_frame_with(&sample, &DecodeOptions { bit_depth: Some(10), threads: true })?;
decode_frame_into(&sample, &DecodeOptions::default(), &mut frame_buf)?; // reuses planes
let hdr: FrameHeader = probe(&sample)?;                    // header only

// Encode (progressive).
let mut enc = Encoder::new(Profile::Hq, 1920, 1080);       // or Encoder::with_config(EncoderConfig { .. })
let sample: Vec<u8> = enc.encode(&frame)?;                 // the QuickTime sample for fourcc Profile::fourcc()

Frame { width, height, chroma: ChromaFormat::{Yuv422, Yuv444}, bit_depth, y, cb, cr: Vec<u16>, alpha: Option<Vec<u16>>, interlace: Interlace, color: ColorInfo { primaries, transfer, matrix }, aspect_ratio, frame_rate_code } holds planar samples with no row padding (chroma_width() is ceil(width/2) for 4:2:2). The colour codes are H.273 code points straight from the frame header.

Profile = Proxy (apco), Lt (apcs), Standard (apcn), Hq (apch), P4444 (ap4h), P4444Xq (ap4x). The frame itself doesn't say which profile it is (only the container fourcc does), so the decoder doesn't need to know.

Decoder

• Frame header (dimensions, chroma format, interlace mode, aspect/frame-rate codes, colour description, alpha type, optional luma/chroma quantisation matrices), picture header(s), slice table, slices.
• Entropy decoding: RDD 36's adaptive Golomb-Rice/Exp-Golomb codebooks. DC is coded as the first value plus sign-predicted differences. AC is run/level coded, interleaved across all blocks of a slice component.
• Progressive and interlaced scan orders. Interlaced frames are two field pictures (top or bottom field first) of ceil(height/32) macroblock rows each.
• Dequantisation F = level × qmat[i] × qscale(qidx) (qscale = qidx up to 128, then 128 + 4·(qidx−128)).
• IDCT: the exact 2-D orthonormal inverse DCT, evaluated separably in f32. It uses even/odd symmetry, skips zero rows and fills DC-only blocks directly. Its error is below 2⁻⁸ in the 12-bit domain. The IDCT output f is the 12-bit-domain sample minus 2048. The output at depth d is floor(f·2^(d−12) + 2^(d−1) + ½), clipped to [4, 2^d − 5] for d ≥ 10 (the four lowest/highest codes are SDI reserved values; this matches common decoders at 10 and 12 bits) and to [1, 2^d − 2] below 10 bits.
• 4:2:2 chroma blocks per macroblock are top/bottom. 4:4:4 chroma blocks run TL, BL, TR, BR, while luma runs TL, TR, BL, BR.
• Alpha (8- or 16-bit coding): lossless difference/run coding in raster order over each slice's 16 lines. The starting value is all-ones. Output is scaled to the output depth: 8-bit coding is bit-replicated, 16-bit coding is truncated.
• Default output depth: 10 bits for 4:2:2 and 12 bits for 4:4:4. Any depth from 8 to 16 is available on request.
• Slice-parallel: each macroblock row, covering both fields of interlaced frames, is decoded by a rayon task directly into the output planes.
• Robustness: every size field is validated before allocation, codewords longer than 56 bits are rejected, and arithmetic is overflow-safe. Mutated streams return Err, never panic (see tests/fuzz.rs).

Encoder

• Progressive ProRes 422 (Proxy/LT/Standard/HQ) and 4444/4444 XQ. For 4444 the frame's alpha plane is coded with 16-bit alpha coding by default (EncoderConfig::encode_alpha).
• Input: planar u16 at any depth from 8 to 16 (normally 10). The chroma format must match the profile. Edge macroblocks are padded by replicating the edge.
• Flat quantisation matrices (all 4, not transmitted), 8 macroblocks per slice by default (log2_slice_mbs), encoder id fmcr.
• Rate control targets Apple's published nominal rates (Mbit/s at 1920×1080, 29.97 fps: 45, 102, 147, 220, 330, 500). These are converted to bits per macroblock (184, 417, 601, 900, 1349, 2045), so frame targets scale with frame area (Encoder::target_frame_bytes). Alpha comes on top of the target. Every slice is transformed once. Then a frame-level binary search finds the smallest uniform quantiser index whose frame fits the budget, and as many slices as the rest of the budget allows get one finer step. Detailed content lands within 0.1% of the target. Simple content uses qidx 1 and comes in under the target. Every slice is kept within the 16-bit size fields.

Accuracy (vs ffmpeg as an external oracle)

tests/oracle_decode.rs decodes 29 fixtures made by ffmpeg's prores_ks and prores_aw encoders. They cover every profile, testsrc2/mandelbrot/SMPTE bars at 1920×1080, 1918×1080, 1280×720, 640×360, 352×288, 70×38 and 333×201, interlaced 720×486 TFF/BFF and 1080i, 2-MB slices, and 4444 with 8- and 16-bit alpha. The results are compared with ffmpeg's decode at the native depth (10-bit 4:2:2, 12-bit 4:4:4). Every fixture is within ±1 LSB. Mean absolute error is 0.0001–0.03 LSB for 422 and up to 0.15 LSB for 12-bit XQ, and 97–99.99% of samples are bit-identical (86% for 12-bit XQ mandelbrot). Alpha is always bit-exact. The remaining differences are the reference decoder's integer-IDCT rounding. This decoder is closer to the exact IDCT and does not reproduce that approximation.

tests/oracle_encode.rs encodes 1920×1080 frames with every profile (testsrc2, mandelbrot and noisy content; 4444 with an alpha disc) and writes them into a minimal test-only MOV. ffmpeg then decodes them with -xerror, with no errors, and agrees with this decoder within ±1. HQ PSNR is 69.9/69.2 dB (Y/C) on mandelbrot at 100% of the target rate. Busy content lands at 99.9–100.4% of the target.

Performance

Measured with cargo test --release -p filmcraft-prores --test perf -- --ignored --nocapture on an M4 Pro (14 cores) that was heavily loaded by other jobs, so these are lower bounds. A 1080p HQ frame at the full nominal rate (918 KB) decoded at up to 318 fps threaded and 53 fps on one thread. A prores_ks HQ mandelbrot frame (657 KB) decoded at 390 fps threaded. Encoding a 1080p HQ frame takes about 40 ms threaded.

Gaps

• The encoder is progressive-only (the decoder handles interlaced), uses flat quantisation matrices (no perceptual matrices), and has no trellis/RDO.
• Output is not bit-exact with ffmpeg's integer IDCT; it stays within ±1 LSB of it (see above).
• The 4:2:2 decoder output is fixed at the coded 10-bit precision unless another depth is requested; there is no direct output of packed formats (v210, etc.).
• No container handling: frames are QuickTime samples, and demuxing/muxing lives in filmcraft-isobmff.