daniel543x/craft-libs
1
0 Fork 0
mirror of https://github.com/storytold/craft-libs.git synced 2026-11-13 22:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

craft-video: HEVC 4:2:2 8/10-bit decoding (Main 4:2:2 10) (#1087)

Browse Source 
Port of storytold/filmcraft#1952, reviewed and approved there before the
decoder moved to craft-libs. 4:2:2 streams that use no range-extension
coding tools (what cameras and libx265 write) now decode:

- chroma geometry from the SPS chroma format (ChromaFormat shifts) in the
  picture store, intra, inter, transform tree, deblocking and SAO;
- Table 8-3 IntraPredModeC mapping for ChromaArrayType 2;
- stacked chroma transform blocks with their second cbf flags;
- chroma QP = Min(qPi, 51) for non-4:2:0 (Table 8-10 applies to 4:2:0 only);
- chroma motion vectors scaled per axis;
- a PPS that scales SAO offsets (a RExt tool) is refused as unsupported.

Tests: 22 libx265 4:2:2 fixtures (8- and 10-bit) compared sample-exactly
with ffmpeg, the Table 8-3 derivation, and the SAO offset scale check.
This commit is contained in:
Melchi Aaron Zedek authored and GitHub committed 2026-11-11 23:52:20 -05:00
1 parent 09dcbc225b
commit bb57d422d0
16 files changed
+526 -178
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

crates/video/src/hevc/decode/README.md
+21 -10
View File
@@ -31,18 +31,19 @@ and fixture generators.
| Frame-level multithreading (CTB-row granularity dependencies) | done |
| Draft mode (non-reference pictures skip deblocking / SAO; reduced-resolution playback only) | done |
| Tiles / WPP | parsed and decoded sequentially within a picture (parallelism is across pictures) |
| 4:2:2 / 4:4:4 / monochrome, range-extension tools (RExt), 12-bit+ with extended precision | not yet (`Error::Unsupported`) |
| 4:2:2 (Main 4:2:2 10 / RExt profile): chroma geometry, Table 8-3 intra mode mapping, stacked chroma transform blocks with their second cbf flags, chroma QP = Min(qPi, 51), chroma motion vectors, deblocking / SAO geometry; 8–12-bit | implemented; bit-exact on the 8- and 10-bit libx265 fixtures and a Fujifilm X-S20 clip; 12-bit not yet validated |
| 4:4:4 / monochrome, range-extension coding tools (RDPCM, cross-component prediction, extended precision, persistent Rice adaptation, bypass alignment, chroma QP offset lists) | not yet (`Error::Unsupported`) |
| SCC, multilayer (SHVC / MV-HEVC), 3D extensions | not supported (layers > 0 are ignored) |
## API
```rust
use craft_video::hevc::decode::{Decoder, Plane};
use craft_video::hevc::decode::{ChromaFormat, Decoder, Plane};
let mut dec = Decoder::new(); // or Decoder::with_threads(n), Decoder::from_hvcc(&hvcc)?
for (pts, access_unit) in access_units {
for pic in dec.decode(access_unit, pts)? {
// pic.width / pic.height (cropped), pic.bit_depth,
// pic.width / pic.height (cropped), pic.bit_depth, pic.chroma (ChromaFormat::C420 / C422),
// pic.y / pic.u / pic.v: Plane::U8(Vec<u8>) for 8-bit, Plane::U16(Vec<u16>) above,
// pic.y_stride / pic.uv_stride, pic.pts, pic.poc, pic.key (IRAP), pic.color (VUI), pic.sar
}
@@ -112,7 +113,8 @@ with ffmpeg; tests print a message and skip when ffmpeg is absent).
- Robustness: hvcC input with pts round trip; randomly corrupted, truncated and garbage input never
panics or deadlocks (single- and multi-threaded).
- Conformance (`tests/it/conformance.rs`): every fixture is decoded with 1 thread, 3 threads and every
core and compared sample-exactly with `ffmpeg -f rawvideo -pix_fmt yuv420p / yuv420p10le`; on mismatch
core and compared sample-exactly with `ffmpeg -f rawvideo -pix_fmt yuv420p / yuv420p10le /
yuv422p / yuv422p10le`; on mismatch
the first differing frame, plane, sample, CTB and 8x8 block are reported. Output pts/POC order is
checked too, and that no picture is flagged draft without draft mode. `compare_detects_mismatch`
guards the harness itself. Draft mode (`draft_mode_changes_only_flagged_non_reference_pictures`):
@@ -148,6 +150,12 @@ libx265 unless noted (x265 enables WPP by default, so most streams carry entry p
| hd_1080p | 1920x1080 | preset fast |
| uhd_4k_main10 | 3840x2160 | Main 10 ultrafast |
| vt_main / vt_main10 | 640x360 | Apple VideoToolbox (hevc_videotoolbox), macOS only |
| intra_422 / intra_422_10 / intra_422_10_nosao_nodbk | 352x288 | 4:2:2 8 / 10-bit intra, with and without the in-loop filters |
| medium_422_10 / slow_422 / p_only_422_10 / bframes_422_10 | 352x288 | 4:2:2 presets, P only, B pyramid |
| no_sao_422_10 / deblock_m3_3_422_10 / rect_amp_422_10 / tskip_422_10 / weightp_422_10 | 352x288 | 4:2:2 tool toggles |
| qp_422_10 / crf51_422_10 / lossless_422_10 | 352x288 / 176x144 | 4:2:2 adaptive QP (cu_qp_delta), quantizer extremes, lossless |
| ctu32_tu4_422_10 / ctu16_422_10 / scaling_list_422_10 / constrained_intra_422_10 / slices4_422_10 | 352x288 | 4:2:2 block structure, scaling lists, constrained intra, slices |
| odd_1918x1078_422_10 / uhd_4k_422_10 | 1918x1078 / 3840x2160 | 4:2:2 cropping and 4K |
| synthetic (7 streams) | 96x64 | tiles, PCM, slices, dependent segments, LTR, list modification |
`cargo test -p craft-video --features hevc-decode,threads --test hevc_decode conformance::coverage_report -- --ignored --nocapture` prints the
@@ -164,6 +172,10 @@ average 50-100), so these are lower bounds:
| 1080p Main, x265 medium CRF 24, 60 frames | ~75 fps | ~225 fps |
| 2160p Main 10, x265 fast CRF 26, 20 frames | ~20 fps | ~95 fps |
4:2:2 10-bit at 2160p (a Fujifilm X-S20 clip, 87 Mbit/s LongGOP) decodes at about 17 fps with 16
threads on a Ryzen 7 7730U laptop (the `hevc_hevcdec` example, all pictures output): below real time
for 24–30 fps material.
No intrinsics: the hot loops are written to auto-vectorise. M4.8
profiling (FilmCraft's `cargo xtask bench`, macOS `sample`) found the inverse transform at over half of the
decoder's CPU time: it evaluated the full matrix product with strided table reads. It now sums
@@ -200,12 +212,11 @@ context-coded bins are branch-free. Single-threaded 4K now splits into CABAC res
## Known gaps
- 4:2:2 (Main 4:2:2 10, RExt) is the next step: chroma geometry is 4:2:0-specific in `slicedec`
(chroma TU placement, second chroma cbf flags, Table 8-3 intra mode mapping), `inter` (vertical
chroma MV scaling), `filter` (chroma edge spacing, QpC = Min(qPi, 51)) and `picture`. Range
extension tools (implicit / explicit RDPCM, cross-component prediction, extended precision,
persistent Rice adaptation, bypass alignment, chroma QP offset lists) are rejected with
`Error::Unsupported`.
- 4:4:4 and monochrome streams, and the range extension coding tools (implicit / explicit RDPCM,
cross-component prediction, extended precision, persistent Rice adaptation, bypass alignment,
chroma QP offset lists, SAO offset scaling) are rejected with `Error::Unsupported`. 4:2:2
streams that use none of those tools (what cameras and libx265 write) decode.
- 12-bit 4:2:2 is implemented but no fixture or ffmpeg comparison covers `yuv422p12le` yet.
- Tiles and WPP substreams are decoded sequentially (entry points are parsed but the decoder finds
substream starts itself); parallelism is at the picture level.
- Error concealment is minimal: missing references are replaced by grey pictures, undecodable
crates/video/src/hevc/decode/decoder.rs
+13 -7
View File
@@ -6,12 +6,12 @@
//! block per row on the reference data they read, so several pictures decode concurrently.
use crate::hevc::decode::dpb::{Dpb, Output, OutputMeta, PocState, RefPicSet, build_ref_lists};
use crate::hevc::decode::error::{Error, Result, ensure, invalid};
use crate::hevc::decode::error::{Error, Result, ensure, invalid, unsupported};
use crate::hevc::decode::params::{Layout, Pps, Sps, Vps};
use crate::hevc::decode::picture::{Frame, FrameRef, Sample, stores_u8};
use crate::hevc::decode::slice::{NalHeader, SliceHeader, nal_type};
use crate::hevc::decode::slicedec::{Blk, F_INTRA, F_SKIP, PicState, SliceDecoder, SliceJob};
use crate::hevc::decode::{ColorInfo, Picture};
use crate::hevc::decode::{ChromaFormat, ColorInfo, Picture};
use craft_media::bitstream::{annexb_nals, length_prefixed_nals, unescape_rbsp};
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
@@ -490,6 +490,7 @@ impl Decoder {
Some(a) => {
a.width != sps.width
|| a.height != sps.height
|| a.chroma_format_idc != sps.chroma_format_idc
|| a.bit_depth_luma != sps.bit_depth_luma
|| a.bit_depth_chroma != sps.bit_depth_chroma
|| a.log2_ctb != sps.log2_ctb
@@ -520,7 +521,10 @@ impl Decoder {
let max_lsb = sps.max_poc_lsb();
let poc = self.poc_state.compute(sh, max_lsb, irap_no_rasl);
let (w, h) = (sps.width as usize, sps.height as usize);
let template = Frame::new(0, 0, w, h, sps.log2_ctb, sps.bit_depth_luma, sps.bit_depth_chroma);
let Some(chroma) = ChromaFormat::from_idc(sps.chroma_array_type()) else {
return unsupported(format!("chroma_format_idc {}", sps.chroma_format_idc));
};
let template = Frame::new(0, 0, w, h, chroma, sps.log2_ctb, sps.bit_depth_luma, sps.bit_depth_chroma);
let next_id = &mut self.next_id;
let gray_reference = &mut self.gray_reference;
let mut missing = 0u64;
@@ -549,7 +553,7 @@ impl Decoder {
let _ = scaling;
let id = self.next_id;
self.next_id += 1;
let frame = Arc::new(Frame::new(id, poc, w, h, sps.log2_ctb, sps.bit_depth_luma, sps.bit_depth_chroma));
let frame = Arc::new(Frame::new(id, poc, w, h, chroma, sps.log2_ctb, sps.bit_depth_luma, sps.bit_depth_chroma));
let output = sh.pic_output && !(hdr.is_rasl() && self.skip_rasl);
// Draft mode: a sub-layer non-reference picture of the highest sub-layer is never used
// for prediction, so its in-loop filters can be left out without changing anything else.
@@ -695,7 +699,8 @@ fn output_meta(sps: &Sps, pts: i64, key: bool, draft: bool) -> OutputMeta {
fn make_picture(o: &Output<FrameRef>) -> Picture {
let (cx, cy, cw, ch) = o.meta.crop;
let (cx, cy, cw, ch) = (cx as usize, cy as usize, cw as usize, ch as usize);
let (ccw, cch) = (cw.div_ceil(2), ch.div_ceil(2));
let chroma = o.frame.chroma;
let (ccw, cch) = (cw.div_ceil(chroma.sub_width()), ch.div_ceil(chroma.sub_height()));
let bd = o.meta.bit_depth;
let (y, u, v) = o.frame.cropped_planes((cx, cy, cw, ch), bd <= 8);
Picture {
@@ -703,6 +708,7 @@ fn make_picture(o: &Output<FrameRef>) -> Picture {
height: ch as u32,
chroma_width: ccw as u32,
chroma_height: cch as u32,
chroma,
bit_depth: bd,
y,
u,
@@ -742,7 +748,7 @@ mod tests {
draft: false,
});
for id in 0..(MAX_QUEUED_AFTER_ERROR as u32 * 4) {
let frame = Arc::new(Frame::new(id, id as i32, 16, 16, 4, 8, 8));
let frame = Arc::new(Frame::new(id, id as i32, 16, 16, ChromaFormat::C420, 4, 8, 8));
decoder.out_queue.push_back(Output { frame, poc: id as i32, meta: meta.clone() });
}
decoder
@@ -770,7 +776,7 @@ mod tests {
bit_depth: 8,
draft: false,
});
decoder.out_queue.push_back(Output { frame: Arc::new(Frame::new(0, 0, 16, 16, 4, 8, 8)), poc: 0, meta });
decoder.out_queue.push_back(Output { frame: Arc::new(Frame::new(0, 0, 16, 16, ChromaFormat::C420, 4, 8, 8)), poc: 0, meta });
assert!(decoder.check_budget(&sps).is_err());
}
crates/video/src/hevc/decode/filter.rs
+42 -26
View File
@@ -7,7 +7,7 @@
use crate::hevc::decode::picture::{ColMv, FrameRow, Sample};
use crate::hevc::decode::slicedec::{E_PU_H, E_PU_V, E_TU_H, E_TU_V, F_CBF, F_CODED, F_INTRA, F_NOFILTER, MvField, PicState};
use crate::hevc::decode::spec_tables::{BETA_TABLE, TC_TABLE};
use crate::hevc::decode::tables::qpc_420;
use crate::hevc::decode::tables::qpc;
/// Called after each decoded CTB of CTB row `ry`.
pub fn ctb_decoded<S: Sample>(pic: &mut PicState<S>, ry: usize) {
@@ -141,6 +141,10 @@ fn deblock_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
let (w, w4) = (pic.width, pic.w4);
let (bd, bdc) = (pic.bd_y, pic.bd_c);
let (cb_off, cr_off) = (pic.pps.cb_qp_offset, pic.pps.cr_qp_offset);
// chroma geometry: edges on the 8x8 chroma sample grid (8.7.2, edgeSpacing), each segment
// 4 chroma samples long, with bS taken from the luma segment at its first sample
let (sx, sy) = pic.chroma.shifts();
let is420 = sy == 1;
// vertical edges
let mut bs_v = vec![0i8; w4 * ((y1 - y0) / 4)];
for by in y0 / 4..y1 / 4 {
@@ -164,10 +168,11 @@ fn deblock_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
filter_luma(&mut pic.planes[0], EdgeSeg { o: y * w + x, step: 1, along: w }, beta, EdgeFilter { tc, nfp, nfq, bd });
}
}
// chroma vertical edges (8x8 chroma grid = 16 luma samples), bS == 2 only
// chroma vertical edges (8 chroma samples = 16 luma samples apart; a 4-line chroma segment
// spans 4 << sy luma lines), bS == 2 only
let cw = pic.cwidth;
for by in (y0 / 4..y1 / 4).step_by(2) {
for bx in (4..w4).step_by(4) {
for by in (y0 / 4..y1 / 4).step_by(1 << sy) {
for bx in (2 << sx..w4).step_by(2 << sx) {
if bs_v[(by - y0 / 4) * w4 + bx] != 2 {
continue;
}
@@ -178,9 +183,13 @@ fn deblock_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
let (nfp, nfq) = (p.flags & F_NOFILTER != 0, q.flags & F_NOFILTER != 0);
for (c, off) in [(1, cb_off), (2, cr_off)] {
let qpi = ((p.qp as i32 + q.qp as i32 + 1) >> 1) + off;
let qt = (qpc_420(qpi) + 2 + tc_off).clamp(0, 53);
let qt = (qpc(qpi, is420) + 2 + tc_off).clamp(0, 53);
let tc = (TC_TABLE[qt as usize] as i32) << (bdc - 8);
filter_chroma(&mut pic.planes[c], EdgeSeg { o: (y / 2) * cw + x / 2, step: 1, along: cw }, EdgeFilter { tc, nfp, nfq, bd: bdc });
filter_chroma(
&mut pic.planes[c],
EdgeSeg { o: (y >> sy) * cw + (x >> sx), step: 1, along: cw },
EdgeFilter { tc, nfp, nfq, bd: bdc },
);
}
}
}
@@ -208,8 +217,10 @@ fn deblock_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
filter_luma(&mut pic.planes[0], EdgeSeg { o: y * w + x, step: w, along: 1 }, beta, EdgeFilter { tc, nfp, nfq, bd });
}
}
for by in (y0 / 4..y1 / 4).filter(|b| b % 4 == 0 && *b > 0) {
for bx in (0..w4).step_by(2) {
// chroma horizontal edges (8 chroma lines = 8 << sy luma lines apart; a 4-sample chroma
// segment spans 4 << sx luma samples), bS == 2 only
for by in (y0 / 4..y1 / 4).filter(|b| b % (2 << sy) == 0 && *b > 0) {
for bx in (0..w4).step_by(1 << sx) {
if bs_h[(by - y0 / 4) * w4 + bx] != 2 {
continue;
}
@@ -220,9 +231,13 @@ fn deblock_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
let (nfp, nfq) = (p.flags & F_NOFILTER != 0, q.flags & F_NOFILTER != 0);
for (c, off) in [(1, cb_off), (2, cr_off)] {
let qpi = ((p.qp as i32 + q.qp as i32 + 1) >> 1) + off;
let qt = (qpc_420(qpi) + 2 + tc_off).clamp(0, 53);
let qt = (qpc(qpi, is420) + 2 + tc_off).clamp(0, 53);
let tc = (TC_TABLE[qt as usize] as i32) << (bdc - 8);
filter_chroma(&mut pic.planes[c], EdgeSeg { o: (y / 2) * cw + x / 2, step: cw, along: 1 }, EdgeFilter { tc, nfp, nfq, bd: bdc });
filter_chroma(
&mut pic.planes[c],
EdgeSeg { o: (y >> sy) * cw + (x >> sx), step: cw, along: 1 },
EdgeFilter { tc, nfp, nfq, bd: bdc },
);
}
}
}
@@ -379,11 +394,13 @@ fn filter_chroma<S: Sample>(s: &mut [S], seg: EdgeSeg, ef: EdgeFilter) {
/// SAO of CTB row `r` from the deblocked planes into new row buffers (luma, Cb, Cr).
fn sao_row<S: Sample>(pic: &PicState<S>, r: usize) -> [Vec<S>; 3] {
let ctb = 1usize << pic.log2_ctb;
let (sx, sy) = pic.chroma.shifts();
let mut out: [Vec<S>; 3] = [Vec::new(), Vec::new(), Vec::new()];
for c in 0..3 {
let (w, h, cs) = if c == 0 { (pic.width, pic.height, ctb) } else { (pic.cwidth, pic.cheight, ctb / 2) };
let y0 = r * cs;
let y1 = (y0 + cs).min(h);
// CTB size in this component's samples, horizontally and vertically
let (w, h, csx, csy) = if c == 0 { (pic.width, pic.height, ctb, ctb) } else { (pic.cwidth, pic.cheight, ctb >> sx, ctb >> sy) };
let y0 = r * csy;
let y1 = (y0 + csy).min(h);
out[c] = pic.planes[c][y0 * w..y1 * w].to_vec();
for rx in (0..pic.wctb).filter(|_| !pic.draft) {
let rs = r * pic.wctb + rx;
@@ -399,8 +416,8 @@ fn sao_row<S: Sample>(pic: &PicState<S>, r: usize) -> [Vec<S>; 3] {
if p.type_idx[c] == 0 {
continue;
}
let x0 = rx * cs;
let x1 = (x0 + cs).min(w);
let x0 = rx * csx;
let x1 = (x0 + csx).min(w);
sao_ctb(pic, &SaoBlock { c, rs, x0, x1, y0, y1, w, h }, &p, &mut out[c]);
}
}
@@ -425,7 +442,8 @@ fn sao_ctb<S: Sample>(pic: &PicState<S>, blk: &SaoBlock, p: &crate::hevc::decode
let SaoBlock { c, rs, x0, x1, y0, y1, w, h } = *blk;
let bd = if c == 0 { pic.bd_y } else { pic.bd_c };
let max = (1i32 << bd) - 1;
let sc = if c == 0 { 0 } else { 1 };
// shifts from this component's sample positions to luma positions
let (scx, scy) = if c == 0 { (0, 0) } else { pic.chroma.shifts() };
let offs = p.offsets[c].map(|v| v as i32);
let plane = &pic.planes[c];
if p.type_idx[c] == 1 {
@@ -439,16 +457,17 @@ fn sao_ctb<S: Sample>(pic: &PicState<S>, blk: &SaoBlock, p: &crate::hevc::decode
// Samples of CUs that must not be modified (transquant bypass, PCM without loop filter) get
// their deblocked values back.
if pic.pps.transquant_bypass || pic.sps.pcm_loop_filter_disabled {
let n = 4 >> sc;
for by in ((y0 << sc) >> 2)..(((y1 << sc) + 3) >> 2).min(pic.h4) {
let ya = (by * 4) >> sc;
for bx in ((x0 << sc) >> 2)..(((x1 << sc) + 3) >> 2).min(pic.w4) {
// one 4x4 luma block covers (4 >> scx) x (4 >> scy) samples of this component
let (nx, ny) = (4 >> scx, 4 >> scy);
for by in ((y0 << scy) >> 2)..(((y1 << scy) + 3) >> 2).min(pic.h4) {
let ya = (by * 4) >> scy;
for bx in ((x0 << scx) >> 2)..(((x1 << scx) + 3) >> 2).min(pic.w4) {
if pic.blk[by * pic.w4 + bx].flags & F_NOFILTER == 0 {
continue;
}
let xa = (bx * 4) >> sc;
let xb = (xa + n).min(x1);
for y in ya..(ya + n).min(y1) {
let xa = (bx * 4) >> scx;
let xb = (xa + nx).min(x1);
for y in ya..(ya + ny).min(y1) {
dst[(y - y0) * w + xa..(y - y0) * w + xb].copy_from_slice(&plane[y * w + xa..y * w + xb]);
}
}
@@ -625,8 +644,6 @@ fn sao_band_run<S: Sample>(src: &[S], out: &mut [S], shift: u32, (band_pos, offs
fn publish_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
let ctb = 1usize << pic.log2_ctb;
let (y0, y1) = (r * ctb, ((r + 1) * ctb).min(pic.height));
let (c0, c1) = (y0 / 2, y1.div_ceil(2));
let (w, cw) = (pic.width, pic.cwidth);
let w16 = pic.width.div_ceil(16);
let rows16 = (y1 - y0).div_ceil(16);
let mut col = vec![ColMv::default(); w16 * rows16];
@@ -662,7 +679,6 @@ fn publish_row<S: Sample>(pic: &mut PicState<S>, r: usize) {
}
}
let [oy, ocb, ocr] = sao_row(pic, r);
let _ = (w, cw, c0, c1);
let row = FrameRow { y: oy.into(), cb: ocb.into(), cr: ocr.into(), col: col.into() };
pic.frame.publish(r, row);
}
crates/video/src/hevc/decode/inter.rs
+57 -28
View File
@@ -272,19 +272,28 @@ pub fn mc_luma<S: Sample>(f: &Frame, (x, y): (i32, i32), mv: [i16; 2], (w, h): (
predict_block::<S, 8>(f, 0, (xi - 3, yi - 3), &k, out, sc);
}
/// Chroma prediction samples of a w x h chroma block at chroma position (x, y) with a 1/8-sample
/// vector (4:2:0).
pub fn mc_chroma<S: Sample>(f: &Frame, c: usize, (x, y): (i32, i32), mv: [i16; 2], (w, h): (usize, usize), out: &mut [i16], sc: &mut McScratch) {
/// Chroma prediction samples of a w x h chroma block at chroma position (x, y) displaced by the
/// chroma motion vector `mv` in units of 1/8 chroma sample (mvCLX of 8.5.3.2.10: the luma vector
/// times 2 / SubWidthC horizontally and 2 / SubHeightC vertically; see [`chroma_mv`]).
pub fn mc_chroma<S: Sample>(f: &Frame, c: usize, (x, y): (i32, i32), mv: [i32; 2], (w, h): (usize, usize), out: &mut [i16], sc: &mut McScratch) {
let bd = f.bit_depth_c;
let (fx, fy) = ((mv[0] & 7) as usize, (mv[1] & 7) as usize);
let xi = x + (mv[0] as i32 >> 3);
let yi = y + (mv[1] as i32 >> 3);
let xi = x + (mv[0] >> 3);
let yi = y + (mv[1] >> 3);
let shift1 = bd.min(12) - 8;
let shift3 = 14 - bd;
let k = Interp { fx, fy, hf: &CHROMA_FILTER[fx], vf: &CHROMA_FILTER[fy], w, h, shift1, shift3 };
predict_block::<S, 4>(f, c + 1, (xi - 1, yi - 1), &k, out, sc);
}
/// The chroma motion vector in 1/8 chroma sample units (8-228 / 8-229 of 8.5.3.2.10) of the luma
/// quarter-sample vector `mv` for chroma shifts `(sx, sy)` = (log2 SubWidthC, log2 SubHeightC):
/// `mv * 2 / SubWidthC` horizontally, `mv * 2 / SubHeightC` vertically (both exact).
#[inline(always)]
pub fn chroma_mv(mv: [i16; 2], (sx, sy): (u32, u32)) -> [i32; 2] {
[(mv[0] as i32 * 2) >> sx, (mv[1] as i32 * 2) >> sy]
}
/// Calls `$f::<_, W>(args)` with `W` the prediction block width `$w` as a constant (block widths are
/// fixed sizes; a constant width lets the row loops vectorise even for 4- and 8-sample rows), or
/// `$f::<0>` for any other width.
@@ -391,6 +400,7 @@ pub fn put_weighted_bi<S: Sample>(
#[cfg(test)]
mod tests {
use super::*;
use crate::hevc::decode::ChromaFormat;
struct Rng(u64);
impl Rng {
@@ -530,36 +540,40 @@ mod tests {
}
}
fn frame(rng: &mut Rng, w: usize, h: usize, bd: u32) -> Frame {
let like = Frame::new(0, 0, w, h, 4, bd, bd);
fn frame(rng: &mut Rng, w: usize, h: usize, chroma: ChromaFormat, bd: u32) -> Frame {
let like = Frame::new(0, 0, w, h, chroma, 4, bd, bd);
let max = (1i32 << bd) - 1;
let n = (w >> chroma.shifts().0) * (h >> chroma.shifts().1);
let y: Vec<u16> = (0..w * h).map(|_| rng.range(0, max) as u16).collect();
let c: Vec<u16> = (0..w * h / 4).map(|_| rng.range(0, max) as u16).collect();
let c2: Vec<u16> = (0..w * h / 4).map(|_| rng.range(0, max) as u16).collect();
let c: Vec<u16> = (0..n).map(|_| rng.range(0, max) as u16).collect();
let c2: Vec<u16> = (0..n).map(|_| rng.range(0, max) as u16).collect();
Frame::from_planes(1, 0, &like, &y, &c, &c2)
}
/// Interpolation from the reference lines in place gives the same prediction samples as from
/// an edge-replicated copy of the window, for every fractional position, block size and bit
/// depth, inside the picture and next to its edges (where in-place reads are used for windows
/// that cross only the top or bottom edge).
/// an edge-replicated copy of the window, for every fractional position, block size, chroma
/// format and bit depth, inside the picture and next to its edges (where in-place reads are
/// used for windows that cross only the top or bottom edge).
#[test]
fn in_place_window_matches_the_copied_window() {
fn check<S: Sample>(rng: &mut Rng, bd: u32) {
fn check<S: Sample>(rng: &mut Rng, chroma: ChromaFormat, bd: u32) {
let mut sc = McScratch::default();
let f = frame(rng, 96, 64, bd);
let f = frame(rng, 96, 64, chroma, bd);
let (sx, sy) = chroma.shifts();
let (mut in_place, mut copied) = (0, 0);
for _ in 0..2000 {
let c = rng.range(0, 2) as usize;
let (w, h) = ([4usize, 8, 12, 16, 24, 32][rng.range(0, 5) as usize], [4usize, 8, 16, 32][rng.range(0, 3) as usize]);
let (w, h) = if c == 0 { (w, h) } else { (w / 2, h / 2) };
let (w, h) = if c == 0 { (w, h) } else { (w >> sx, h >> sy) };
let (x, y) = (rng.range(-12, 100), rng.range(-12, 70));
let mv = [rng.range(-64, 64) as i16, rng.range(-64, 64) as i16];
let mvc = chroma_mv(mv, (sx, sy));
let mut a = vec![0i16; w * h];
let mut b = vec![0i16; w * h];
let (t, frac, unit) = if c == 0 { (8usize, 3, 2) } else { (4, 7, 3) };
let (fx, fy) = ((mv[0] & frac) as usize, (mv[1] & frac) as usize);
let (x0, y0) = (x + (mv[0] as i32 >> unit) - (t as i32 / 2 - 1), y + (mv[1] as i32 >> unit) - (t as i32 / 2 - 1));
let mvu = if c == 0 { [mv[0] as i32, mv[1] as i32] } else { mvc };
let (fx, fy) = ((mvu[0] & frac) as usize, (mvu[1] & frac) as usize);
let (x0, y0) = (x + (mvu[0] >> unit) - (t as i32 / 2 - 1), y + (mvu[1] >> unit) - (t as i32 / 2 - 1));
let (shift1, shift3) = (bd - 8, 14 - bd);
let mut rows: Vec<&[S]> = vec![&[]; h + t - 1];
if f.window_rows(c, x0, y0, w + t - 1, &mut rows) {
@@ -580,17 +594,29 @@ mod tests {
let k = Interp { fx, fy, hf: &LUMA_FILTER[fx], vf: &LUMA_FILTER[fy], w, h, shift1, shift3 };
interpolate::<i16, 8>(strided(&win, WIN), &mut tmp, &k, &mut b);
} else {
mc_chroma::<S>(&f, c - 1, (x, y), mv, (w, h), &mut a, &mut sc);
mc_chroma::<S>(&f, c - 1, (x, y), mvc, (w, h), &mut a, &mut sc);
let k = Interp { fx, fy, hf: &CHROMA_FILTER[fx], vf: &CHROMA_FILTER[fy], w, h, shift1, shift3 };
interpolate::<i16, 4>(strided(&win, WIN), &mut tmp, &k, &mut b);
}
assert_eq!(a, b, "plane {c} bd {bd} at ({x}, {y}) mv {mv:?} {w}x{h}");
assert_eq!(a, b, "plane {c} {chroma:?} bd {bd} at ({x}, {y}) mv {mv:?} {w}x{h}");
}
assert!(in_place > 500 && copied > 100, "{in_place} in place, {copied} copied");
}
let mut rng = Rng(0x5851_f42d_4c95_7f2d);
check::<u8>(&mut rng, 8);
check::<u16>(&mut rng, 10);
for chroma in [ChromaFormat::C420, ChromaFormat::C422] {
check::<u8>(&mut rng, chroma, 8);
check::<u16>(&mut rng, chroma, 10);
}
}
/// The chroma vector of 8-228 / 8-229: the luma vector in 4:2:0 (1/8 chroma sample units),
/// doubled vertically in 4:2:2 (only even vertical fractions).
#[test]
fn chroma_vectors_follow_the_subsampling() {
assert_eq!(chroma_mv([5, -7], ChromaFormat::C420.shifts()), [5, -7]);
assert_eq!(chroma_mv([5, -7], ChromaFormat::C422.shifts()), [5, -14]);
assert_eq!(chroma_mv([i16::MIN, i16::MAX], ChromaFormat::C422.shifts()), [i16::MIN as i32, i16::MAX as i32 * 2]);
assert_eq!(chroma_mv([i16::MIN, i16::MAX], ChromaFormat::C420.shifts()), [i16::MIN as i32, i16::MAX as i32]);
}
/// The full-sample uni-prediction shortcut (`Frame::copy_block`) gives what interpolation and
@@ -598,9 +624,10 @@ mod tests {
/// ones as `u16`).
#[test]
fn full_sample_copy_matches_interpolation() {
fn check<S: Sample>(rng: &mut Rng, bd: u32) {
fn check<S: Sample>(rng: &mut Rng, chroma: ChromaFormat, bd: u32) {
let mut sc = McScratch::default();
let f = frame(rng, 48, 40, bd);
let f = frame(rng, 48, 40, chroma, bd);
let (sx, sy) = chroma.shifts();
for _ in 0..300 {
let (w, h) = ([4usize, 8, 12, 16, 24, 32][rng.range(0, 5) as usize], [4usize, 8, 16][rng.range(0, 2) as usize]);
let (x, y) = (rng.range(-40, 80), rng.range(-40, 70));
@@ -613,17 +640,19 @@ mod tests {
// (the window really was read: not all samples equal the default)
assert!(a.iter().any(|&v| v != S::default()) || w * h < 8);
for c in 1..3 {
let (cw, ch) = (w / 2, h / 2);
mc_chroma::<S>(&f, c - 1, (0, 0), [(x * 8) as i16, (y * 8) as i16], (cw, ch), &mut pred, &mut sc);
let (cw, ch) = (w >> sx, h >> sy);
mc_chroma::<S>(&f, c - 1, (0, 0), [x * 8, y * 8], (cw, ch), &mut pred, &mut sc);
let (mut a, mut b) = (vec![S::default(); cw * ch], vec![S::default(); cw * ch]);
put_uni(&pred, cw, ch, bd, &mut a, cw);
f.copy_block(c, (x, y), (cw, ch), &mut b, cw);
assert_eq!(a, b, "chroma {c} bd {bd} at ({x}, {y}) {cw}x{ch}");
assert_eq!(a, b, "chroma {c} {chroma:?} bd {bd} at ({x}, {y}) {cw}x{ch}");
}
}
}
let mut rng = Rng(0x0123_4567_89ab_cdef);
check::<u8>(&mut rng, 8);
check::<u16>(&mut rng, 10);
for chroma in [ChromaFormat::C420, ChromaFormat::C422] {
check::<u8>(&mut rng, chroma, 8);
check::<u16>(&mut rng, chroma, 10);
}
}
}
crates/video/src/hevc/decode/mod.rs
+44 -3
View File
@@ -1,7 +1,7 @@
//! Clean-room pure-Rust H.265 / HEVC decoder, implemented from ITU-T Rec. H.265 (ISO/IEC 23008-2).
//!
//! Supported: Main and Main 10 profiles (4:2:0, 8/10-bit; up to 12-bit 4:2:0 without range-extension
//! tools), all coding tools of those profiles: CABAC, coding / transform quadtrees, AMP, PCM,
//! Supported: Main, Main 10 and Main 4:2:2 10 profiles (4:2:0 and 4:2:2, 8/10-bit; up to 12-bit
//! without range-extension tools), all coding tools of those profiles: CABAC, coding / transform quadtrees, AMP, PCM,
//! transform skip, scaling lists, sign data hiding, 35 intra modes with strong intra smoothing,
//! merge / AMVP / TMVP, weighted prediction, deblocking, SAO, tiles, wavefront parallel processing
//! (parsing), dependent slice segments, short- and long-term reference picture sets.
@@ -52,6 +52,45 @@ pub struct ColorInfo {
pub matrix: u8,
}
/// Chroma sampling of a decoded picture (`chroma_format_idc`; 4:4:4 and monochrome are not
/// decoded).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum ChromaFormat {
/// 4:2:0: chroma planes are half the luma width and height.
C420,
/// 4:2:2: chroma planes are half the luma width and the full luma height.
C422,
}
impl ChromaFormat {
/// The decoded format for `chroma_format_idc`, if this decoder handles it.
pub fn from_idc(idc: u32) -> Option<ChromaFormat> {
match idc {
1 => Some(ChromaFormat::C420),
2 => Some(ChromaFormat::C422),
_ => None,
}
}
/// log2(SubWidthC), log2(SubHeightC): the shifts from luma to chroma sample positions.
#[inline(always)]
pub fn shifts(self) -> (u32, u32) {
match self {
ChromaFormat::C420 => (1, 1),
ChromaFormat::C422 => (1, 0),
}
}
/// SubWidthC.
#[inline(always)]
pub fn sub_width(self) -> usize {
1 << self.shifts().0
}
/// SubHeightC.
#[inline(always)]
pub fn sub_height(self) -> usize {
1 << self.shifts().1
}
}
/// Samples of one plane: 8-bit streams produce `U8`, higher bit depths `U16` (value range
/// `0..(1 << bit_depth)`).
#[derive(Clone, Debug, PartialEq, Eq)]
@@ -98,7 +137,7 @@ impl Plane {
}
}
/// A decoded, cropped picture in planar 4:2:0.
/// A decoded, cropped picture in planar 4:2:0 or 4:2:2 ([`Picture::chroma`]).
#[derive(Clone, Debug)]
pub struct Picture {
/// Cropped luma width.
@@ -107,6 +146,8 @@ pub struct Picture {
pub height: u32,
pub chroma_width: u32,
pub chroma_height: u32,
/// Chroma sampling of the `u` / `v` planes.
pub chroma: ChromaFormat,
/// Luma bit depth (chroma uses the same storage type).
pub bit_depth: u32,
pub y: Plane,
crates/video/src/hevc/decode/params.rs
+16 -3
View File
@@ -659,8 +659,14 @@ impl Sps {
/// Features this decoder does not handle yet.
pub fn check_supported(&self) -> Result<()> {
if self.chroma_format_idc != 1 {
return unsupported(format!("chroma_format_idc {} (only 4:2:0)", self.chroma_format_idc));
if self.chroma_array_type() != 1 && self.chroma_array_type() != 2 {
let what = match self.chroma_format_idc {
0 => "monochrome".to_string(),
3 if self.separate_colour_plane => "4:4:4 with separate colour planes".to_string(),
3 => "4:4:4".to_string(),
idc => format!("chroma_format_idc {idc}"),
};
return unsupported(format!("{what} (only 4:2:0 and 4:2:2)"));
}
if self.bit_depth_luma > 12 || self.bit_depth_chroma > 12 {
return unsupported(format!("bit depth {}/{}", self.bit_depth_luma, self.bit_depth_chroma));
@@ -801,7 +807,14 @@ impl Pps {
any |= r.read_ue()? != 0;
}
any |= r.read_flag()?; // cross_component_prediction
any |= r.read_flag()?; // chroma_qp_offset_list
let chroma_qp_offset_list = r.read_flag()?;
any |= chroma_qp_offset_list;
if !chroma_qp_offset_list {
// (with the list enabled the PPS is unsupported already and the list sits
// before these fields)
any |= r.read_ue()? != 0; // log2_sao_offset_scale_luma
any |= r.read_ue()? != 0; // log2_sao_offset_scale_chroma
}
range_extension = any;
}
}
crates/video/src/hevc/decode/picture.rs
+42 -27
View File
@@ -2,7 +2,7 @@
//! deblocking and SAO) together with the compressed motion field used for temporal motion vector
//! prediction.
use crate::hevc::decode::Plane;
use crate::hevc::decode::{ChromaFormat, Plane};
use std::sync::{Arc, OnceLock};
/// Motion data of one 16x16 block of a decoded picture (8.5.3.2.9 uses the motion of the top-left
@@ -128,6 +128,7 @@ pub struct Frame {
pub height: usize,
pub cwidth: usize,
pub cheight: usize,
pub chroma: ChromaFormat,
pub log2_ctb: u32,
pub bit_depth: u32,
pub bit_depth_c: u32,
@@ -148,13 +149,21 @@ fn wait<T>(cell: &OnceLock<T>) -> Option<&T> {
}
impl Frame {
pub fn new(id: u32, poc: i32, width: usize, height: usize, log2_ctb: u32, bit_depth: u32, bit_depth_c: u32) -> Self {
#[allow(clippy::too_many_arguments)]
pub fn new(id: u32, poc: i32, width: usize, height: usize, chroma: ChromaFormat, log2_ctb: u32, bit_depth: u32, bit_depth_c: u32) -> Self {
let n = height.div_ceil(1 << log2_ctb);
fn cells<S: Sample>(n: usize) -> Rows {
S::wrap_rows((0..n).map(|_| OnceLock::new()).collect())
}
let rows = if stores_u8(bit_depth, bit_depth_c) { cells::<u8>(n) } else { cells::<u16>(n) };
Frame { id, poc, width, height, cwidth: width / 2, cheight: height / 2, log2_ctb, bit_depth, bit_depth_c, rows }
let (sx, sy) = chroma.shifts();
Frame { id, poc, width, height, cwidth: width >> sx, cheight: height >> sy, chroma, log2_ctb, bit_depth, bit_depth_c, rows }
}
/// log2 of the chroma lines per CTB row (log2(CtbSizeY / SubHeightC)).
#[inline(always)]
pub fn log2_ctb_c(&self) -> u32 {
self.log2_ctb - self.chroma.shifts().1
}
pub fn num_rows(&self) -> usize {
@@ -239,7 +248,7 @@ impl Frame {
/// Copy a w x h window of chroma component `c` (0 = Cb, 1 = Cr) with edge replication.
pub fn chroma_window<S: Sample>(&self, c: usize, (x0, y0): (i32, i32), (w, h): (usize, usize), out: &mut [i16], os: usize) {
let (cw, ch) = (self.cwidth as i32, self.cheight as i32);
let sh = self.log2_ctb - 1;
let sh = self.log2_ctb_c();
let mask = (1i32 << sh) - 1;
for r in 0..h {
let yy = (y0 + r as i32).clamp(0, ch - 1);
@@ -255,7 +264,7 @@ impl Frame {
/// (8-228, 8-229): the reference window of a prediction block without copying it. False (and
/// `rows` partly filled) when the runs are not inside the picture horizontally.
pub fn window_rows<'f, S: Sample>(&'f self, c: usize, x0: i32, y0: i32, w: usize, rows: &mut [&'f [S]]) -> bool {
let (pw, ph, sh) = if c == 0 { (self.width, self.height, self.log2_ctb) } else { (self.cwidth, self.cheight, self.log2_ctb - 1) };
let (pw, ph, sh) = if c == 0 { (self.width, self.height, self.log2_ctb) } else { (self.cwidth, self.cheight, self.log2_ctb_c()) };
let Some(x) = usize::try_from(x0).ok().filter(|&x| x.saturating_add(w) <= pw) else { return false };
let mask = (1usize << sh) - 1;
let max_y = ph as i32 - 1;
@@ -285,7 +294,7 @@ impl Frame {
/// Copy the w x h block at (x0, y0) of plane `c` (0 = luma, 1 = Cb, 2 = Cr) into `dst`
/// (stride `ds`), samples outside the picture replicated from its edges (8-228, 8-229).
pub fn copy_block<S: Sample>(&self, c: usize, (x0, y0): (i32, i32), (w, h): (usize, usize), dst: &mut [S], ds: usize) {
let (pw, ph, sh) = if c == 0 { (self.width, self.height, self.log2_ctb) } else { (self.cwidth, self.cheight, self.log2_ctb - 1) };
let (pw, ph, sh) = if c == 0 { (self.width, self.height, self.log2_ctb) } else { (self.cwidth, self.cheight, self.log2_ctb_c()) };
let inside = usize::try_from(x0).ok().filter(|&x| x.saturating_add(w) <= pw);
for (r, d) in dst.chunks_mut(ds).take(h).enumerate() {
let yy = y0.saturating_add(r as i32).min(ph as i32 - 1).max(0) as usize;
@@ -347,8 +356,9 @@ impl Frame {
let Some(row) = self.row::<S>(r / ctb) else { break };
put(&mut y, &row.y, (r % ctb) * self.width + cx, cw);
}
let (ccx, ccy, ccw, cch) = (cx / 2, cy / 2, cw.div_ceil(2), ch.div_ceil(2));
let cctb = ctb / 2;
let (sw, sh) = (self.chroma.sub_width(), self.chroma.sub_height());
let (ccx, ccy, ccw, cch) = (cx / sw, cy / sh, cw.div_ceil(sw), ch.div_ceil(sh));
let cctb = ctb / sh;
u.reserve_exact(ccw.saturating_mul(cch));
v.reserve_exact(ccw.saturating_mul(cch));
for r in ccy..ccy.saturating_add(cch) {
@@ -371,7 +381,7 @@ impl Frame {
continue;
}
let lines = f.row_lines(r);
let clines = lines.div_ceil(2);
let clines = lines.div_ceil(f.chroma.sub_height());
f.publish(
r,
FrameRow::<S> {
@@ -391,7 +401,7 @@ impl Frame {
/// A mid-grey frame with the geometry of `like`.
pub fn gray(id: u32, poc: i32, like: &Frame) -> Frame {
let f = Frame::new(id, poc, like.width, like.height, like.log2_ctb, like.bit_depth, like.bit_depth_c);
let f = Frame::new(id, poc, like.width, like.height, like.chroma, like.log2_ctb, like.bit_depth, like.bit_depth_c);
f.publish_gray();
f
}
@@ -406,6 +416,7 @@ impl Frame {
height: self.height,
cwidth: self.cwidth,
cheight: self.cheight,
chroma: self.chroma,
log2_ctb: self.log2_ctb,
bit_depth: self.bit_depth,
bit_depth_c: self.bit_depth_c,
@@ -418,13 +429,14 @@ impl Frame {
pub fn from_planes(id: u32, poc: i32, like: &Frame, y: &[u16], cb: &[u16], cr: &[u16]) -> Frame {
fn fill<S: Sample>(f: &Frame, y: &[u16], cb: &[u16], cr: &[u16]) {
let ctb = 1usize << f.log2_ctb;
let sh = f.chroma.sub_height();
let w16 = f.width16();
let conv = |s: &[u16]| -> Box<[S]> { s.iter().map(|&v| S::from_u16(v)).collect() };
for r in 0..f.num_rows() {
let lines = f.row_lines(r);
let (y0, y1) = (r * ctb * f.width, (r * ctb + lines) * f.width);
let clines = lines.div_ceil(2);
let (c0, c1) = (r * ctb / 2 * f.cwidth, (r * ctb / 2 + clines) * f.cwidth);
let clines = lines.div_ceil(sh);
let (c0, c1) = (r * ctb / sh * f.cwidth, (r * ctb / sh + clines) * f.cwidth);
f.publish(
r,
FrameRow::<S> {
@@ -436,7 +448,7 @@ impl Frame {
);
}
}
let f = Frame::new(id, poc, like.width, like.height, like.log2_ctb, like.bit_depth, like.bit_depth_c);
let f = Frame::new(id, poc, like.width, like.height, like.chroma, like.log2_ctb, like.bit_depth, like.bit_depth_c);
match &f.rows {
Rows::U8(_) => fill::<u8>(&f, y, cb, cr),
Rows::U16(_) => fill::<u16>(&f, y, cb, cr),
@@ -540,7 +552,8 @@ pub type RefPic = crate::hevc::decode::dpb::Ref<FrameRef>;
mod tests {
use super::*;
fn planes(w: usize, h: usize, bd: u32, seed: u64) -> (Vec<u16>, Vec<u16>, Vec<u16>) {
fn planes(w: usize, h: usize, chroma: ChromaFormat, bd: u32, seed: u64) -> (Vec<u16>, Vec<u16>, Vec<u16>) {
let (cw, ch) = (w / chroma.sub_width(), h / chroma.sub_height());
let mut x = seed;
let mut next = move || {
x ^= x << 13;
@@ -549,33 +562,35 @@ mod tests {
(x % (1 << bd)) as u16
};
let y = (0..w * h).map(|_| next()).collect();
let cb = (0..w * h / 4).map(|_| next()).collect();
let cr = (0..w * h / 4).map(|_| next()).collect();
let cb = (0..cw * ch).map(|_| next()).collect();
let cr = (0..cw * ch).map(|_| next()).collect();
(y, cb, cr)
}
/// 8-bit pictures are stored as `u8`, deeper ones as `u16`, and both crop back to exactly the
/// samples they were given.
/// samples they were given, in 4:2:0 and 4:2:2.
#[test]
fn storage_type_follows_the_bit_depth_and_crops_exactly() {
let (w, h) = (40usize, 36usize);
let crop = (2usize, 4usize, 33usize, 27usize);
for bd in [8u32, 10] {
let (y, cb, cr) = planes(w, h, bd, 0x9e37_79b9_7f4a_7c15 + bd as u64);
let like = Frame::new(0, 0, w, h, 4, bd, bd);
for (bd, chroma) in [(8u32, ChromaFormat::C420), (10, ChromaFormat::C420), (8, ChromaFormat::C422), (10, ChromaFormat::C422)] {
let (sw, sh) = (chroma.sub_width(), chroma.sub_height());
let (y, cb, cr) = planes(w, h, chroma, bd, 0x9e37_79b9_7f4a_7c15 + bd as u64);
let like = Frame::new(0, 0, w, h, chroma, 4, bd, bd);
let f = Frame::from_planes(1, 0, &like, &y, &cb, &cr);
assert_eq!(matches!(f.rows, Rows::U8(_)), bd == 8);
assert!(f.is_complete());
let (py, pu, pv) = f.cropped_planes(crop, bd == 8);
let (cx, cy, cw, ch) = crop;
let want_y: Vec<u16> = (cy..cy + ch).flat_map(|r| y[r * w + cx..r * w + cx + cw].to_vec()).collect();
let (ccw, cch) = (cw.div_ceil(2), ch.div_ceil(2));
let (ccw, cch) = (cw.div_ceil(sw), ch.div_ceil(sh));
let pw = w / sw;
let want_c =
|p: &[u16]| -> Vec<u16> { (cy / 2..cy / 2 + cch).flat_map(|r| p[r * w / 2 + cx / 2..r * w / 2 + cx / 2 + ccw].to_vec()).collect() };
|p: &[u16]| -> Vec<u16> { (cy / sh..cy / sh + cch).flat_map(|r| p[r * pw + cx / sw..r * pw + cx / sw + ccw].to_vec()).collect() };
let got = |p: &Plane| -> Vec<u16> { (0..p.len()).map(|i| p.get(i)).collect() };
assert_eq!(got(&py), want_y, "luma bd {bd}");
assert_eq!(got(&pu), want_c(&cb), "cb bd {bd}");
assert_eq!(got(&pv), want_c(&cr), "cr bd {bd}");
assert_eq!(got(&py), want_y, "luma bd {bd} {chroma:?}");
assert_eq!(got(&pu), want_c(&cb), "cb bd {bd} {chroma:?}");
assert_eq!(got(&pv), want_c(&cr), "cr bd {bd} {chroma:?}");
assert_eq!(py.as_u8().is_some(), bd == 8);
}
}
@@ -602,7 +617,7 @@ mod tests {
/// nothing instead of a panic, and a grey frame of either type is complete.
#[test]
fn wrong_sample_type_is_harmless() {
let f = Frame::new(0, 0, 32, 32, 4, 8, 8);
let f = Frame::new(0, 0, 32, 32, ChromaFormat::C420, 4, 8, 8);
f.publish(0, FrameRow::<u16> { y: vec![1; 16 * 32].into(), cb: vec![1; 8 * 16].into(), cr: vec![1; 8 * 16].into(), col: Box::new([]) });
assert!(!f.is_published(0));
f.publish_gray();
@@ -615,7 +630,7 @@ mod tests {
let mut dst = [5u16; 16];
f.copy_block::<u16>(0, (0, 0), (4, 4), &mut dst, 4);
assert!(dst.iter().all(|&v| v == 5));
let g = Frame::gray(1, 0, &Frame::new(0, 0, 16, 8, 4, 10, 10));
let g = Frame::gray(1, 0, &Frame::new(0, 0, 16, 8, ChromaFormat::C420, 4, 10, 10));
assert!(g.is_complete());
assert_eq!(g.row::<u16>(0).map(|r| r.cb[0]), Some(512));
assert_eq!(g.col_mv(15, 7), ColMv::default());
crates/video/src/hevc/decode/slicedec.rs
+6 -2
View File
@@ -3,6 +3,7 @@
//! compensation, scaling and inverse transforms. In-loop filters and row publication live in
//! `filter.rs`.
use crate::hevc::decode::ChromaFormat;
use crate::hevc::decode::cabac::{Cabac, Contexts, init_contexts};
use crate::hevc::decode::error::{Result, ensure, invalid};
use crate::hevc::decode::inter;
@@ -15,7 +16,7 @@ use crate::hevc::decode::spec_tables::{
INTRA_CHROMA, LAST_X_PREFIX, LAST_Y_PREFIX, MERGE_FLAG, MERGE_IDX, MVP_FLAG, PART_MODE, PRED_MODE, PREV_INTRA_LUMA, REF_IDX, RQT_ROOT_CBF,
SAO_MERGE, SAO_TYPE, SIG_COEFF, SPLIT_CU, SPLIT_TRANSFORM, TRANSFORM_SKIP,
};
use crate::hevc::decode::tables::{CTX_IDX_MAP, LEVEL_SCALE, SCAN_4, qpc_420, scan_order};
use crate::hevc::decode::tables::{CTX_IDX_MAP, INTRA_CHROMA_MODE_422, LEVEL_SCALE, SCAN_4, qpc, scan_order};
use crate::hevc::decode::transform;
use std::sync::Arc;
@@ -114,6 +115,7 @@ pub struct PicState<S> {
pub height: usize,
pub cwidth: usize,
pub cheight: usize,
pub chroma: ChromaFormat,
pub w4: usize,
pub h4: usize,
pub log2_ctb: u32,
@@ -147,7 +149,8 @@ pub struct PicState<S> {
impl<S: Sample> PicState<S> {
pub fn new(frame: FrameRef, sps: Arc<Sps>, pps: Arc<Pps>, layout: Arc<Layout>, scaling: Option<Arc<Vec<Vec<u8>>>>) -> Self {
let (width, height) = (sps.width as usize, sps.height as usize);
let (cwidth, cheight) = (width / 2, height / 2);
let chroma = frame.chroma;
let (cwidth, cheight) = (frame.cwidth, frame.cheight);
let (w4, h4) = (width.div_ceil(4), height.div_ceil(4));
let log2_ctb = sps.log2_ctb;
let (wctb, hctb) = (sps.pic_width_in_ctbs() as usize, sps.pic_height_in_ctbs() as usize);
@@ -164,6 +167,7 @@ impl<S: Sample> PicState<S> {
height,
cwidth,
cheight,
chroma,
w4,
h4,
log2_ctb,
crates/video/src/hevc/decode/slicedec/coding_unit.rs
+20 -12
View File
@@ -1,7 +1,7 @@
use super::transform_tree::TtNode;
use super::{
Blk, CU_SKIP, CU_TRANSQUANT_BYPASS, E_PU_H, E_PU_V, E_TU_H, E_TU_V, F_CODED, F_INTRA, F_NOFILTER, F_SKIP, INTRA_CHROMA, PART_MODE, PRED_MODE,
PREV_INTRA_LUMA, PartMode, RQT_ROOT_CBF, Result, SPLIT_CU, Sample, SliceDecoder, ensure, qpc_420,
Blk, CU_SKIP, CU_TRANSQUANT_BYPASS, ChromaFormat, E_PU_H, E_PU_V, E_TU_H, E_TU_V, F_CODED, F_INTRA, F_NOFILTER, F_SKIP, INTRA_CHROMA,
INTRA_CHROMA_MODE_422, PART_MODE, PRED_MODE, PREV_INTRA_LUMA, PartMode, RQT_ROOT_CBF, Result, SPLIT_CU, Sample, SliceDecoder, ensure, qpc,
};
use crate::hevc::decode::mvpred::PredBlock;
@@ -81,8 +81,9 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
let offc = 6 * (self.pic.bd_c as i32 - 8);
let qpi_cb = (self.qp_y + self.pps.cb_qp_offset + self.sh.cb_qp_offset).clamp(-offc, 57);
let qpi_cr = (self.qp_y + self.pps.cr_qp_offset + self.sh.cr_qp_offset).clamp(-offc, 57);
self.qp_cb = qpc_420(qpi_cb) + offc;
self.qp_cr = qpc_420(qpi_cr) + offc;
let is420 = self.pic.chroma == ChromaFormat::C420;
self.qp_cb = qpc(qpi_cb, is420) + offc;
self.qp_cr = qpc(qpi_cr, is420) + offc;
}
fn coding_unit(&mut self, x0: i32, y0: i32, log2: u32, depth: u32) -> Result<()> {
@@ -147,7 +148,7 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
}
self.parse_intra_modes(x0, y0, n, part)?;
self.max_trafo_depth = self.sps.max_th_depth_intra + self.intra_split as u32;
self.transform_tree(TtNode { x0, y0, xb: x0, yb: y0, log2, blk_idx: 0 }, 0, [true, true])?;
self.transform_tree(TtNode { x0, y0, xb: x0, yb: y0, log2, blk_idx: 0 }, 0, [[true; 2]; 2])?;
} else {
self.intra_split = false;
let mut merge2nx2n = false;
@@ -179,7 +180,7 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
let root_cbf = if !merge2nx2n { self.dd(RQT_ROOT_CBF) == 1 } else { true };
if root_cbf {
self.max_trafo_depth = self.sps.max_th_depth_inter;
self.transform_tree(TtNode { x0, y0, xb: x0, yb: y0, log2, blk_idx: 0 }, 0, [true, true])?;
self.transform_tree(TtNode { x0, y0, xb: x0, yb: y0, log2, blk_idx: 0 }, 0, [[true; 2]; 2])?;
}
}
self.finish_cu(x0, y0, n);
@@ -251,8 +252,10 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
let start = self.c.bit_pos().div_ceil(8);
let data = self.c.data();
let n = 1usize << log2;
let (csx, csy) = self.pic.chroma.shifts();
let (ncw, nch) = (n >> csx, n >> csy);
let (bl, bc) = (self.sps.pcm_bit_depth_luma, self.sps.pcm_bit_depth_chroma);
let total_bits = n * n * bl as usize + 2 * (n / 2) * (n / 2) * bc as usize;
let total_bits = n * n * bl as usize + 2 * ncw * nch * bc as usize;
ensure!(start + total_bits.div_ceil(8) <= data.len(), "PCM samples truncated");
let mut r = craft_media::bitstream::BitReader::new(&data[start..]);
let (sy, sc) = (self.pic.bd_y - bl, self.pic.bd_c - bc);
@@ -265,10 +268,10 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
}
let cw = self.pic.cwidth;
for c in 1..3 {
for y in 0..n / 2 {
for x in 0..n / 2 {
for y in 0..nch {
for x in 0..ncw {
let v = r.read_bits(bc)? as u16;
self.pic.planes[c][(y0 as usize / 2 + y) * cw + x0 as usize / 2 + x] = S::from_u16(v << sc);
self.pic.planes[c][((y0 as usize >> csy) + y) * cw + (x0 as usize >> csx) + x] = S::from_u16(v << sc);
}
}
}
@@ -336,16 +339,21 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
self.fill_blk(xp, yp, pb, pb, |b| b.ipm = mode as u8);
}
}
// chroma mode (4:2:0: one per CU, derived from the first PU's luma mode)
// chroma mode (one per CU in 4:2:0 and 4:2:2, derived from the first PU's luma mode:
// Table 8-2, then the 4:2:2 mapping of Table 8-3)
let icpm = if self.dd(INTRA_CHROMA) == 0 { 4 } else { self.bypass_bits(2) };
let luma = self.pic.blk_at(x0, y0).ipm as u32;
self.intra_chroma_mode = match icpm {
let mode = match icpm {
4 => luma,
_ => {
let m = [0, 26, 10, 1][icpm as usize];
if m == luma { 34 } else { m }
}
};
self.intra_chroma_mode = match self.pic.chroma {
ChromaFormat::C420 => mode,
ChromaFormat::C422 => INTRA_CHROMA_MODE_422.get(mode as usize).copied().unwrap_or(mode as u8) as u32,
};
Ok(())
}
}
crates/video/src/hevc/decode/slicedec/prediction_unit.rs
+7 -6
View File
@@ -129,8 +129,9 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
};
let bi = f.pred(0) && f.pred(1);
let pwt = self.sh.pwt.as_ref().filter(|_| weighted);
let (sx, sy) = self.pic.chroma.shifts();
for c in 0..3 {
let (x, y, bw, bh, bd) = if c == 0 { (xp, yp, w, h, self.pic.bd_y) } else { (xp / 2, yp / 2, w / 2, h / 2, self.pic.bd_c) };
let (x, y, bw, bh, bd) = if c == 0 { (xp, yp, w, h, self.pic.bd_y) } else { (xp >> sx, yp >> sy, w >> sx, h >> sy, self.pic.bd_c) };
// Explicit weights equal to the default ones (weight 2^denom and offset 0 for every
// list used, as signalled for most slices of a weighted-prediction stream) give exactly
// the default prediction: with log2WD = denom + shift1 the weighted sums of 8-265 /
@@ -140,14 +141,14 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
// ((s << shift3) + off) >> shift3 (8-228, 8-262) is the reference sample itself.
if !bi && pwt.is_none() {
let l = if f.pred(0) { 0 } else { 1 };
let frac = if c == 0 { 3 } else { 7 };
let mv = f.mv[l];
// the vector in this component's units: quarter luma samples, 1/8 chroma samples
let (mv, frac, sh) =
if c == 0 { ([f.mv[l][0] as i32, f.mv[l][1] as i32], 3, 2) } else { (inter::chroma_mv(f.mv[l], (sx, sy)), 7, 3) };
if f.pred(l) && mv[0] & frac == 0 && mv[1] & frac == 0 {
let sh = if c == 0 { 2 } else { 3 };
let stride = if c == 0 { self.pic.width } else { self.pic.cwidth };
let frame = &self.refs[l][f.ref_idx[l] as usize].frame;
let dst = &mut self.pic.planes[c][y as usize * stride + x as usize..];
frame.copy_block(c, (x + (mv[0] as i32 >> sh), y + (mv[1] as i32 >> sh)), (bw, bh), dst, stride);
frame.copy_block(c, (x + (mv[0] >> sh), y + (mv[1] >> sh)), (bw, bh), dst, stride);
continue;
}
}
@@ -162,7 +163,7 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
if c == 0 {
inter::mc_luma::<S>(frame, (x, y), f.mv[l], (bw, bh), buf, &mut self.mc);
} else {
inter::mc_chroma::<S>(frame, c - 1, (x, y), f.mv[l], (bw, bh), buf, &mut self.mc);
inter::mc_chroma::<S>(frame, c - 1, (x, y), inter::chroma_mv(f.mv[l], (sx, sy)), (bw, bh), buf, &mut self.mc);
}
}
let stride = if c == 0 { self.pic.width } else { self.pic.cwidth };
crates/video/src/hevc/decode/slicedec/transform_tree.rs
+53 -29
View File
@@ -1,7 +1,7 @@
use super::{
CBF_CHROMA, CBF_LUMA, CODED_SUB_BLOCK, CTX_IDX_MAP, CU_QP_DELTA_ABS, E_TU_H, E_TU_V, F_CBF, F_INTRA, GT1, GT2, LAST_X_PREFIX, LAST_Y_PREFIX,
LEVEL_SCALE, PartMode, Result, SCAN_4, SIG_COEFF, SPLIT_TRANSFORM, Sample, SliceDecoder, TRANSFORM_SKIP, ensure, for_runs, intra, scan_order,
transform,
CBF_CHROMA, CBF_LUMA, CODED_SUB_BLOCK, CTX_IDX_MAP, CU_QP_DELTA_ABS, ChromaFormat, E_TU_H, E_TU_V, F_CBF, F_INTRA, GT1, GT2, LAST_X_PREFIX,
LAST_Y_PREFIX, LEVEL_SCALE, PartMode, Result, SCAN_4, SIG_COEFF, SPLIT_TRANSFORM, Sample, SliceDecoder, TRANSFORM_SKIP, ensure, for_runs, intra,
scan_order, transform,
};
/// Significant scan positions of one sub-block, in decoding (descending) order.
@@ -107,7 +107,9 @@ pub(super) struct TtNode {
}
impl<'a, S: Sample> SliceDecoder<'a, S> {
pub(super) fn transform_tree(&mut self, node: TtNode, depth: u32, parent_cbf_c: [bool; 2]) -> Result<()> {
/// `parent_cbf_c[c][t]`: cbf_cb / cbf_cr (c) of the parent node, for the upper (t = 0) and, in
/// 4:2:2, the lower (t = 1) chroma block (7.3.8.8).
pub(super) fn transform_tree(&mut self, node: TtNode, depth: u32, parent_cbf_c: [[bool; 2]; 2]) -> Result<()> {
let TtNode { x0, y0, log2, .. } = node;
let split =
if log2 <= self.sps.log2_max_tb && log2 > self.sps.log2_min_tb && depth < self.max_trafo_depth && !(self.intra_split && depth == 0) {
@@ -117,18 +119,25 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
let inter_split = self.sps.max_th_depth_inter == 0 && !self.cu_intra && self.cu_part != PartMode::P2Nx2N && depth == 0;
log2 > self.sps.log2_max_tb || (self.intra_split && depth == 0) || inter_split
};
let mut cbf_c = [false, false];
let mut cbf_c = [[false; 2]; 2];
if log2 > 2 {
// the second cbf of each chroma component (the lower block of the 4:2:2 chroma
// rectangle) is coded for a leaf, and at 8x8 (whose chroma is coded with the 4x4 luma
// blocks) also when split
let second = self.pic.chroma == ChromaFormat::C422 && (!split || log2 == 3);
for (c, cbf) in cbf_c.iter_mut().enumerate() {
if depth == 0 || parent_cbf_c[c] {
*cbf = self.dd(CBF_CHROMA + depth as usize) == 1;
if depth == 0 || parent_cbf_c[c][0] {
cbf[0] = self.dd(CBF_CHROMA + depth as usize) == 1;
if second {
cbf[1] = self.dd(CBF_CHROMA + depth as usize) == 1;
}
}
}
} else {
// 4x4 luma blocks: chroma cbfs are those of the parent (coded at blkIdx 3)
cbf_c = parent_cbf_c;
if depth == 0 {
cbf_c = [false, false];
cbf_c = [[false; 2]; 2];
}
}
if split {
@@ -139,8 +148,8 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
}
return Ok(());
}
let cbf_luma =
if self.cu_intra || depth != 0 || cbf_c[0] || cbf_c[1] { self.dd(CBF_LUMA + if depth == 0 { 1 } else { 0 }) == 1 } else { true };
let any_cbf_c = cbf_c.iter().flatten().any(|&b| b);
let cbf_luma = if self.cu_intra || depth != 0 || any_cbf_c { self.dd(CBF_LUMA + if depth == 0 { 1 } else { 0 }) == 1 } else { true };
let n = 1 << log2;
if depth > 0 {
self.mark_edges(x0, y0, n, n, E_TU_V, E_TU_H);
@@ -148,11 +157,11 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
self.transform_unit(node, cbf_luma, cbf_c)
}
fn transform_unit(&mut self, node: TtNode, cbf_luma: bool, cbf_c: [bool; 2]) -> Result<()> {
fn transform_unit(&mut self, node: TtNode, cbf_luma: bool, cbf_c: [[bool; 2]; 2]) -> Result<()> {
let TtNode { x0, y0, xb, yb, log2, blk_idx } = node;
let n = 1i32 << log2;
let chroma_here = log2 > 2 || blk_idx == 3;
let cbf_chroma = cbf_c[0] || cbf_c[1];
let cbf_chroma = cbf_c.iter().flatten().any(|&b| b);
if (cbf_luma || cbf_chroma) && self.pps.cu_qp_delta_enabled && !self.is_cu_qp_delta_coded {
// cu_qp_delta_abs: prefix TU(5) with ctx 0,1,1,1,1, suffix EG0
let mut v = 0;
@@ -191,15 +200,24 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
if !chroma_here {
return Ok(());
}
let (xc, yc, log2c) = if log2 > 2 { (x0 / 2, y0 / 2, log2 - 1) } else { (xb / 2, yb / 2, 2) };
// Chroma blocks of this transform unit (7.3.8.10): square blocks of (luma size) /
// SubWidthC at the chroma position of the unit (of the 8x8 parent for 4x4 luma blocks);
// in 4:2:2 the chroma rectangle is twice as tall and coded as two stacked square blocks,
// the lower one predicted from the reconstructed upper one (8.4.4.1).
let (sx, sy) = self.pic.chroma.shifts();
let (xc, yc, log2c) = if log2 > 2 { (x0 >> sx, y0 >> sy, log2 - 1) } else { (xb >> sx, yb >> sy, 2) };
let blocks = if self.pic.chroma == ChromaFormat::C422 { 2 } else { 1 };
for c in 1..3 {
if self.cu_intra {
let mode = self.intra_chroma_mode;
self.intra_predict(xc, yc, log2c, c, mode);
}
if cbf_c[c - 1] {
let qp = if c == 1 { self.qp_cb } else { self.qp_cr };
self.residual(xc, yc, log2c, c, qp)?;
for t in 0..blocks {
let yt = yc + ((t as i32) << log2c);
if self.cu_intra {
let mode = self.intra_chroma_mode;
self.intra_predict(xc, yt, log2c, c, mode);
}
if cbf_c[c - 1][t] {
let qp = if c == 1 { self.qp_cb } else { self.qp_cr };
self.residual(xc, yt, log2c, c, qp)?;
}
}
}
Ok(())
@@ -208,25 +226,31 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
/// Intra prediction of an n x n block of component `c` at component position (x0, y0).
fn intra_predict(&mut self, x0: i32, y0: i32, log2: u32, c: usize, mode: u32) {
let n = 1usize << log2;
let sc = if c == 0 { 1 } else { 2 };
let (xl, yl) = (x0 * sc, y0 * sc);
let unit = 4 / sc as usize; // component samples per 4x4 luma block
// shifts from component to luma positions (8.4.4.2.1: the availability of neighbouring
// samples is derived at the luma positions (xNbCmp * SubWidthC, yNbCmp * SubHeightC))
let (sx, sy) = if c == 0 { (0, 0) } else { self.pic.chroma.shifts() };
let (xl, yl) = (x0 << sx, y0 << sy);
// component samples per 4x4 luma block, down the left column and along the top row
// (4:2:2 chroma: 4 rows but 2 columns)
let (unit_y, unit_x) = (4usize >> sy, 4usize >> sx);
let stride = if c == 0 { self.pic.width } else { self.pic.cwidth };
let constrained = self.pps.constrained_intra_pred;
let mut al = [false; 129];
let mut at = [false; 129];
let addr = self.slice_addr;
let pic = &*self.pic;
let units = (2 * n / unit) as u32;
let mut ml = pic.zavail_run((xl, yl), (xl - 1, yl), true, units, addr);
let mut mt = pic.zavail_run((xl, yl), (xl, yl - 1), false, units, addr);
let (units_l, units_t) = ((2 * n / unit_y) as u32, (2 * n / unit_x) as u32);
let mut ml = pic.zavail_run((xl, yl), (xl - 1, yl), true, units_l, addr);
let mut mt = pic.zavail_run((xl, yl), (xl, yl - 1), false, units_t, addr);
let mut corner = pic.zavail(xl, yl, xl - 1, yl - 1, addr);
if constrained {
let intra = |x: i32, y: i32| pic.blk_at(x, y).flags & F_INTRA != 0;
for k in 0..units {
for k in 0..units_l {
if ml & (1 << k) != 0 && !intra(xl - 1, yl + 4 * k as i32) {
ml &= !(1 << k);
}
}
for k in 0..units_t {
if mt & (1 << k) != 0 && !intra(xl + 4 * k as i32, yl - 1) {
mt &= !(1 << k);
}
@@ -242,14 +266,14 @@ impl<'a, S: Sample> SliceDecoder<'a, S> {
r.left[0] = v;
r.top[0] = v;
}
for_runs(ml, unit, |a, b| {
for_runs(ml, unit_y, |a, b| {
let left = (y0 as usize + a) * stride + x0 as usize - 1;
al[1 + a..1 + b].fill(true);
for (k, v) in r.left[1 + a..1 + b].iter_mut().enumerate() {
*v = plane[left + k * stride].to_u16();
}
});
for_runs(mt, unit, |a, b| {
for_runs(mt, unit_x, |a, b| {
let top = (y0 as usize - 1) * stride + x0 as usize;
at[1 + a..1 + b].fill(true);
for (v, s) in r.top[1 + a..1 + b].iter_mut().zip(&plane[top + a..top + b]) {
crates/video/src/hevc/decode/synth_tests.rs
+53
View File
@@ -751,3 +751,56 @@ fn hvcc_rejects_a_valid_picture_slice_without_starting_a_picture() {
assert!(decoder.flush().is_empty());
assert_eq!(decoder.stats().pictures, 0);
}
/// The RBSP of a minimal PPS (no transform skip) with a pps_range_extension() whose only
/// non-default fields are the SAO offset scales.
fn pps_rbsp_with_sao_offset_scale(luma: u32, chroma: u32) -> Vec<u8> {
let mut w = BitWriter::new();
w.write_ue(0); // pps_pic_parameter_set_id
w.write_ue(0); // pps_seq_parameter_set_id
w.write_bits(0, 1); // dependent slice segments
w.write_bits(0, 1); // output flag present
w.write_bits(0, 3); // num_extra_slice_header_bits
w.write_bits(0, 1); // sign data hiding
w.write_bits(0, 1); // cabac_init_present
w.write_ue(0);
w.write_ue(0);
w.write_se(0); // init_qp 26
w.write_bits(0, 1); // constrained intra
w.write_bits(0, 1); // transform skip
w.write_bits(0, 1); // cu_qp_delta
w.write_se(0);
w.write_se(0);
w.write_bits(0, 1); // slice chroma qp offsets present
w.write_bits(0, 1); // weighted pred
w.write_bits(0, 1); // weighted bipred
w.write_bits(0, 1); // transquant bypass
w.write_bits(0, 1); // tiles
w.write_bits(0, 1); // entropy coding sync
w.write_bits(1, 1); // loop filter across slices
w.write_bits(0, 1); // deblocking control
w.write_bits(0, 1); // scaling list
w.write_bits(0, 1); // lists modification
w.write_ue(0); // log2_parallel_merge_level - 2
w.write_bits(0, 1); // slice header extension
w.write_bits(1, 1); // pps_extension_present
w.write_bits(0x80, 8); // pps_range_extension_flag only
w.write_bits(0, 1); // cross_component_prediction
w.write_bits(0, 1); // chroma_qp_offset_list
w.write_ue(luma); // log2_sao_offset_scale_luma
w.write_ue(chroma); // log2_sao_offset_scale_chroma
w.rbsp_trailing();
w.finish()
}
/// SAO offsets are applied unscaled, so a PPS scaling them (a range-extension tool) must be
/// refused rather than decoded to wrong samples; a range extension that enables nothing is fine.
#[test]
fn a_pps_scaling_sao_offsets_is_unsupported() {
let plain = crate::hevc::decode::params::Pps::parse(&pps_rbsp_with_sao_offset_scale(0, 0)).expect("parses");
assert!(plain.check_supported().is_ok());
for (luma, chroma) in [(1, 0), (0, 2)] {
let pps = crate::hevc::decode::params::Pps::parse(&pps_rbsp_with_sao_offset_scale(luma, chroma)).expect("parses");
assert!(pps.check_supported().is_err(), "luma {luma}, chroma {chroma}");
}
}
crates/video/src/hevc/decode/tables.rs
+49
View File
@@ -68,6 +68,18 @@ pub fn qpc_420(qpi: i32) -> i32 {
}
}
/// Table 8-3: IntraPredModeC for modeIdx (Table 8-2) when ChromaArrayType is 2 (the 4:2:2
/// chroma intra prediction mode: the luma direction with its horizontal component halved).
pub static INTRA_CHROMA_MODE_422: [u8; 35] =
[0, 1, 2, 2, 2, 2, 3, 5, 7, 8, 10, 12, 13, 15, 17, 18, 19, 20, 21, 22, 23, 23, 24, 24, 25, 25, 26, 27, 27, 28, 28, 29, 29, 30, 31];
/// QpC for the index qPi (8.6.1, 8.7.2.5.5): Table 8-10 when ChromaArrayType is 1, otherwise
/// Min(qPi, 51).
#[inline]
pub fn qpc(qpi: i32, chroma_array_type_1: bool) -> i32 {
if chroma_array_type_1 { qpc_420(qpi) } else { qpi.min(51) }
}
/// Table 9-50 ctxIdxMap for 4x4 sig_coeff_flag contexts.
pub static CTX_IDX_MAP: [u8; 16] = [0, 1, 4, 5, 2, 3, 4, 5, 6, 6, 8, 8, 7, 7, 8, 8];
@@ -81,6 +93,40 @@ pub static DST_MATRIX: [[i8; 4]; 4] = [[29, 55, 74, 84], [74, 74, 0, -74], [84,
mod tests {
use super::*;
/// Table 8-3 (Rec. ITU-T H.265 v8 (08/2021), 8.4.3) is the 4:2:2 form of each luma direction:
/// chroma has half the horizontal resolution, so the direction of mode m (Table 8-5's
/// intraPredAngle, horizontal modes 2..=17 and vertical 18..=34) with its horizontal component
/// halved, mapped to the angular mode nearest in angle. Planar and DC stay. This rule
/// reproduces every one of the 35 entries, so a mistyped entry fails here without an oracle.
#[test]
fn the_422_chroma_mode_table_is_the_luma_direction_with_half_the_width() {
let dir = |m: usize| -> (f64, f64) {
let a = f64::from(INTRA_PRED_ANGLE[m]);
if m < 18 { (-32.0, a) } else { (a, -32.0) }
};
let angle = |(x, y): (f64, f64)| y.atan2(x);
let gap = |a: f64, b: f64| {
let d = (a - b).rem_euclid(std::f64::consts::TAU);
d.min(std::f64::consts::TAU - d)
};
for m in 0..35usize {
let want = if m < 2 {
m
} else {
let (x, y) = dir(m);
let t = angle((x / 2.0, y));
let mut best = 2;
for k in 2..35 {
if gap(angle(dir(k)), t) < gap(angle(dir(best)), t) - 1e-12 {
best = k;
}
}
best
};
assert_eq!(INTRA_CHROMA_MODE_422[m] as usize, want, "luma mode {m}");
}
}
#[test]
fn diagonal_scan_4x4() {
let s = &SCAN_4[0];
@@ -119,5 +165,8 @@ mod tests {
assert_eq!(qpc_420(30), 29);
assert_eq!(qpc_420(43), 37);
assert_eq!(qpc_420(51), 45);
assert_eq!(qpc(43, true), 37);
assert_eq!(qpc(43, false), 43);
assert_eq!(qpc(57, false), 51);
}
}
crates/video/tests/hevc_decode/common/mod.rs
+76 -18
View File
@@ -2,7 +2,7 @@
//! comparison helpers.
#![allow(dead_code)]
use craft_video::hevc::decode::{Decoder, Picture};
use craft_video::hevc::decode::{ChromaFormat, Decoder, Picture};
use std::path::{Path, PathBuf};
use std::process::Command;
@@ -18,6 +18,8 @@ pub struct Fixture {
pub filter: &'static str,
/// 10 (yuv420p10le) or 8.
pub bit_depth: u32,
/// 4:2:0 or 4:2:2.
pub chroma: ChromaFormat,
/// Encoder arguments (libx265 unless `-c:v` is given).
pub args: &'static [&'static str],
}
@@ -26,7 +28,10 @@ const NOISE: &str = "noise=alls=12:allf=t+u";
macro_rules! fx {
($name:expr, $src:expr, $w:expr, $h:expr, $n:expr, $filter:expr, $bd:expr, [$($a:expr),* $(,)?]) => {
Fixture { name: $name, source: $src, width: $w, height: $h, frames: $n, filter: $filter, bit_depth: $bd, args: &[$($a),*] }
Fixture { name: $name, source: $src, width: $w, height: $h, frames: $n, filter: $filter, bit_depth: $bd, chroma: ChromaFormat::C420, args: &[$($a),*] }
};
($name:expr, $src:expr, $w:expr, $h:expr, $n:expr, $filter:expr, $bd:expr, 422, [$($a:expr),* $(,)?]) => {
Fixture { name: $name, source: $src, width: $w, height: $h, frames: $n, filter: $filter, bit_depth: $bd, chroma: ChromaFormat::C422, args: &[$($a),*] }
};
}
@@ -73,6 +78,39 @@ pub const FIXTURES: &[Fixture] = &[
fx!("odd_1918x1078", "testsrc2", 1918, 1078, 3, NOISE, 8, ["-preset", "veryfast", "-x265-params", "log-level=error"]),
fx!("hd_1080p", "testsrc2", 1920, 1080, 3, "", 8, ["-preset", "fast", "-x265-params", "log-level=error"]),
fx!("uhd_4k_main10", "testsrc2", 3840, 2160, 2, "", 10, ["-preset", "ultrafast", "-x265-params", "log-level=error"]),
// 4:2:2 (Main 4:2:2 10 / RExt profile without range-extension tools: libx265 with yuv422p / yuv422p10le)
fx!("intra_422", "testsrc2", 352, 288, 3, NOISE, 8, 422, ["-x265-params", "keyint=1:log-level=error"]),
fx!("intra_422_10", "testsrc2", 352, 288, 3, NOISE, 10, 422, ["-x265-params", "keyint=1:log-level=error"]),
fx!("intra_422_10_nosao_nodbk", "testsrc2", 352, 288, 2, NOISE, 10, 422, ["-x265-params", "keyint=1:no-sao=1:no-deblock=1:log-level=error"]),
fx!("medium_422_10", "testsrc2", 352, 288, 20, NOISE, 10, 422, ["-preset", "medium", "-x265-params", "log-level=error"]),
fx!("slow_422", "mandelbrot", 352, 288, 12, NOISE, 8, 422, ["-preset", "slow", "-x265-params", "log-level=error"]),
fx!("p_only_422_10", "testsrc2", 352, 288, 20, NOISE, 10, 422, ["-x265-params", "bframes=0:log-level=error"]),
fx!("bframes_422_10", "testsrc2", 352, 288, 30, NOISE, 10, 422, ["-x265-params", "bframes=8:b-pyramid=1:ref=4:log-level=error"]),
fx!("no_sao_422_10", "testsrc2", 352, 288, 12, NOISE, 10, 422, ["-x265-params", "no-sao=1:log-level=error"]),
fx!("deblock_m3_3_422_10", "testsrc2", 352, 288, 12, NOISE, 10, 422, ["-x265-params", "deblock=-3,3:log-level=error"]),
fx!("rect_amp_422_10", "mandelbrot", 352, 288, 12, NOISE, 10, 422, ["-x265-params", "rect=1:amp=1:log-level=error"]),
fx!("tskip_422_10", "testsrc2", 352, 288, 12, NOISE, 10, 422, ["-x265-params", "tskip=1:log-level=error"]),
fx!("weightp_422_10", "testsrc2", 352, 288, 20, "fade=in:0:18", 10, 422, ["-x265-params", "weightp=1:weightb=1:log-level=error"]),
fx!("qp_422_10", "mandelbrot", 352, 288, 8, NOISE, 10, 422, ["-x265-params", "aq-mode=3:cutree=1:log-level=error"]),
fx!("crf51_422_10", "testsrc2", 352, 288, 8, NOISE, 10, 422, ["-crf", "51", "-x265-params", "log-level=error"]),
fx!("lossless_422_10", "testsrc2", 176, 144, 4, NOISE, 10, 422, ["-x265-params", "lossless=1:log-level=error"]),
fx!(
"ctu32_tu4_422_10",
"testsrc2",
352,
288,
12,
NOISE,
10,
422,
["-x265-params", "ctu=32:tu-intra-depth=4:tu-inter-depth=4:max-tu-size=16:log-level=error"]
),
fx!("ctu16_422_10", "testsrc2", 352, 288, 8, NOISE, 10, 422, ["-x265-params", "ctu=16:log-level=error"]),
fx!("scaling_list_422_10", "testsrc2", 352, 288, 8, NOISE, 10, 422, ["-x265-params", "scaling-list=default:log-level=error"]),
fx!("constrained_intra_422_10", "testsrc2", 352, 288, 8, NOISE, 10, 422, ["-x265-params", "constrained-intra=1:log-level=error"]),
fx!("slices4_422_10", "testsrc2", 352, 288, 8, NOISE, 10, 422, ["-x265-params", "slices=4:log-level=error"]),
fx!("odd_1918x1078_422_10", "testsrc2", 1918, 1078, 3, NOISE, 10, 422, ["-preset", "veryfast", "-x265-params", "log-level=error"]),
fx!("uhd_4k_422_10", "testsrc2", 3840, 2160, 2, "", 10, 422, ["-preset", "ultrafast", "-x265-params", "log-level=error"]),
// performance reference streams (tests/it/perf.rs)
fx!("bench_1080p", "testsrc2", 1920, 1080, 60, "noise=alls=3:allf=t", 8, ["-preset", "medium", "-crf", "24", "-x265-params", "log-level=error"]),
fx!("bench_4k", "testsrc2", 3840, 2160, 20, "noise=alls=3:allf=t", 10, ["-preset", "fast", "-crf", "26", "-x265-params", "log-level=error"]),
@@ -108,8 +146,21 @@ fn run(cmd: &mut Command) -> bool {
}
}
pub fn pix_fmt(bd: u32) -> &'static str {
if bd > 8 { "yuv420p10le" } else { "yuv420p" }
pub fn pix_fmt(bd: u32, chroma: ChromaFormat) -> &'static str {
match (chroma, bd > 8) {
(ChromaFormat::C420, false) => "yuv420p",
(ChromaFormat::C420, true) => "yuv420p10le",
(ChromaFormat::C422, false) => "yuv422p",
(ChromaFormat::C422, true) => "yuv422p10le",
}
}
/// Bytes of one raw frame of the fixture as ffmpeg writes it.
pub fn frame_size(f: &Fixture) -> usize {
let (w, h) = (f.width as usize, f.height as usize);
let bps = if f.bit_depth > 8 { 2 } else { 1 };
let (cw, ch) = (w.div_ceil(f.chroma.sub_width()), h.div_ceil(f.chroma.sub_height()));
(w * h + 2 * cw * ch) * bps
}
/// The pixel format ffprobe reports for the video stream in `path`.
@@ -129,7 +180,7 @@ pub fn ensure(f: &Fixture) -> Option<(PathBuf, PathBuf)> {
let yuv = dir.join(format!("{}.yuv", f.name));
if !hevc.exists() {
let tmp = craft_media_testkit::temp_path(&hevc);
let mut vf = format!("{}=size={}x{}:rate=25,format={}", f.source, f.width, f.height, pix_fmt(f.bit_depth));
let mut vf = format!("{}=size={}x{}:rate=25,format={}", f.source, f.width, f.height, pix_fmt(f.bit_depth, f.chroma));
if !f.filter.is_empty() {
vf.push(',');
vf.push_str(f.filter);
@@ -140,7 +191,7 @@ pub fn ensure(f: &Fixture) -> Option<(PathBuf, PathBuf)> {
if !f.args.contains(&"-c:v") {
c.args(["-c:v", "libx265"]);
}
c.args(["-pix_fmt", pix_fmt(f.bit_depth)]);
c.args(["-pix_fmt", pix_fmt(f.bit_depth, f.chroma)]);
c.args(f.args);
c.args(["-f", "hevc"]).arg(&tmp);
if !run(&mut c) {
@@ -157,10 +208,10 @@ pub fn ensure(f: &Fixture) -> Option<(PathBuf, PathBuf)> {
// 10, and the reference decode scales that up to 10 bits, so no sample would match.
let probe = craft_media_testkit::ffprobe()?;
let got = stream_pix_fmt(&probe, &hevc);
if got.as_deref() != Some(pix_fmt(f.bit_depth)) {
if got.as_deref() != Some(pix_fmt(f.bit_depth, f.chroma)) {
let _ = std::fs::remove_file(&hevc);
let _ = std::fs::remove_file(&yuv);
let reason = format!("the encoder made {} instead of {}", got.as_deref().unwrap_or("no stream"), pix_fmt(f.bit_depth));
let reason = format!("the encoder made {} instead of {}", got.as_deref().unwrap_or("no stream"), pix_fmt(f.bit_depth, f.chroma));
craft_media_testkit::skip_test(f.name, &format!("{reason} (libx265 built for 8 bits only?)"));
return None;
}
@@ -169,7 +220,7 @@ pub fn ensure(f: &Fixture) -> Option<(PathBuf, PathBuf)> {
let tmp = craft_media_testkit::temp_path(&yuv);
let mut c = Command::new(&ff);
c.args(["-hide_banner", "-loglevel", "error", "-y", "-i"]).arg(&hevc);
c.args(["-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", pix_fmt(f.bit_depth)]).arg(&tmp);
c.args(["-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", pix_fmt(f.bit_depth, f.chroma)]).arg(&tmp);
assert!(run(&mut c), "reference decode failed for {}", f.name);
craft_media_testkit::fixtures::publish(&tmp, &yuv).unwrap();
}
@@ -246,11 +297,13 @@ pub fn decode_file_opts(path: &Path, threads: usize, draft: bool) -> Result<Vec<
Ok(out)
}
/// Compare decoded pictures with a raw yuv420p / yuv420p10le reference; returns a diagnostic on the
/// first mismatch (frame, plane, sample position, CTB (64x64) and 8x8 block).
pub fn compare(pics: &[Picture], reference: &[u8], w: usize, h: usize, bd: u32) -> Result<(), String> {
/// Compare decoded pictures with a raw planar reference (yuv420p / yuv420p10le / yuv422p /
/// yuv422p10le); returns a diagnostic on the first mismatch (frame, plane, sample position, CTB
/// (64x64) and 8x8 block).
pub fn compare(pics: &[Picture], reference: &[u8], w: usize, h: usize, bd: u32, chroma: ChromaFormat) -> Result<(), String> {
let bps = if bd > 8 { 2 } else { 1 };
let (cw, ch) = (w.div_ceil(2), h.div_ceil(2));
let (sw, sh) = (chroma.sub_width(), chroma.sub_height());
let (cw, ch) = (w.div_ceil(sw), h.div_ceil(sh));
let fsize = (w * h + 2 * cw * ch) * bps;
let nref = reference.len() / fsize;
for (i, p) in pics.iter().enumerate() {
@@ -260,11 +313,15 @@ pub fn compare(pics: &[Picture], reference: &[u8], w: usize, h: usize, bd: u32)
if p.width as usize != w || p.height as usize != h {
return Err(format!("frame {i}: size {}x{} != {}x{}", p.width, p.height, w, h));
}
if p.chroma != chroma || p.chroma_width as usize != cw || p.chroma_height as usize != ch {
return Err(format!("frame {i}: chroma {:?} {}x{} != {chroma:?} {cw}x{ch}", p.chroma, p.chroma_width, p.chroma_height));
}
let f = &reference[i * fsize..(i + 1) * fsize];
let sample = |buf: &[u8], k: usize| if bps == 2 { u16::from_le_bytes([buf[2 * k], buf[2 * k + 1]]) } else { buf[k] as u16 };
let (ys, cs) = (w * h * bps, cw * ch * bps);
let planes = [("Y", &p.y, &f[..ys], w, h, 1), ("U", &p.u, &f[ys..ys + cs], cw, ch, 2), ("V", &p.v, &f[ys + cs..], cw, ch, 2)];
for (name, got, exp, pw, ph, sub) in planes {
let planes =
[("Y", &p.y, &f[..ys], w, h, (1, 1)), ("U", &p.u, &f[ys..ys + cs], cw, ch, (sw, sh)), ("V", &p.v, &f[ys + cs..], cw, ch, (sw, sh))];
for (name, got, exp, pw, ph, (subx, suby)) in planes {
let mut first = None;
let mut count = 0usize;
let mut maxd = 0i32;
@@ -281,7 +338,7 @@ pub fn compare(pics: &[Picture], reference: &[u8], w: usize, h: usize, bd: u32)
}
}
if let Some((x, y, a, b)) = first {
let (lx, ly) = (x * sub, y * sub);
let (lx, ly) = (x * subx, y * suby);
return Err(format!(
"frame {i} (poc {}, pts {}) plane {name}: first mismatch at ({x},{y}) [luma ({lx},{ly}), CTB64 ({},{}), 8x8 ({},{})]: got {a} want {b}; {count} samples differ, max diff {maxd}",
p.poc,
@@ -310,11 +367,12 @@ pub fn check_fixture(name: &str) -> Result<bool, String> {
let pics = match decode_file_threads(&hevc, threads) {
Ok(p) => p,
Err((au, e, partial)) => {
let cmp = compare(&partial, &reference, f.width as usize, f.height as usize, f.bit_depth).err().unwrap_or_default();
let cmp = compare(&partial, &reference, f.width as usize, f.height as usize, f.bit_depth, f.chroma).err().unwrap_or_default();
return Err(format!("{name} (threads {threads}): decode error at access unit {au}: {e} (after {} pictures) {cmp}", partial.len()));
}
};
compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth).map_err(|e| format!("{name} (threads {threads}): {e}"))?;
compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth, f.chroma)
.map_err(|e| format!("{name} (threads {threads}): {e}"))?;
check_pts(&pics).map_err(|e| format!("{name} (threads {threads}): {e}"))?;
if pics.iter().any(|p| p.draft) {
return Err(format!("{name} (threads {threads}): picture flagged draft without draft mode"));
crates/video/tests/hevc_decode/conformance.rs
+26 -6
View File
@@ -60,6 +60,28 @@ fixture_tests!(
uhd_4k_main10,
vt_main,
vt_main10,
intra_422,
intra_422_10,
intra_422_10_nosao_nodbk,
medium_422_10,
slow_422,
p_only_422_10,
bframes_422_10,
no_sao_422_10,
deblock_m3_3_422_10,
rect_amp_422_10,
tskip_422_10,
weightp_422_10,
qp_422_10,
crf51_422_10,
lossless_422_10,
ctu32_tu4_422_10,
ctu16_422_10,
scaling_list_422_10,
constrained_intra_422_10,
slices4_422_10,
odd_1918x1078_422_10,
uhd_4k_422_10,
);
/// Draft mode: sub-layer non-reference pictures skip deblocking and SAO and are flagged; every
@@ -67,16 +89,14 @@ fixture_tests!(
#[test]
fn draft_mode_changes_only_flagged_non_reference_pictures() {
let mut checked = 0;
for name in ["bframes", "medium", "slow", "weightp_main10"] {
for name in ["bframes", "medium", "slow", "weightp_main10", "bframes_422_10"] {
let f = common::fixture(name);
let Some((hevc, yuv)) = common::ensure(f) else {
eprintln!("skipped {name}");
continue;
};
let reference = std::fs::read(&yuv).unwrap();
let (w, h) = (f.width as usize, f.height as usize);
let bps = if f.bit_depth > 8 { 2 } else { 1 };
let fsize = (w * h + 2 * w.div_ceil(2) * h.div_ceil(2)) * bps;
let fsize = common::frame_size(f);
for threads in [1, 3] {
let pics = common::decode_file_opts(&hevc, threads, true).unwrap_or_else(|(au, e, _)| panic!("{name}: error at {au}: {e}"));
common::check_pts(&pics).unwrap_or_else(|e| panic!("{name}: {e}"));
@@ -125,10 +145,10 @@ fn compare_detects_mismatch() {
let Some((hevc, yuv)) = common::ensure(f) else { return };
let pics = common::decode_file_threads(&hevc, 1).map_err(|e| e.1).unwrap();
let mut reference = std::fs::read(yuv).unwrap();
common::compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth).unwrap();
common::compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth, f.chroma).unwrap();
let pos = reference.len() - 7;
reference[pos] ^= 1;
let err = common::compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth).unwrap_err();
let err = common::compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth, f.chroma).unwrap_err();
assert!(err.contains("plane V"), "{err}");
}
crates/video/tests/hevc_decode/robustness.rs
+1 -1
View File
@@ -57,7 +57,7 @@ fn hvcc_input_matches_reference() {
}
pics.extend(dec.flush());
let reference = std::fs::read(yuv).unwrap();
common::compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth).unwrap();
common::compare(&pics, &reference, f.width as usize, f.height as usize, f.bit_depth, f.chroma).unwrap();
let mut pts: Vec<i64> = pics.iter().map(|p| p.pts).collect();
pts.sort();
assert_eq!(pts, (0..samples.len() as i64).map(|i| i * 1000).collect::<Vec<_>>());