Files
vangel76andClaude Sonnet 5.5 80af9b45e2 av1enc: fix the bitrate overshoot of the bitrate mode rate control
Bitrate mode gave key frames keyint-dependent extra bits (about 6.8 frame
budgets at the 60 frame default) but left the inter frames at the full
frame budget, so every group of frames overshot by (boost - 1) / keyint,
9.6 % at keyint 60 (up to +187 % at keyint 5 on some content). The
accumulated deviation was tracked against the boosted budget, so the
feedback never saw it.

Now a group of keyint frames adds up to keyint * frame budget: inter
frames get (keyint - boost) / (keyint - 1) of it. Further changes to the
frame level control:

- The quantizer follows the size error with a measured slope (quantizer
  steps per halving of the size, taken from consecutive inter frames)
  instead of a fixed 12 steps per doubling. On noisy content the curve is
  several times steeper than that and the old gain oscillated between
  1 kB and 42 kB frames.
- Key frames have their own quantizer offset, adapted from key frame
  sizes only, instead of feeding the key frame error into the inter
  quantizer. A key frame after the first that misses its budget by more
  than 19 % is coded again (at most twice more), with a corrected
  quantizer found in the log domain of the quantizer step.
- A deviation from the budget is paid back over 16 frames, bounded to
  0.35x..2x of the budget.

Constant quantizer mode is untouched (byte identical output). The
encoded stream in bitrate mode changes.

Adds a test that the rate stays within 5 % of the target over 180
frames for three 8 bit and one 10 bit setting.

Part of #2391

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 13:38:37 +01:00
..