Commit f03ca051da for ffmpeg
commit f03ca051da510ea71972c1ed46eb18b07798cb40
Author: Luke McGartland <lmcgartland@sequence.film>
Date: Fri Oct 16 16:15:13 2026 -0700
vulkan/prores_ks_estimate_slice: gallop and bisect the overquant search
A slice still over budget at max_quant searches for a quantizer that
fits by estimating each quantizer above it in turn, up to 127. With the
4:2:2 profiles' budgets on detailed content many slices take dozens of
steps, and since the loop is subgroup-uniform every lane of the subgroup
waits for the slowest slice. On a Tesla T4, 1080p HQ ran slower than
4444.
Gallop upwards until an estimate fits, then bisect between the highest
quantizer known to overshoot and the lowest known to fit, in about
2 * log2(distance) estimates. When bits fall monotonically with the
quantizer, this finds the same quantizer as the linear search. The DC
and run/level estimates are context-adaptive, so occasionally they
don't, and the search settles on a higher quantizer that also fits.
1080p, steady state, fps, on a Tesla T4 (NVIDIA 580.159.04), measured
with 9.0.2 plus this shader's fixes from master (e54f60ed, ddc140eb,
444564d5, 03e58384):
linear bisect
HQ, SMPTE bars 17.9 73.6
HQ, noise 14.8 56.2
Proxy, noise 16.5 71.0
4444, SMPTE bars 30.9 63.1
4444, noise 24.4 47.8
Output was byte-identical there for every row. On master with
KosmicKrisp (Apple M4), 1080p testsrc2, smptebars, mandelbrot, a camera
recording and a screen recording in all six profiles, 27 of 30 encodes
were byte-identical. The other three were Proxy, the tightest budget:
each changed by a few bytes, and average PSNR moved by at most 0.0003 dB.
diff --git a/libavcodec/vulkan/prores_ks_estimate_slice.comp.glsl b/libavcodec/vulkan/prores_ks_estimate_slice.comp.glsl
index 26720229cf..1179956356 100644
--- a/libavcodec/vulkan/prores_ks_estimate_slice.comp.glsl
+++ b/libavcodec/vulkan/prores_ks_estimate_slice.comp.glsl
@@ -290,10 +290,21 @@ void main() [[maximally_reconverges]]
scores[slice].score[max_quant + 1][plane] = error;
scores[slice].overquant = int(max_quant);
} else {
- /* Keep searching until an encoding fits our budget. The loop stays
- * subgroup-uniform, as the sums must not run under divergence. */
- bool done = false;
- for (uint oq = max_quant + 1; oq < 128 && !subgroupAll(done); ++oq) {
+ /* Search upwards for a quantizer whose encoding fits our budget.
+ * Gallop until an estimate fits, then bisect between the highest
+ * quantizer known to overshoot and the lowest known to fit: about
+ * 2 * log2(distance) estimates instead of one per step. Bits nearly
+ * always fall as the quantizer rises, and then this finds the same
+ * quantizer as stepping up one at a time. When they don't, it may
+ * settle on a higher one that also fits. A slice that fits nowhere
+ * ends on q = 127's estimates, as before.
+ * The loop stays subgroup-uniform, as the sums must not run under
+ * divergence. */
+ uint below = max_quant, fit = 128, step = 1;
+ int fit_bits = 0, fit_error = 0, fit_total_bits = 0, fit_total_score = 0;
+ bool galloping = true, done = false;
+ while (!subgroupAll(done)) {
+ uint oq = galloping ? min(below + step, 127u) : (below + fit) / 2;
int e = 0, b = 0;
if (!done)
b = plane == 3 ? estimate_alpha_plane(slice) :
@@ -306,11 +317,30 @@ void main() [[maximally_reconverges]]
error = e;
total_bits = tb;
total_score = ts;
- q = oq;
- done = total_bits <= bits_per_mb * mbs_per_slice;
+ if (total_bits <= bits_per_mb * mbs_per_slice) {
+ fit = oq;
+ fit_bits = b;
+ fit_error = e;
+ fit_total_bits = tb;
+ fit_total_score = ts;
+ galloping = false;
+ } else {
+ below = oq;
+ step <<= 1;
+ }
+ done = galloping ? below == 127 : fit - below <= 1;
}
}
+ q = 127;
+ if (fit < 128) {
+ bits = fit_bits;
+ error = fit_error;
+ total_bits = fit_total_bits;
+ total_score = fit_total_score;
+ q = fit;
+ }
+
scores[slice].bits[max_quant + 1][plane] = bits;
scores[slice].score[max_quant + 1][plane] = error;
scores[slice].total_bits[max_quant + 1] = total_bits;