Commit 38ac3a9237 for ffmpeg
commit 38ac3a9237d1175e1461e6aee2dee24be1710a55
Author: Michael Niedermayer <michael@niedermayer.cc>
Date: Thu Oct 15 15:51:52 2026 +0200
avfilter/vf_codecview: compute the arrow head length in 64bit
Fixes: division by zero
Fixes: signed integer overflow
Fixes: NaN to int conversion
Fixes: 2qHh10XPbgiO
Fixes: swarm-Signed-integer-overflow-in-draw_arrow-length-calc-8f1f15
Division by zero Replicated through UnModified FFmpeg with ASAN
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN
NaN to int conversion Replicated through UnModified FFmpeg with UBSAN
Found-by: zhang xingxing <2388969553@qq.com>
The report suggested doing the shift in a wider unsigned type.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
diff --git a/libavfilter/vf_codecview.c b/libavfilter/vf_codecview.c
index f20056b91b..cd445e4e9c 100644
--- a/libavfilter/vf_codecview.c
+++ b/libavfilter/vf_codecview.c
@@ -190,7 +190,7 @@ static void draw_arrow(uint8_t *buf, int sx, int sy, int ex,
if (dx * dx + dy * dy > 3 * 3) {
int rx = dx + dy;
int ry = -dx + dy;
- int length = sqrt((rx * rx + ry * ry) << 8);
+ int length = sqrt((rx * (int64_t)rx + ry * (int64_t)ry) << 8);
// FIXME subpixel accuracy
rx = ROUNDED_DIV(rx * (3 << 4), length);