Commit 78a29fd62f for ffmpeg
commit 78a29fd62f090b9ab42feb2cf9dddbb5d4d42f1e
Author: Michael Niedermayer <michael@niedermayer.cc>
Date: Tue Oct 6 07:11:38 2026 +0200
avformat/latmenc: Take the extradata size as size_t
Validating the size before it is narrowed was suggested in the report.
Fixes: out of array write (unsupported av_max_alloc() > 2gb)
Fixes: uBIqgh88C9dX
Fixes: AISLE-2026-0111-00265
Out of array write Replicated through API with ASAN, with av_max_alloc() raised above 4 GiB
Found-by: Joshua Rogers <joshua.rogers@aisle.com>
diff --git a/libavformat/latmenc.c b/libavformat/latmenc.c
index 05004048aa..b17ffa254a 100644
--- a/libavformat/latmenc.c
+++ b/libavformat/latmenc.c
@@ -57,7 +57,7 @@ static const AVClass latm_muxer_class = {
.version = LIBAVUTIL_VERSION_INT,
};
-static int latm_decode_extradata(AVFormatContext *s, uint8_t *buf, int size)
+static int latm_decode_extradata(AVFormatContext *s, uint8_t *buf, size_t size)
{
LATMContext *ctx = s->priv_data;
MPEG4AudioConfig m4ac;