[FFmpeg-cvslog] avcodec/xpmdec: Check size before allocation to avoid truncation

[Michael Niedermayer]

ffmpeg | branch: release/6.0 | Michael Niedermayer <michael at niedermayer.cc> | Thu Jan 12 22:05:07 2023 +0100| [1eb1acb62b5afe215643a55f06c16545e5b20fb6] | committer: Michael Niedermayer

avcodec/xpmdec: Check size before allocation to avoid truncation

Fixes:OOM
Fixes:out of array access (no testcase)
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XPM_fuzzer-6573323838685184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 95f0f84dae4f040d91f1e60dc5438612c58e8906)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=1eb1acb62b5afe215643a55f06c16545e5b20fb6
---

 libavcodec/xpmdec.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/xpmdec.c b/libavcodec/xpmdec.c
index ff1f51dd32..2550afb9d6 100644
--- a/libavcodec/xpmdec.c
+++ b/libavcodec/xpmdec.c
@@ -354,6 +354,9 @@ static int xpm_decode_frame(AVCodecContext *avctx, AVFrame *p,
         return AVERROR_INVALIDDATA;
     }
 
+    if (size > SIZE_MAX / 4)
+        return AVERROR(ENOMEM);
+
     size *= 4;
 
     ptr += mod_strcspn(ptr, ",") + 1;