[FFmpeg-devel] [PATCH 2/5] avcodec/flacdec: Fix integer overflow in "33bit" DECODER_SUBFRAME_FIXED_WIDE()

[Michael Niedermayer]

Fixes: signed integer overflow: 4 * 2307917133220067266 cannot be represented in type 'long'
Fixes: 62164/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLAC_fuzzer-6307690022043648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
---
 libavcodec/flacdec.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libavcodec/flacdec.c b/libavcodec/flacdec.c
index 0569f019b3a..ed2de14d0a9 100644
--- a/libavcodec/flacdec.c
+++ b/libavcodec/flacdec.c
@@ -366,19 +366,19 @@ static int decode_subframe_fixed(FLACContext *s, int32_t *decoded,
         break;                                                        \
     case 1:                                                           \
         for (int i = pred_order; i < blocksize; i++)                  \
-            decoded[i] = (int64_t)residual[i] + (int64_t)decoded[i-1];\
+            decoded[i] = (uint64_t)residual[i] + (uint64_t)decoded[i-1];\
         break;                                                        \
     case 2:                                                           \
         for (int i = pred_order; i < blocksize; i++)                  \
-            decoded[i] = (int64_t)residual[i] + 2*(int64_t)decoded[i-1] - (int64_t)decoded[i-2];  \
+            decoded[i] = (uint64_t)residual[i] + 2*(uint64_t)decoded[i-1] - (uint64_t)decoded[i-2];  \
         break;                                                        \
     case 3:                                                           \
         for (int i = pred_order; i < blocksize; i++)                  \
-            decoded[i] = (int64_t)residual[i] + 3*(int64_t)decoded[i-1] - 3*(int64_t)decoded[i-2] + (int64_t)decoded[i-3];   \
+            decoded[i] = (uint64_t)residual[i] + 3*(uint64_t)decoded[i-1] - 3*(uint64_t)decoded[i-2] + (uint64_t)decoded[i-3];   \
         break;                                                        \
     case 4:                                                           \
         for (int i = pred_order; i < blocksize; i++)                  \
-            decoded[i] = (int64_t)residual[i] + 4*(int64_t)decoded[i-1] - 6*(int64_t)decoded[i-2] + 4*(int64_t)decoded[i-3] - (int64_t)decoded[i-4];   \
+            decoded[i] = (uint64_t)residual[i] + 4*(uint64_t)decoded[i-1] - 6*(uint64_t)decoded[i-2] + 4*(uint64_t)decoded[i-3] - (uint64_t)decoded[i-4];   \
         break;                                                        \
     default:                                                          \
         av_log(s->avctx, AV_LOG_ERROR, "illegal pred order %d\n", pred_order);   \
-- 
2.17.1