[FFmpeg-devel] [PATCH] oggparsedaala: reject too large gpshift

[Andreas Cadhalpun]

Also use uint32_t for the shift calculation, as 1 << 31 is undefined
for int32_t.

This fixes ubsan runtime error: shift exponent is too large for
32-bit type 'int'

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
---
 libavformat/oggparsedaala.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavformat/oggparsedaala.c b/libavformat/oggparsedaala.c
index 24567f9..9f27ba6 100644
--- a/libavformat/oggparsedaala.c
+++ b/libavformat/oggparsedaala.c
@@ -123,7 +123,12 @@ static int daala_header(AVFormatContext *s, int idx)
 
         hdr->frame_duration = bytestream2_get_ne32(&gb);
         hdr->gpshift = bytestream2_get_byte(&gb);
-        hdr->gpmask  = (1 << hdr->gpshift) - 1;
+        if (hdr->gpshift >= 32) {
+            av_log(s, AV_LOG_ERROR, "Too large gpshift %d (>= 32).\n",
+                   hdr->gpshift);
+            return AVERROR_INVALIDDATA;
+        }
+        hdr->gpmask  = ((uint32_t)1 << hdr->gpshift) - 1;
 
         hdr->format.depth  = 8 + 2*(bytestream2_get_byte(&gb)-1);
 
-- 
2.6.4