[FFmpeg-trac] #6224(undetermined:new): signed integer overflow in decode_coeffs_b_generic()
FFmpeg
trac at avcodec.org
Tue Mar 7 01:02:03 EET 2017
#6224: signed integer overflow in decode_coeffs_b_generic()
-------------------------------------+-------------------------------------
Reporter: tsmith | Type: defect
Status: new | Priority: normal
Component: | Version:
undetermined | unspecified
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Summary of the bug:
UBSan: libavcodec/vp9.c:2255:13: runtime error: signed integer overflow:
-262210 * 18766 cannot be represented in type 'int'
How to reproduce:
Built with Undefined behavior sanitizer (-fsanitize=undefined)
{{{
% ffmpeg -f ivf -i <test_case> -f null -
ffmpeg version N-83756-g8aa4f3b Copyright (c) 2000-2017 the FFmpeg
developers
built with clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
configuration: --cc=afl-clang-fast --cxx=afl-clang-fast++ --disable-
libxcb --disable-xlib --disable-logging --disable-ffprobe --disable-ffplay
--disable-sdl2 --disable-ffserver --disable-doc --disable-pthreads
--disable-network --disable-d3d11va --disable-dxva2 --disable-vaapi
--disable-vda --disable-vdpau --disable-stripping --disable-runtime-
cpudetect --disable-postproc --disable-securetransport --disable-iconv
--disable-swscale --disable-avdevice --disable-videotoolbox --disable-
everything --enable-encoder=wrapped_avframe --disable-lzma --enable-
protocol='file,pipe' --enable-muxer=null --enable-demuxer=ivf --enable-
parser=vp9 --enable-decoder=vp9 --disable-pic
libavcodec/vp9.c:2255:13: runtime error: signed integer overflow: -262210
* 18766 cannot be represented in type 'int'
#0 0x9610af in decode_coeffs_b_generic libavcodec/vp9.c:2255:13
#1 0x9610af in decode_coeffs_b_16bpp libavcodec/vp9.c:2289
#2 0x8fb97a in decode_coeffs libavcodec/vp9.c:2408:9
#3 0x8fb97a in decode_coeffs_16bpp libavcodec/vp9.c:2477
#4 0x80377d in decode_b libavcodec/vp9.c:3240:30
#5 0x7e632f in decode_sb libavcodec/vp9.c:3476:13
#6 0x7e6b50 in decode_sb libavcodec/vp9.c:3444:17
#7 0x7b1de2 in vp9_decode_frame libavcodec/vp9.c:4204:29
#8 0x78369a in avcodec_decode_video2 libavcodec/utils.c:2263:19
#9 0x791e73 in do_decode libavcodec/utils.c:2796:15
#10 0x791354 in avcodec_send_packet libavcodec/utils.c:2885:12
#11 0x7038f0 in try_decode_frame libavformat/utils.c:3000:19
#12 0x6ee52b in avformat_find_stream_info libavformat/utils.c:3701:9
#13 0x5094c8 in open_input_file ffmpeg_opt.c:1012:11
#14 0x5059dc in open_files ffmpeg_opt.c:3197:15
#15 0x505071 in ffmpeg_parse_options ffmpeg_opt.c:3237:11
#16 0x5743d0 in main ffmpeg.c:4679:11
#17 0x7fd38f21082f in __libc_start_main /build/glibc-
t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291
#18 0x4198b8 in _start (ffmpeg+0x4198b8)
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/6224>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list