[FFmpeg-trac] #6224(undetermined:new): signed integer overflow in decode_coeffs_b_generic()

FFmpeg trac at avcodec.org
Tue Mar 7 01:02:03 EET 2017


#6224: signed integer overflow in decode_coeffs_b_generic()
-------------------------------------+-------------------------------------
             Reporter:  tsmith       |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:
  undetermined                       |  unspecified
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 UBSan: libavcodec/vp9.c:2255:13: runtime error: signed integer overflow:
 -262210 * 18766 cannot be represented in type 'int'

 How to reproduce:
 Built with Undefined behavior sanitizer (-fsanitize=undefined)
 {{{
 % ffmpeg -f ivf -i <test_case> -f null -
 ffmpeg version N-83756-g8aa4f3b Copyright (c) 2000-2017 the FFmpeg
 developers
   built with clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
   configuration: --cc=afl-clang-fast --cxx=afl-clang-fast++ --disable-
 libxcb --disable-xlib --disable-logging --disable-ffprobe --disable-ffplay
 --disable-sdl2 --disable-ffserver --disable-doc --disable-pthreads
 --disable-network --disable-d3d11va --disable-dxva2 --disable-vaapi
 --disable-vda --disable-vdpau --disable-stripping --disable-runtime-
 cpudetect --disable-postproc --disable-securetransport --disable-iconv
 --disable-swscale --disable-avdevice --disable-videotoolbox --disable-
 everything --enable-encoder=wrapped_avframe --disable-lzma --enable-
 protocol='file,pipe' --enable-muxer=null --enable-demuxer=ivf --enable-
 parser=vp9 --enable-decoder=vp9 --disable-pic

 libavcodec/vp9.c:2255:13: runtime error: signed integer overflow: -262210
 * 18766 cannot be represented in type 'int'
     #0 0x9610af in decode_coeffs_b_generic libavcodec/vp9.c:2255:13
     #1 0x9610af in decode_coeffs_b_16bpp libavcodec/vp9.c:2289
     #2 0x8fb97a in decode_coeffs libavcodec/vp9.c:2408:9
     #3 0x8fb97a in decode_coeffs_16bpp libavcodec/vp9.c:2477
     #4 0x80377d in decode_b libavcodec/vp9.c:3240:30
     #5 0x7e632f in decode_sb libavcodec/vp9.c:3476:13
     #6 0x7e6b50 in decode_sb libavcodec/vp9.c:3444:17
     #7 0x7b1de2 in vp9_decode_frame libavcodec/vp9.c:4204:29
     #8 0x78369a in avcodec_decode_video2 libavcodec/utils.c:2263:19
     #9 0x791e73 in do_decode libavcodec/utils.c:2796:15
     #10 0x791354 in avcodec_send_packet libavcodec/utils.c:2885:12
     #11 0x7038f0 in try_decode_frame libavformat/utils.c:3000:19
     #12 0x6ee52b in avformat_find_stream_info libavformat/utils.c:3701:9
     #13 0x5094c8 in open_input_file ffmpeg_opt.c:1012:11
     #14 0x5059dc in open_files ffmpeg_opt.c:3197:15
     #15 0x505071 in ffmpeg_parse_options ffmpeg_opt.c:3237:11
     #16 0x5743d0 in main ffmpeg.c:4679:11
     #17 0x7fd38f21082f in __libc_start_main /build/glibc-
 t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291
     #18 0x4198b8 in _start (ffmpeg+0x4198b8)
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6224>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list