[FFmpeg-trac] #5063(avcodec:new): signed integer overflow in get_scale_factor

[FFmpeg]

#5063: signed integer overflow in get_scale_factor
---------------------------------+--------------------------------------
             Reporter:  tsmith   |                     Type:  defect
               Status:  new      |                 Priority:  normal
            Component:  avcodec  |                  Version:  git-master
             Keywords:           |               Blocked By:
             Blocking:           |  Reproduced by developer:  0
Analyzed by developer:  0        |
---------------------------------+--------------------------------------
 Summary of the bug:

 This is an Undefined behavior sanitizer (UBSan) runtime error.

 libavcodec/h264_direct.c:45:35: runtime error: signed integer overflow:
 -2147483648 - 1 cannot be represented in type 'int'
     #0 0x81043c in get_scale_factor
 /home/user/code/ffmpeg/libavcodec/h264_direct.c:41:32
     #1 0x80f4c0 in ff_h264_direct_dist_scale_factor
 /home/user/code/ffmpeg/libavcodec/h264_direct.c:69:36
     #2 0x951a0a in ff_h264_decode_slice_header
 /home/user/code/ffmpeg/libavcodec/h264_slice.c:1791:9
     #3 0x7a2b82 in decode_nal_units
 /home/user/code/ffmpeg/libavcodec/h264.c:1532:28
     #4 0x7b4469 in h264_decode_frame
 /home/user/code/ffmpeg/libavcodec/h264.c:1840:17
     #5 0xe89945 in avcodec_decode_video2
 /home/user/code/ffmpeg/libavcodec/utils.c:2105:19
     #6 0x5b383a in decode_video /home/user/code/ffmpeg/ffmpeg.c:2090:11
     #7 0x5b383a in process_input_packet
 /home/user/code/ffmpeg/ffmpeg.c:2339
     #8 0x5d683d in process_input /home/user/code/ffmpeg/ffmpeg.c:3960:5
     #9 0x5810a8 in transcode_step /home/user/code/ffmpeg/ffmpeg.c:4048:11
     #10 0x5810a8 in transcode /home/user/code/ffmpeg/ffmpeg.c:4102
     #11 0x57af12 in main /home/user/code/ffmpeg/ffmpeg.c:4295:9
     #12 0x7f847ff83ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu
 /libc-start.c:287
     #13 0x41ad25 in _start (/home/user/Desktop/ffmpeg/ffmpeg_ub+0x41ad25)

 How to reproduce:
 {{{
 % ./ffmpeg -v 0 -nostats -f h264 -i test_case.264 -f null -
 ffmpeg version N-76984-g259c71c
 built on Linux x86_64
 }}}
 Patches should be submitted to the ffmpeg-devel mailing list and not this
 bug tracker.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/5063>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker