[FFmpeg-trac] #5060(avcodec:new): signed integer overflow in decode_cabac_residual_internal

[FFmpeg]

#5060: signed integer overflow in decode_cabac_residual_internal
------------------------------------+-----------------------------------
             Reporter:  tsmith      |                    Owner:
                 Type:  defect      |                   Status:  new
             Priority:  normal      |                Component:  avcodec
              Version:  git-master  |               Resolution:
             Keywords:              |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+-----------------------------------
Description changed by Timothy_Gu:

Old description:

> Summary of the bug:
>
> This is an Undefined behavior sanitizer (UBSan) runtime error.
>
> libavcodec/h264_cabac.c:1755:9: runtime error: signed integer overflow:
> 2147483647 + 14 cannot be represented in type 'int'
>     #0 0x17ec392 in decode_cabac_residual_internal
> /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1755:9
>     #1 0x17ec392 in decode_cabac_residual_nondc_internal
> /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1795
>     #2 0x17b00ec in decode_cabac_residual_nondc
> /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1856:5
>     #3 0x17b00ec in decode_cabac_luma_residual
> /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1878
>     #4 0x17b00ec in ff_h264_decode_mb_cabac
> /home/user/code/ffmpeg/libavcodec/h264_cabac.c:2403
>     #5 0x96cad2 in decode_slice
> /home/user/code/ffmpeg/libavcodec/h264_slice.c:2404:23
>     #6 0x9697d2 in ff_h264_execute_decode_slices
> /home/user/code/ffmpeg/libavcodec/h264_slice.c:2567:15
>     #7 0x7a1dcc in decode_nal_units
> /home/user/code/ffmpeg/libavcodec/h264.c:1655:23
>     #8 0x7b4469 in h264_decode_frame
> /home/user/code/ffmpeg/libavcodec/h264.c:1840:17
>     #9 0xe89945 in avcodec_decode_video2
> /home/user/code/ffmpeg/libavcodec/utils.c:2105:19
>     #10 0x756b2d in try_decode_frame
> /home/user/code/ffmpeg/libavformat/utils.c:2752:19
>     #11 0x7434ab in avformat_find_stream_info
> /home/user/code/ffmpeg/libavformat/utils.c:3401:9
>     #12 0x507779 in open_input_file
> /home/user/code/ffmpeg/ffmpeg_opt.c:970:11
>     #13 0x50365f in open_files
> /home/user/code/ffmpeg/ffmpeg_opt.c:2939:15
>     #14 0x502706 in ffmpeg_parse_options
> /home/user/code/ffmpeg/ffmpeg_opt.c:2976:11
>     #15 0x57ab79 in main /home/user/code/ffmpeg/ffmpeg.c:4273:11
>     #16 0x7f965275eec4 in __libc_start_main /build/buildd/eglibc-2.19/csu
> /libc-start.c:287
>     #17 0x41ad25 in _start (/home/user/Desktop/ffmpeg/ffmpeg_ub+0x41ad25)
>
> How to reproduce:
> {{{
> % ./ffmpeg -v 0 -nostats -f h264 -i test_case.264 -f null -
> ffmpeg version N-76984-g259c71c
> built on Linux x86_64
> }}}
> Patches should be submitted to the ffmpeg-devel mailing list and not this
> bug tracker.

New description:

 Summary of the bug:

 This is an Undefined behavior sanitizer (UBSan) runtime error.

 {{{
 libavcodec/h264_cabac.c:1755:9: runtime error: signed integer overflow:
 2147483647 + 14 cannot be represented in type 'int'
     #0 0x17ec392 in decode_cabac_residual_internal
 /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1755:9
     #1 0x17ec392 in decode_cabac_residual_nondc_internal
 /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1795
     #2 0x17b00ec in decode_cabac_residual_nondc
 /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1856:5
     #3 0x17b00ec in decode_cabac_luma_residual
 /home/user/code/ffmpeg/libavcodec/h264_cabac.c:1878
     #4 0x17b00ec in ff_h264_decode_mb_cabac
 /home/user/code/ffmpeg/libavcodec/h264_cabac.c:2403
     #5 0x96cad2 in decode_slice
 /home/user/code/ffmpeg/libavcodec/h264_slice.c:2404:23
     #6 0x9697d2 in ff_h264_execute_decode_slices
 /home/user/code/ffmpeg/libavcodec/h264_slice.c:2567:15
     #7 0x7a1dcc in decode_nal_units
 /home/user/code/ffmpeg/libavcodec/h264.c:1655:23
     #8 0x7b4469 in h264_decode_frame
 /home/user/code/ffmpeg/libavcodec/h264.c:1840:17
     #9 0xe89945 in avcodec_decode_video2
 /home/user/code/ffmpeg/libavcodec/utils.c:2105:19
     #10 0x756b2d in try_decode_frame
 /home/user/code/ffmpeg/libavformat/utils.c:2752:19
     #11 0x7434ab in avformat_find_stream_info
 /home/user/code/ffmpeg/libavformat/utils.c:3401:9
     #12 0x507779 in open_input_file
 /home/user/code/ffmpeg/ffmpeg_opt.c:970:11
     #13 0x50365f in open_files /home/user/code/ffmpeg/ffmpeg_opt.c:2939:15
     #14 0x502706 in ffmpeg_parse_options
 /home/user/code/ffmpeg/ffmpeg_opt.c:2976:11
     #15 0x57ab79 in main /home/user/code/ffmpeg/ffmpeg.c:4273:11
     #16 0x7f965275eec4 in __libc_start_main /build/buildd/eglibc-2.19/csu
 /libc-start.c:287
     #17 0x41ad25 in _start (/home/user/Desktop/ffmpeg/ffmpeg_ub+0x41ad25)
 }}}

 How to reproduce:
 {{{
 % ./ffmpeg -v 0 -nostats -f h264 -i test_case.264 -f null -
 ffmpeg version N-76984-g259c71c
 built on Linux x86_64
 }}}
 Patches should be submitted to the ffmpeg-devel mailing list and not this
 bug tracker.

--

--
Ticket URL: <https://trac.ffmpeg.org/ticket/5060#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker