[FFmpeg-trac] #3456(undetermined:new): crash in h263 video encoding

[FFmpeg]

#3456: crash in h263 video encoding
-------------------------------------+-------------------------------------
             Reporter:  rogerdpack   |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:
  undetermined                       |  unspecified
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug: segfault in some sse code with the h263 video encoder.

 NB I could only reproduce this on intel processors seemingly, not my AMD
 box. FWIW.

 How to reproduce:
 {{{
 % ffmpeg  -y -f lavfi -i testsrc -vf scale=256:144,crop=176:144:40:0  -an
 -f 3gp a.3gp
 ffmpeg version N-61143-g64e4bd7 Copyright (c) 2000-2014 the FFmpeg
 developers
   built on Mar  6 2014 17:25:54 with gcc 4.8.2 (GCC)
   configuration: --arch=x86 --target-os=mingw32 --cross-
 prefix=/home/rogerdpack/dev/ffmpeg-windows-build-
 helpers/sandbox/mingw-w64-i686/bin/i686-w64-mingw32- --pkg-config=pkg-
 config --enable-gpl --enable-libx264 --enable-avisynth --enable-libxvid
 --enable-libmp3lame --enable-version3 --enable-zlib --enable-librtmp
 --enable-libvorbis --enable-libtheora --enable-libspeex --enable-
 libopenjpeg --enable-gnutls --enable-libgsm --enable-libfreetype --enable-
 libopus --disable-w32threads --enable-frei0r --enable-filter=frei0r
 --enable-libvo-aacenc --enable-bzlib --enable-libxavs --extra-
 cflags=-DPTW32_STATIC_LIB --enable-libopencore-amrnb --enable-libopencore-
 amrwb --enable-libvo-amrwbenc --enable-libschroedinger --enable-libvpx
 --enable-libilbc --prefix=/home/rogerdpack/dev/ffmpeg-windows-build-
 helpers/sandbox/mingw-w64-i686/i686-w64-mingw32 --enable-static --disable-
 shared --enable-libsoxr --enable-fontconfig --enable-libass --enable-
 libutvideo --enable-libbluray --enable-iconv --enable-libtwolame --extra-
 cflags=-DLIBTWOLAME_STATIC --enable-libzvbi --enable-libcaca --enable-
 libmodplug --extra-libs=-lstdc++ --extra-libs=-lpng --enable-libvidstab
 --enable-libx265 --extra-cflags= --extra-cflags= --enable-runtime-
 cpudetect
   libavutil      52. 66.101 / 52. 66.101
   libavcodec     55. 52.102 / 55. 52.102
   libavformat    55. 33.101 / 55. 33.101
   libavdevice    55. 11.100 / 55. 11.100
   libavfilter     4.  3.100 /  4.  3.100
   libswscale      2.  5.101 /  2.  5.101
   libswresample   0. 18.100 /  0. 18.100
   libpostproc    52.  3.100 / 52.  3.100
 Input #0, lavfi, from 'testsrc':
   Duration: N/A, start: 0.000000, bitrate: N/A
     Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240
 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
 Output #0, 3gp, to 'a.3gp':
   Metadata:
     encoder         : Lavf55.33.101
     Stream #0:0: Video: h263 (s263 / 0x33363273), yuv420p, 176x144 [SAR
 3:4 DAR 11:12], q=2-31, 200 kb/s, 12800 tbn, 25 tbc
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo -> h263)
 Press [q] to stop, [?] for help
 }}}

 (hard crash).

 gdb bt at crash time:


 Program received signal SIGSEGV, Segmentation fault.
 0x00aca839 in sad16_sse2 () at libavcodec/x86/motion_est.c:102
 102     libavcodec/x86/motion_est.c: No such file or directory.
         in libavcodec/x86/motion_est.c
 (gdb) bt
 #0  0x00aca839 in sad16_sse2 () at libavcodec/x86/motion_est.c:102
 #1  0x009aa4e4 in ff_epzs_motion_search ()
 #2  0x009afd9a in ff_estimate_p_frame_motion ()
 #3  0x00694f6c in estimate_motion_thread () at
 libavcodec/mpegvideo_enc.c:2528
 #4  0x005c8b0c in avcodec_default_execute () at libavcodec/utils.c:1060
 #5  0x006965b5 in ff_MPV_encode_picture () at
 libavcodec/mpegvideo_enc.c:3418
 #6  0x005c95c4 in avcodec_encode_video2 () at libavcodec/utils.c:1892
 #7  0x00418b3f in reap_filters () at ffmpeg.c:997
 #8  0x0160cad5 in main () at ffmpeg.c:3399


 Also of note, this version of ffmpeg did not seem to exhibit the problem:

 ffmpeg version N-59140-ga8d7028 Copyright (c) 2000-2013 the FFmpeg
 developers
   built on Dec 16 2013 08:29:51 with gcc 4.8.1 (GCC)

 Also of note: seems reproducible on both linux and windows.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/3456>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker