[FFmpeg-user] Segfault in mpdecimate filter when using ssse3/mmx extensions

Thu Dec 19 11:04:00 CET 2013

I’m trying to use the mpdecimate filter to chop out a series of duplicated frames in some media, but ffmpeg segfaults on the second frame passed through the filter.

The segfault does not occur if I compile with --disable-mmx, but still occurs with --disable-ssse3. Full console log and GDB trace of the segfault below.

The source media is available here: http://edward.torbett.co.uk/combined.mp4.

I’m building and running on the same machine, so no cross-compilation issues. I’m also building static executables.

I’ve included /proc/cpuinfo at the end for reference.

Any suggestions?

Regards,
Ed Torbett

$ /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg -v debug -i combined.mp4 -vf mpdecimate decimated.mp4
ffmpeg version N-41485-gf5d0398 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec 19 2013 08:55:36 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
  configuration: --enable-debug=3 --disable-stripping --prefix=/ldisk/ffmpeg/ffmpeg_build --extra-cflags='-I/ldisk/ffmpeg/ffmpeg_build/include -static' --extra-ldflags='-L/ldisk/ffmpeg/ffmpeg_build/lib -static' --bindir=/ldisk/ffmpeg/ffmpeg_build/bin --extra-libs='-ldl -lfreetype -lpng16 -lz -lbz2 -lm -lxml2 -lSDL' --enable-gpl --enable-nonfree --enable-libx264 --enable-libfreetype --disable-shared --enable-static
  libavutil      52. 58.101 / 52. 58.101
  libavcodec     55. 45.102 / 55. 45.102
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  0.100 /  4.  0.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input file with argument 'combined.mp4'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'mpdecimate'.
Reading option 'decimated.mp4' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file combined.mp4.
Successfully parsed a group of options.
Opening an input file: combined.mp4.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xb091d00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xb091d00] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xb091d00] Before avformat_find_stream_info() pos: 2826966 bytes read:41094 seeks:1
[h264 @ 0xb092660] no picture
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xb091d00] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xb091d00] After avformat_find_stream_info() pos: 64663 bytes read:105709 seeks:2 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'combined.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.9.100
  Duration: 00:00:50.08, start: 0.000000, bitrate: 451 kb/s
    Stream #0:0(und), 1, 1/12800: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 1/25, 450 kb/s, 12.50 fps, 12.50 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Successfully opened the file.
Parsing a group of options: output file decimated.mp4.
Applying option vf (set video filters) with argument mpdecimate.
Successfully parsed a group of options.
Opening an output file: decimated.mp4.
Successfully opened the file.
detected 2 logical cores
[Parsed_mpdecimate_0 @ 0xb0b2100] max_drop_count:0 hi:768 lo:320 frac:0.330000
[graph 0 input from stream 0:0 @ 0xb099e00] Setting 'video_size' to value '720x576'
[graph 0 input from stream 0:0 @ 0xb099e00] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0xb099e00] Setting 'time_base' to value '1/12800'
[graph 0 input from stream 0:0 @ 0xb099e00] Setting 'pixel_aspect' to value '16/15'
[graph 0 input from stream 0:0 @ 0xb099e00] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0xb099e00] Setting 'frame_rate' to value '25/2'
[graph 0 input from stream 0:0 @ 0xb099e00] w:720 h:576 pixfmt:yuv420p tb:1/12800 fr:25/2 sar:16/15 sws_param:flags=2
[format @ 0xb099920] compat: called with args=[yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16]
[format @ 0xb099920] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16'
[AVFilterGraph @ 0xb085e00] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[libx264 @ 0xb098980] using mv_range_thread = 88
[libx264 @ 0xb098980] using SAR=16/15
[libx264 @ 0xb098980] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
[libx264 @ 0xb098980] profile High, level 2.2
[libx264 @ 0xb098980] 264 - core 140 r34 1ca7bb9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=12 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'decimated.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.22.100
    Stream #0:0(und), 0, 1/12800: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 2/25, q=-1--1, 12800 tbn, 12.50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> libx264)
Press [q] to stop, [?] for help
[h264 @ 0xb5d22c0] no picture
[h264 @ 0xb6170a0] no picture
[Parsed_mpdecimate_0 @ 0xb0b2100] keep pts:0 pts_time:0 drop_count:-1
Segmentation fault

When run with gdb:

$ gdb /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-37.el5_7.1)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg...done.
(gdb) r -v debug -i combined.mp4 -vf mpdecimate decimated.mp4
Starting program: /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg -v debug -i combined.mp4 -vf mpdecimate decimated.mp4
[Thread debugging using libthread_db enabled]
ffmpeg version N-41485-gf5d0398 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec 19 2013 08:55:36 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
  configuration: --enable-debug=3 --disable-stripping --prefix=/ldisk/ffmpeg/ffmpeg_build --extra-cflags='-I/ldisk/ffmpeg/ffmpeg_build/include -static' --extra-ldflags='-L/ldisk/ffmpeg/ffmpeg_build/lib -static' --bindir=/ldisk/ffmpeg/ffmpeg_build/bin --extra-libs='-ldl -lfreetype -lpng16 -lz -lbz2 -lm -lxml2 -lSDL' --enable-gpl --enable-nonfree --enable-libx264 --enable-libfreetype --disable-shared --enable-static
  libavutil      52. 58.101 / 52. 58.101
  libavcodec     55. 45.102 / 55. 45.102
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  0.100 /  4.  0.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input file with argument 'combined.mp4'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'mpdecimate'.
Reading option 'decimated.mp4' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file combined.mp4.
Successfully parsed a group of options.
Opening an input file: combined.mp4.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x94d9d00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x94d9d00] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x94d9d00] Before avformat_find_stream_info() pos: 2826966 bytes read:41094 seeks:1
[h264 @ 0x94da660] no picture
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x94d9d00] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x94d9d00] After avformat_find_stream_info() pos: 64663 bytes read:105709 seeks:2 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'combined.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.9.100
  Duration: 00:00:50.08, start: 0.000000, bitrate: 451 kb/s
    Stream #0:0(und), 1, 1/12800: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 1/25, 450 kb/s, 12.50 fps, 12.50 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Successfully opened the file.
Parsing a group of options: output file decimated.mp4.
Applying option vf (set video filters) with argument mpdecimate.
Successfully parsed a group of options.
Opening an output file: decimated.mp4.
File 'decimated.mp4' already exists. Overwrite ? [y/N] y
Successfully opened the file.
detected 2 logical cores
[New Thread 0xb7f9bb90 (LWP 24760)]
[New Thread 0xb759bb90 (LWP 24761)]
[New Thread 0xb6b9bb90 (LWP 24762)]
[Parsed_mpdecimate_0 @ 0x94fa100] max_drop_count:0 hi:768 lo:320 frac:0.330000
[graph 0 input from stream 0:0 @ 0x94e1e00] Setting 'video_size' to value '720x576'
[graph 0 input from stream 0:0 @ 0x94e1e00] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x94e1e00] Setting 'time_base' to value '1/12800'
[graph 0 input from stream 0:0 @ 0x94e1e00] Setting 'pixel_aspect' to value '16/15'
[graph 0 input from stream 0:0 @ 0x94e1e00] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x94e1e00] Setting 'frame_rate' to value '25/2'
[graph 0 input from stream 0:0 @ 0x94e1e00] w:720 h:576 pixfmt:yuv420p tb:1/12800 fr:25/2 sar:16/15 sws_param:flags=2
[format @ 0x94e1920] compat: called with args=[yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16]
[format @ 0x94e1920] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16'
[AVFilterGraph @ 0x94cde00] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[libx264 @ 0x94e0980] using mv_range_thread = 88
[libx264 @ 0x94e0980] using SAR=16/15
[libx264 @ 0x94e0980] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
[New Thread 0xb602cb90 (LWP 24763)]
[New Thread 0xb562cb90 (LWP 24764)]
[New Thread 0xb4c2cb90 (LWP 24765)]
[New Thread 0xb32e8b90 (LWP 24766)]
[libx264 @ 0x94e0980] profile High, level 2.2
[libx264 @ 0x94e0980] 264 - core 140 r34 1ca7bb9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=12 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[New Thread 0xb28e8b90 (LWP 24767)]
[New Thread 0xb1ee8b90 (LWP 24768)]
[New Thread 0xb14e8b90 (LWP 24769)]
Output #0, mp4, to 'decimated.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.22.100
    Stream #0:0(und), 0, 1/12800: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 2/25, q=-1--1, 12800 tbn, 12.50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> libx264)
Press [q] to stop, [?] for help
[h264 @ 0x9a1a2c0] no picture
[h264 @ 0x9a5f0a0] no picture
[Parsed_mpdecimate_0 @ 0x94fa100] keep pts:0 pts_time:0 drop_count:-1

Program received signal SIGSEGV, Segmentation fault.
0x087150ac in sum_abs_dctelem_ssse3 (block=0xbfff8fbc) at libavcodec/x86/dsputilenc_mmx.c:833
833     DCT_SAD_FUNC(ssse3)
(gdb) where
#0  0x087150ac in sum_abs_dctelem_ssse3 (block=0xbfff8fbc) at libavcodec/x86/dsputilenc_mmx.c:833
#1  0x080ad23b in decimate_frame (inlink=0x94c8ae0, cur=0x94f8d80) at libavfilter/vf_mpdecimate.c:90
#2  filter_frame (inlink=0x94c8ae0, cur=0x94f8d80) at libavfilter/vf_mpdecimate.c:192
#3  0x08074272 in ff_filter_frame_framed (link=0x94c8ae0, frame=0x94f8d80) at libavfilter/avfilter.c:1072
#4  0x08074500 in ff_filter_frame (link=0x94c8ae0, frame=0x94f8d80) at libavfilter/avfilter.c:1147
#5  0x08078e50 in request_frame (link=0x94c8ae0) at libavfilter/buffersrc.c:491
#6  0x08079436 in av_buffersrc_add_frame_internal (ctx=0x94e1e00, frame=0x9b24000, flags=<value optimized out>) at libavfilter/buffersrc.c:170
#7  0x0807959a in av_buffersrc_add_frame_flags (ctx=0x94e1e00, frame=0x9b24000, flags=4) at libavfilter/buffersrc.c:107
#8  0x0805fd92 in decode_video (ist=0x94e2140, pkt=0xbfff9810) at ffmpeg.c:1777
#9  output_packet (ist=0x94e2140, pkt=0xbfff9810) at ffmpeg.c:1907
#10 0x08062eb8 in process_input () at ffmpeg.c:3215
#11 transcode_step () at ffmpeg.c:3311
#12 0x08065b9c in transcode () at ffmpeg.c:3363
#13 0x08066a95 in main (argc=-1850552116, argv=<value optimized out>) at ffmpeg.c:3543

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
stepping        : 13
cpu MHz         : 1200.000
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl est tm2 ssse3 cx16 xtpr lahf_lm
bogomips        : 4788.27

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
stepping        : 13
cpu MHz         : 1200.000
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl est tm2 ssse3 cx16 xtpr lahf_lm
bogomips        : 4787.95