[FFmpeg-trac] #6450(avcodec:new): segfault / x265 crash with duplicated frames

FFmpeg trac at avcodec.org
Thu Jun 8 16:58:05 EEST 2017


#6450: segfault / x265 crash with duplicated frames
---------------------------------+--------------------------------------
             Reporter:  slhck    |                     Type:  defect
               Status:  new      |                 Priority:  normal
            Component:  avcodec  |                  Version:  git-master
             Keywords:  x265     |               Blocked By:
             Blocking:           |  Reproduced by developer:  0
Analyzed by developer:  0        |
---------------------------------+--------------------------------------
 I can make ffmpeg/x265 crash reliably using one particular source file and
 two-pass encoding.

 Source file available here: http://dev.slhck.info/download/sintel_02.avi

 {{{
 ffmpeg -y -i sintel_02.avi -t 10 -c:v libx265 -x265-params vbv-
 maxrate=30000:vbv-bufsize=30000:pass=1 -f null /dev/null && \
 ffmpeg -y -i sintel_02.avi -t 10 -c:v libx265 -x265-params vbv-
 maxrate=30000:vbv-bufsize=30000:pass=2 out.mp4
 }}}

 Output from ffmpeg 3.2.2 with x265 2.2:

 {{{
 ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
   built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
   configuration: --prefix=/home/ubuntu/ffmpeg_build --pkg-config-
 flags=--static --extra-cflags=-I/home/ubuntu/ffmpeg_build/include --extra-
 ldflags=-L/home/ubuntu/ffmpeg_build/lib --bindir=/home/ubuntu/bin
 --enable-gpl --enable-libass --enable-libfreetype --enable-libfdk-aac
 --enable-libx264 --enable-libx265 --enable-libvpx --enable-nonfree
   libavutil      55. 34.100 / 55. 34.100
   libavcodec     57. 64.101 / 57. 64.101
   libavformat    57. 56.100 / 57. 56.100
   libavdevice    57.  1.100 / 57.  1.100
   libavfilter     6. 65.100 /  6. 65.100
   libswscale      4.  2.100 /  4.  2.100
   libswresample   2.  3.100 /  2.  3.100
   libpostproc    54.  1.100 / 54.  1.100
 Input #0, avi, from '/databases/sintel_02.avi':
   Metadata:
     encoder         : Lavf56.34.100
   Duration: 00:00:10.00, start: 0.000000, bitrate: 617112 kb/s
     Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080,
 619581 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
 x265 [info]: HEVC encoder version 2.2
 x265 [info]: build info [Linux][GCC 5.4.0][64 bit] 8bit
 x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT
 x265 [info]: Main profile, Level-4 (High tier)
 x265 [info]: Thread pool created using 2 threads
 x265 [info]: Slices                              : 1
 x265 [info]: frame threads / pool features       : 1 / wpp(17 rows)
 x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
 x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
 x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
 x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
 x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
 x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
 x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
 x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
 x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
 x265 [info]: VBV/HRD buffer / max-rate / init    : 30000 / 30000 / 0.900
 x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp strong-intra-
 smoothing
 x265 [info]: tools: lslices=6 deblock sao stats-write
 Output #0, null, to '/dev/null':
   Metadata:
     encoder         : Lavf57.56.100
     Stream #0:0: Video: hevc (libx265), yuv420p, 1920x1080, q=2-31, 25
 fps, 25 tbn, 25 tbc
     Metadata:
       encoder         : Lavc57.64.101 libx265
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (libx265))
 Press [q] to stop, [?] for help
 frame=  248 fps=7.3 q=-0.0 Lsize=N/A time=00:00:09.92 bitrate=N/A
 speed=0.293x
 video:1508kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
 muxing overhead: unknown
 x265 [info]: frame I:      4, Avg QP:26.96  kb/s: 7693.55
 x265 [info]: frame P:     70, Avg QP:29.91  kb/s: 2690.72
 x265 [info]: frame B:    174, Avg QP:34.85  kb/s: 508.29
 x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
 x265 [info]: consecutive B-frames: 14.9% 4.1% 20.3% 52.7% 8.1%
 encoded 248 frames in 33.88s (7.32 fps), 1240.19 kb/s, Avg QP:33.33

 ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
   built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
   configuration: --prefix=/home/ubuntu/ffmpeg_build --pkg-config-
 flags=--static --extra-cflags=-I/home/ubuntu/ffmpeg_build/include --extra-
 ldflags=-L/home/ubuntu/ffmpeg_build/lib --bindir=/home/ubuntu/bin
 --enable-gpl --enable-libass --enable-libfreetype --enable-libfdk-aac
 --enable-libx264 --enable-libx265 --enable-libvpx --enable-nonfree
   libavutil      55. 34.100 / 55. 34.100
   libavcodec     57. 64.101 / 57. 64.101
   libavformat    57. 56.100 / 57. 56.100
   libavdevice    57.  1.100 / 57.  1.100
   libavfilter     6. 65.100 /  6. 65.100
   libswscale      4.  2.100 /  4.  2.100
   libswresample   2.  3.100 /  2.  3.100
   libpostproc    54.  1.100 / 54.  1.100
 Input #0, avi, from '/databases/sintel_02.avi':
   Metadata:
     encoder         : Lavf56.34.100
   Duration: 00:00:10.00, start: 0.000000, bitrate: 617112 kb/s
     Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080,
 619581 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
 x265 [info]: HEVC encoder version 2.2
 x265 [info]: build info [Linux][GCC 5.4.0][64 bit] 8bit
 x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT
 x265 [info]: Main profile, Level-4 (High tier)
 x265 [info]: Thread pool created using 2 threads
 x265 [info]: Slices                              : 1
 x265 [info]: frame threads / pool features       : 1 / wpp(17 rows)
 x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
 x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
 x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
 x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
 x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
 x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
 x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
 x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
 x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
 x265 [info]: VBV/HRD buffer / max-rate / init    : 30000 / 30000 / 0.900
 x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp strong-intra-
 smoothing
 x265 [info]: tools: lslices=6 deblock sao stats-read
 Output #0, mp4, to 'out.mp4':
   Metadata:
     encoder         : Lavf57.56.100
     Stream #0:0: Video: hevc (libx265) ([35][0][0][0] / 0x0023), yuv420p,
 1920x1080, q=2-31, 25 fps, 12800 tbn, 25 tbc
     Metadata:
       encoder         : Lavc57.64.101 libx265
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (libx265))
 Press [q] to stop, [?] for help
 x265 [error]: Incomplete CU-tree stats file.ime=00:00:08.68
 bitrate=1572.4kbits/s dup=2 drop=0 speed=0.315x
 Video encoding failed
 x265 [info]: frame I:      4, Avg QP:26.80  kb/s: 12299.20
 x265 [info]: frame P:     64, Avg QP:29.68  kb/s: 3124.61
 x265 [info]: frame B:    153, Avg QP:34.55  kb/s: 603.77
 x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
 x265 [info]: consecutive B-frames: 15.7% 4.3% 21.4% 52.9% 5.7%
 encoded 221 frames in 27.87s (7.93 fps), 1545.47 kb/s, Avg QP:33.00
 Conversion failed!
 }}}

 Output with a newer ffmpeg build and x265 version 2.4+37-e75d5f5eeae3:

 {{{
 ffmpeg version N-86395-g497a4b554c-static http://johnvansickle.com/ffmpeg/
 Copyright (c) 2000-2017 the FFmpeg developers
   built with gcc 5.4.1 (Debian 5.4.1-11) 20170519
   configuration: --enable-gpl --enable-version3 --enable-static --disable-
 debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio
 --cc=gcc-5 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-
 gray --enable-libass --enable-libfreetype --enable-libfribidi --enable-
 libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-
 libopenjpeg --enable-libopus --enable-librtmp --enable-libsoxr --enable-
 libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc
 --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264
 --enable-libx265 --enable-libxvid --enable-libzimg
   libavutil      55. 63.100 / 55. 63.100
   libavcodec     57. 96.101 / 57. 96.101
   libavformat    57. 72.101 / 57. 72.101
   libavdevice    57.  7.100 / 57.  7.100
   libavfilter     6. 91.100 /  6. 91.100
   libswscale      4.  7.101 /  4.  7.101
   libswresample   2.  8.100 /  2.  8.100
   libpostproc    54.  6.100 / 54.  6.100
 Input #0, avi, from 'sintel_02.avi':
   Metadata:
     encoder         : Lavf56.34.100
   Duration: 00:00:10.00, start: 0.000000, bitrate: 617112 kb/s
     Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080,
 619581 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (libx265))
 Press [q] to stop, [?] for help
 x265 [info]: HEVC encoder version 2.4+37-e75d5f5eeae3
 x265 [info]: build info [Linux][GCC 5.4.1][64 bit] 8bit+10bit+12bit
 x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2
 FMA3 LZCNT BMI2
 x265 [info]: Main profile, Level-4 (High tier)
 x265 [info]: Thread pool created using 8 threads
 x265 [info]: Slices                              : 1
 x265 [info]: frame threads / pool features       : 3 / wpp(17 rows)
 x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
 x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
 x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
 x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
 x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
 x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
 x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
 x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
 x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
 x265 [info]: VBV/HRD buffer / max-rate / init    : 30000 / 30000 / 0.900
 x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp strong-intra-
 smoothing
 x265 [info]: tools: lslices=6 deblock sao stats-write
 Output #0, null, to '/dev/null':
   Metadata:
     encoder         : Lavf57.72.101
     Stream #0:0: Video: hevc (libx265), yuv420p, 1920x1080, q=2-31, 25
 fps, 25 tbn, 25 tbc
     Metadata:
       encoder         : Lavc57.96.101 libx265
 frame=  248 fps= 21 q=-0.0 Lsize=N/A time=00:00:09.92 bitrate=N/A
 speed=0.826x
 video:1594kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
 muxing overhead: unknown
 x265 [info]: frame I:      4, Avg QP:26.63  kb/s: 7903.85
 x265 [info]: frame P:     64, Avg QP:29.79  kb/s: 3011.51
 x265 [info]: frame B:    180, Avg QP:34.81  kb/s: 559.52
 x265 [info]: Weighted P-Frames: Y:1.6% UV:0.0%
 x265 [info]: consecutive B-frames: 8.8% 4.4% 10.3% 66.2% 10.3%
 encoded 248 frames in 11.99s (20.68 fps), 1310.74 kb/s, Avg QP:33.38

 ffmpeg version N-86395-g497a4b554c-static http://johnvansickle.com/ffmpeg/
 Copyright (c) 2000-2017 the FFmpeg developers
   built with gcc 5.4.1 (Debian 5.4.1-11) 20170519
   configuration: --enable-gpl --enable-version3 --enable-static --disable-
 debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio
 --cc=gcc-5 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-
 gray --enable-libass --enable-libfreetype --enable-libfribidi --enable-
 libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-
 libopenjpeg --enable-libopus --enable-librtmp --enable-libsoxr --enable-
 libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc
 --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264
 --enable-libx265 --enable-libxvid --enable-libzimg
   libavutil      55. 63.100 / 55. 63.100
   libavcodec     57. 96.101 / 57. 96.101
   libavformat    57. 72.101 / 57. 72.101
   libavdevice    57.  7.100 / 57.  7.100
   libavfilter     6. 91.100 /  6. 91.100
   libswscale      4.  7.101 /  4.  7.101
   libswresample   2.  8.100 /  2.  8.100
   libpostproc    54.  6.100 / 54.  6.100
 Input #0, avi, from 'sintel_02.avi':
   Metadata:
     encoder         : Lavf56.34.100
   Duration: 00:00:10.00, start: 0.000000, bitrate: 617112 kb/s
     Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080,
 619581 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (libx265))
 Press [q] to stop, [?] for help
 x265 [info]: HEVC encoder version 2.4+37-e75d5f5eeae3
 x265 [info]: build info [Linux][GCC 5.4.1][64 bit] 8bit+10bit+12bit
 x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2
 FMA3 LZCNT BMI2
 x265 [info]: Main profile, Level-4 (High tier)
 x265 [info]: Thread pool created using 8 threads
 x265 [info]: Slices                              : 1
 x265 [info]: frame threads / pool features       : 3 / wpp(17 rows)
 x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
 x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
 x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
 x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
 x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
 x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
 x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
 x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
 x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
 x265 [info]: VBV/HRD buffer / max-rate / init    : 30000 / 30000 / 0.900
 x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp strong-intra-
 smoothing
 x265 [info]: tools: lslices=6 deblock sao stats-read
 Output #0, mp4, to 'out.mp4':
   Metadata:
     encoder         : Lavf57.72.101
     Stream #0:0: Video: hevc (libx265) ([35][0][0][0] / 0x0023), yuv420p,
 1920x1080, q=2-31, 25 fps, 12800 tbn, 25 tbc
     Metadata:
       encoder         : Lavc57.96.101 libx265
 [1]    20242 segmentation fault (core dumped)  /usr/local/bin/ffmpeg -y -i
 sintel_02.avi -t 10 -c:v libx265 -x265-params
 }}}

 Interestingly, now I only get a segfault, while with the older version, I
 got an error message from x265.

 It seems that the problem is that frames are duplicated by ffmpeg. The
 decimate filter would drop frames 2 and 3. The source file has 250 frames
 but x265 only encodes 248.

 When I encode directly with x265, it works fine and encodes all 250
 frames:

 {{{
 ffmpeg -i sintel_02.avi -t 10 -f yuv4mpegpipe - | ~/ffmpeg_build/bin/x265
 --y4m - --vbv-maxrate=30000 --vbv-bufsize=30000 --pass=1 -o /dev/null
 ffmpeg -i sintel_02.avi -t 10 -f yuv4mpegpipe - | ~/ffmpeg_build/bin/x265
 --y4m - --vbv-maxrate=30000 --vbv-bufsize=30000 --pass=2 -o out.265
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6450>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list