[FFmpeg-trac] #7177(undetermined:reopened): crash when using scale with dst_format on 10 bit HEVC source file

[FFmpeg]

#7177: crash when using scale with dst_format on 10 bit HEVC source file
-------------------------------------+-------------------------------------
             Reporter:  Rollinnn     |                    Owner:
                 Type:  defect       |                   Status:  reopened
             Priority:  important    |                Component:
              Version:  git-master   |  undetermined
             Keywords:  libx264      |               Resolution:
  crash assert                       |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Changes (by cehoyos):

 * keywords:  libx264 => libx264 crash assert
 * priority:  normal => important


Comment:

 The issue is reproducible with current FFmpeg and current libx264:
 {{{
 $ ffmpeg -i x264assert.nut -vcodec libx264 -preset ultrafast -f null -
 ffmpeg version N-91983-gcd732ac Copyright (c) 2000-2018 the FFmpeg
 developers
   built with gcc 6.4.0 (GCC)
   configuration: --enable-gpl --enable-gnutls --enable-libxml2 --enable-
 libx264
   libavutil      56. 19.101 / 56. 19.101
   libavcodec     58. 30.100 / 58. 30.100
   libavformat    58. 18.101 / 58. 18.101
   libavdevice    58.  4.103 / 58.  4.103
   libavfilter     7. 32.100 /  7. 32.100
   libswscale      5.  2.100 /  5.  2.100
   libswresample   3.  2.100 /  3.  2.100
   libpostproc    55.  2.100 / 55.  2.100
 Input #0, nut, from 'x264assert.nut':
   Metadata:
     encoder         : Lavf58.18.101
   Duration: 00:00:00.00, start: 0.000000, bitrate: N/A
     Stream #0:0: Video: rawvideo (Y3[11][10] / 0xA0B3359), yuv420p10le,
 1280x534, SAR 801:800 DAR 12:5, 23.98 tbr, 48k tbn, 48k tbc (default)
     Metadata:
       BPS             : 7373812
       BPS-eng         : 7373812
       DURATION-eng    : 02:09:07.240000000
       NUMBER_OF_FRAMES: 185748
       NUMBER_OF_FRAMES-eng: 185748
       NUMBER_OF_BYTES : 7140836609
       NUMBER_OF_BYTES-eng: 7140836609
       _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
       _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
       _STATISTICS_WRITING_DATE_UTC: 2017-09-14 16:29:54
       _STATISTICS_WRITING_DATE_UTC-eng: 2017-09-14 16:29:54
       _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
       DURATION        : 00:00:06.130000000
       encoder         : Lavc58.30.100 rawvideo
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
 Press [q] to stop, [?] for help
 [libx264 @ 0x27b7c00] using SAR=801/800
 [libx264 @ 0x27b7c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
 AVX
 [libx264 @ 0x27b7c00] profile High 10, level 3.1, 4:2:0, 10-bit
 Output #0, null, to 'pipe:':
   Metadata:
     encoder         : Lavf58.18.101
     Stream #0:0: Video: h264 (libx264), yuv420p10le, 1280x534 [SAR 801:800
 DAR 12:5], q=-1--1, 23.98 fps, 23.98 tbn, 23.98 tbc (default)
     Metadata:
       BPS             : 7373812
       BPS-eng         : 7373812
       DURATION-eng    : 02:09:07.240000000
       NUMBER_OF_FRAMES: 185748
       NUMBER_OF_FRAMES-eng: 185748
       NUMBER_OF_BYTES : 7140836609
       NUMBER_OF_BYTES-eng: 7140836609
       _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
       _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
       _STATISTICS_WRITING_DATE_UTC: 2017-09-14 16:29:54
       _STATISTICS_WRITING_DATE_UTC-eng: 2017-09-14 16:29:54
       _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
       DURATION        : 00:00:06.130000000
       encoder         : Lavc58.30.100 libx264
     Side data:
       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
 ffmpeg: encoder/slicetype.c:1989: x264_10_rc_analyse_slice: Assertion
 `cost >= 0' failed.
 Aborted
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7177#comment:6>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker