[FFmpeg-trac] #6864(undetermined:new): H.264 with multiple PPS - extradata in Matroska not complete upon remuxing

FFmpeg trac at avcodec.org
Fri Nov 24 19:59:03 EET 2017 

#6864: H.264 with multiple PPS - extradata in Matroska not complete upon remuxing
-------------------------------------+-------------------------------------
             Reporter:  mkver        |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:
  undetermined                       |  unspecified
             Keywords:  SPS, PPS     |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Hello,

 my sample is a transport stream that contains (among other tracks) an
 H.264 track with one SPS and three PPS with different pic_parameter_set_id
 values that reference the unique SPS. These SPS and PPS are repeated
 throughout the stream (in front of every keyframe). When remuxing this
 track to Matroska, only one of these PPS ends up in the CodecPrivate
 although all could fit in there (they are compatible as their ids are
 different). Given that the bitstream still contains all SPS and PPS, the
 resulting file is playable, but this is suboptimal because there are
 bitstream filters that rely on correct extradata: Given that the SPS and
 PPS don't change one could save some bytes by removing these SPS and PPS
 from the bitstream with the remove_extra bitstream filter, but the
 resulting file would be unplayable, because only one of the PPS is in the
 CodecPrivate.
 I have used the latest Zeranoe build to test this. I hope this to be
 acceptable.
 How to reproduce:
 {{{
 ffmpeg -report -i multiple.pps.ts -map 0:0 -c copy multiple.pps.mkv
 ffmpeg started on 2017-11-24 at 18:45:53
 Report written to "ffmpeg-20171124-184553.log"
 ffmpeg version N-89212-ga60b2425c3 Copyright (c) 2000-2017 the FFmpeg
 developers
   built with gcc 7.2.0 (GCC)
   configuration: --disable-static --enable-shared --enable-gpl --enable-
 version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls
 --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype
 --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-
 libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-
 libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-
 libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma
 --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis
 --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-
 dxva2 --enable-avisynth --enable-libmfx
   libavutil      56.  2.100 / 56.  2.100
   libavcodec     58.  3.105 / 58.  3.105
   libavformat    58.  2.102 / 58.  2.102
   libavdevice    58.  0.100 / 58.  0.100
   libavfilter     7.  2.102 /  7.  2.102
   libswscale      5.  0.101 /  5.  0.101
   libswresample   3.  0.101 /  3.  0.101
   libpostproc    55.  0.100 / 55.  0.100
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] SPS unavailable in decode_picture_timing
 [h264 @ 000000000057a5a0] non-existing PPS 0 referenced
 [h264 @ 000000000057a5a0] decode_slice_header error
 [h264 @ 000000000057a5a0] no frame!
 [mpegts @ 0000000000576940] PES packet size mismatch
     Last message repeated 3 times
 Input #0, mpegts, from 'multiple.pps.ts':
   Duration: 00:00:07.08, start: 45067.839433, bitrate: 11848 kb/s
   Program 10352
     Stream #0:0[0x14d3]: Video: h264 (High) ([27][0][0][0] / 0x001B),
 yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k
 tbn, 100 tbc
     Stream #0:1[0x14d4](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000
 Hz, stereo, s16p, 192 kb/s
     Stream #0:2[0x14d5](mis): Audio: mp2 ([3][0][0][0] / 0x0003), 48000
 Hz, stereo, s16p, 192 kb/s
     Stream #0:3[0x14d8](deu): Audio: ac3 ([6][0][0][0] / 0x0006), 48000
 Hz, stereo, fltp, 448 kb/s
     Stream #0:4[0x14d7](deu): Subtitle: dvb_subtitle ([6][0][0][0] /
 0x0006) (hearing impaired)
     Stream #0:5[0x14d6](deu): Subtitle: dvb_teletext ([6][0][0][0] /
 0x0006)
 Output #0, matroska, to 'multiple.pps.mkv':
   Metadata:
     encoder         : Lavf58.2.102
     Stream #0:0: Video: h264 (High) (H264 / 0x34363248),
 yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 50 fps, 50 tbr,
 1k tbn, 90k tbc
 Stream mapping:
   Stream #0:0 -> #0:0 (copy)
 Press [q] to stop, [?] for help
 frame=  257 fps=0.0 q=-1.0 Lsize=    8219kB time=00:00:05.66
 bitrate=11893.4kbits/s speed=  59x
 video:8216kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
 muxing overhead: 0.037966%
 }}}
 I also attach the output of the trace_headers bitstream-filter for the
 transport stream and the Matroska file. I want to draw your attention to
 the fact that the trace_headers output for the transport stream reports
 all three PPS as extradata; the extradata-section of the trace_headers-
 output for the Matroska file only shows one PPS as extradata (the one with
 the id 2 (which is the highest id of the three PPS)). So maybe the problem
 is in the component that converts from annex B H.264 to the MP4 format
 H.264. Maybe it is not made to put multiple different, but compatible SPS
 and PPS in the CodecPrivate.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6864>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

More information about the FFmpeg-trac mailing list