[FFmpeg-trac] #8820(avcodec:new): skip_frame nokey skips everything but IDR frames (H.264)

[FFmpeg]

#8820: skip_frame nokey skips everything but IDR frames (H.264)
-------------------------------------+-------------------------------------
             Reporter:               |                    Owner:
  skippableframe                     |
                 Type:  defect       |                   Status:  new
             Priority:  minor        |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  h264         |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------

Comment (by skippableframe):

 Built from git-master.

 I will attach another sample that was created with current x264. This
 behavior can be replicated easily with `--open-gop`. This issue was
 identified because some commercial Blu-ray releases are encoded in this
 way.

 `% x264 --vf=resize:360,150 --open-gop -o unknown.opengop.mp4
 unknown-3.mp4`

 This file has 37 key frames, per ffprobe:
 {{{
 % ffprobe -show_frames -i unknown.opengop.mp4 | grep -c key_frame=1
 ffprobe version N-98575-g16c2ed4362 Copyright (c) 2007-2020 the FFmpeg
 developers
   built with FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581)
 (based on LLVM 8.0.1)
   configuration: --cc=cc --cxx=c++
 --pkgconfigdir=/usr/local/libdata/pkgconfig --prefix=/usr/local
 --datadir=/usr/local/share/ffmpeg --mandir=/usr/local/man --enable-shared
 --enable-version3 --enable-nonfree --enable-gpl --enable-pic --enable-
 runtime-cpudetect --enable-fontconfig --enable-libaom --enable-libass
 --enable-libdav1d --enable-libdrm --enable-libfdk-aac --enable-libfreetype
 --enable-libopus --enable-libvmaf --enable-libvorbis --enable-libvpx
 --enable-libx264 --enable-libx265 --enable-libxml2
   libavutil      56. 57.100 / 56. 57.100
   libavcodec     58. 97.102 / 58. 97.102
   libavformat    58. 49.100 / 58. 49.100
   libavdevice    58. 11.101 / 58. 11.101
   libavfilter     7. 87.100 /  7. 87.100
   libswscale      5.  8.100 /  5.  8.100
   libswresample   3.  8.100 /  3.  8.100
   libpostproc    55.  8.100 / 55.  8.100
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'unknown.opengop.mp4':
   Metadata:
     major_brand     : mp42
     minor_version   : 0
     compatible_brands: mp42mp41isom
     creation_time   : 2020-07-28T05:57:08.000000Z
   Duration: 00:02:30.36, start: 0.000000, bitrate: 114 kb/s
     Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv),
 360x150 [SAR 1:1 DAR 12:5], 112 kb/s, 23.94 fps, 23.98 tbr, 11988 tbn,
 23976 tbc (default)
     Metadata:
       creation_time   : 2020-07-28T05:57:08.000000Z
       handler_name    : L-SMASH Video Media Handler
       encoder         : AVC Coding
 37
 }}}

 (And it has 45 `pict_type=I` frames.)

 But when used with `-skip_frame nokey`, only 4 key frames remain:
 {{{
 % ffprobe -v0 -skip_frame nokey -show_frames -i unknown.opengop.mp4 | grep
 -c key_frame=1
 4
 }}}

 When using `ffmpeg` with `-skip_frame nokey` to extract keyframes, only 3
 files are produced:
 {{{
 % ffmpeg -v0 -skip_frame nokey -i unknown.opengop.mp4 -vsync 0
 img-%06d.jpg; ls *jpg | wc -l
        3
 }}}

 When using `ffmpeg` with a video filter to extract I frames, 45 are
 produced (as expected):

 {{{
 % ffmpeg -v0 -i unknown.opengop.mp4 -vf
 "select=eq(pict_type\,PICT_TYPE_I)" -vsync 0 img-%06d.jpg ; ls *jpg | wc
 -l
       45
 }}}

 I understand that "What's a key frame, anyway?" isn't exactly defined.

 I think that `ffprobe -show_frames` is producing useful output. Treating I
 frames (and other recovery frames?) as key frames is useful and I think is
 the expected behavior. I'd like `-skip_frame nokey` to behave the same
 way.

 It would be understandable if `-skip_frame nokey` only used IDR frames,
 but less useful. It doesn't match the `ffprobe -show_frames` output, and
 it doesn't match the behavior of `ffmpeg` with H.265.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8820#comment:6>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker