[FFmpeg-trac] #11131(ffmpeg:open): Nvenc b_ref_mode middle does not write max_num_reorder_frames == 2 in SPS
FFmpeg
trac at avcodec.org
Thu Aug 8 01:55:33 EEST 2024
#11131: Nvenc b_ref_mode middle does not write max_num_reorder_frames == 2 in SPS
-------------------------------------+-----------------------------------
Reporter: Balling | Owner: Timo R.
Type: defect | Status: open
Priority: minor | Component: ffmpeg
Version: git-master | Resolution:
Keywords: cuda, nvenc | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-----------------------------------
Description changed by Balling:
Old description:
> Summary of the bug: see #6810, we can just add
> vui->bitstreamRestrictionFlag = 1; to ffmpeg code to write
> bitstream_restriction_flag 1 in SPS and that will allow Nvidia driver to
> set max_num_reorder_frames to 2, which is needed for -b_ref_mode middle
> and max_num_reorder_frames to 1 which is needed for b_ref_mode each.
>
> Also see https://forums.developer.nvidia.com/t/sei-nal-unit-resend-each-
> packet-on-h264/145653/3
>
> And people crying about it in
> https://github.com/bluenviron/mediamtx/issues/989#issuecomment-1163015224
>
> How to reproduce:
> {{{
> PS > ffmpeg.exe -lavfi "ddagrab=framerate=240" -c:v h264_nvenc
> -b_ref_mode middle ddagrab.h264
> ffmpeg version N-116525-gde7f999481-20240803 Copyright (c) 2000-2024 the
> FFmpeg developers
> built with gcc 14.1.0 (crosstool-NG 1.26.0.93_a87bf7f)
> configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static
> --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64
> --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug
> --enable-shared --disable-static --disable-w32threads --enable-pthreads
> --enable-iconv --enable-zlib --enable-libfreetype --enable-libfribidi
> --enable-gmp --enable-libxml2 --enable-lzma --enable-fontconfig --enable-
> libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse
> --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-
> libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint
> --enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-
> libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm
> --enable-frei0r --enable-libgme --enable-libkvazaar --enable-
> libaribcaption --enable-libass --enable-libbluray --enable-libjxl
> --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh
> --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2
> --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-
> libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-
> libopenmpt --enable-librav1e --enable-librubberband --enable-schannel
> --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1
> --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi
> --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-
> libplacebo --enable-libvvenc --enable-libx264 --enable-libx265 --enable-
> libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-
> cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-libs=-lgomp --extra-
> ldflags=-pthread --extra-ldexeflags= --cc=x86_64-w64-mingw32-gcc
> --cxx=x86_64-w64-mingw32-g++ --ar=x86_64-w64-mingw32-gcc-ar
> --ranlib=x86_64-w64-mingw32-gcc-ranlib --nm=x86_64-w64-mingw32-gcc-nm
> --extra-version=20240803
> libavutil 59. 31.100 / 59. 31.100
> libavcodec 61. 11.100 / 61. 11.100
> libavformat 61. 5.101 / 61. 5.101
> libavdevice 61. 2.100 / 61. 2.100
> libavfilter 10. 2.102 / 10. 2.102
> libswscale 8. 2.100 / 8. 2.100
> libswresample 5. 2.100 / 5. 2.100
> libpostproc 58. 2.100 / 58. 2.100
> Stream mapping:
> ddagrab:default -> Stream #0:0 (h264_nvenc)
> Press [q] to stop, [?] for help
> Output #0, h264, to 'ddagrab.h264':
> Metadata:
> encoder : Lavf61.5.101
> Stream #0:0: Video: h264 (Main), d3d11(pc, gbr/bt709/iec61966-2-1,
> progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 240 fps,
> 240 tbn
> Metadata:
> encoder : Lavc61.11.100 h264_nvenc
> Side data:
> cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000
> vbv_delay: N/A
> [out#0/h264 @ 00000243bca73e80] video:1981KiB audio:0KiB subtitle:0KiB
> other streams:0KiB global headers:0KiB muxing overhead: 0.000000%
> frame= 951 fps=113 q=42.0 Lsize= 1981KiB time=00:00:03.95
> bitrate=4107.6kbits/s dup=945 drop=0 speed=0.468x
> Exiting normally, received signal 2.
> }}}
>
> And then use VQAnalyzer that it has wrong SPS. AS you can see because
> flag is not set the motion_vectors_over_pic_boundaries_flag and
> og2_max_mv_length_horizontal and max_num_reorder_frames are not
> populated.
>
> Patches should be submitted to the ffmpeg-devel mailing list and not this
> bug tracker.
New description:
Summary of the bug: see #6810, we can just add
vui->bitstreamRestrictionFlag = 1; to ffmpeg code to write
bitstream_restriction_flag 1 in SPS and that will allow Nvidia driver to
set max_num_reorder_frames to 2, which is needed for -b_ref_mode middle
and max_num_reorder_frames to 1 which is needed for b_ref_mode each.
Also see https://forums.developer.nvidia.com/t/sei-nal-unit-resend-each-
packet-on-h264/145653/3
And people crying about it in
https://github.com/bluenviron/mediamtx/issues/989#issuecomment-1163015224
How to reproduce:
{{{
PS > ffmpeg.exe -lavfi "ddagrab=framerate=240" -c:v h264_nvenc -b_ref_mode
middle ddagrab.h264
ffmpeg version N-116525-gde7f999481-20240803 Copyright (c) 2000-2024 the
FFmpeg developers
built with gcc 14.1.0 (crosstool-NG 1.26.0.93_a87bf7f)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static
--pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64
--target-os=mingw32 --enable-gpl --enable-version3 --disable-debug
--enable-shared --disable-static --disable-w32threads --enable-pthreads
--enable-iconv --enable-zlib --enable-libfreetype --enable-libfribidi
--enable-gmp --enable-libxml2 --enable-lzma --enable-fontconfig --enable-
libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse
--enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-
libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint
--enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-libdvdnav
--disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r
--enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-
libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-
libopus --enable-librist --enable-libssh --enable-libtheora --enable-
libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264
--enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-
librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-
libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d
--disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan
--enable-libshaderc --enable-libplacebo --enable-libvvenc --enable-libx264
--enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg
--enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags=
--extra-libs=-lgomp --extra-ldflags=-pthread --extra-ldexeflags=
--cc=x86_64-w64-mingw32-gcc --cxx=x86_64-w64-mingw32-g++
--ar=x86_64-w64-mingw32-gcc-ar --ranlib=x86_64-w64-mingw32-gcc-ranlib
--nm=x86_64-w64-mingw32-gcc-nm --extra-version=20240803
libavutil 59. 31.100 / 59. 31.100
libavcodec 61. 11.100 / 61. 11.100
libavformat 61. 5.101 / 61. 5.101
libavdevice 61. 2.100 / 61. 2.100
libavfilter 10. 2.102 / 10. 2.102
libswscale 8. 2.100 / 8. 2.100
libswresample 5. 2.100 / 5. 2.100
libpostproc 58. 2.100 / 58. 2.100
Stream mapping:
ddagrab:default -> Stream #0:0 (h264_nvenc)
Press [q] to stop, [?] for help
Output #0, h264, to 'ddagrab.h264':
Metadata:
encoder : Lavf61.5.101
Stream #0:0: Video: h264 (Main), d3d11(pc, gbr/bt709/iec61966-2-1,
progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 240 fps,
240 tbn
Metadata:
encoder : Lavc61.11.100 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000
vbv_delay: N/A
[out#0/h264 @ 00000243bca73e80] video:1981KiB audio:0KiB subtitle:0KiB
other streams:0KiB global headers:0KiB muxing overhead: 0.000000%
frame= 951 fps=113 q=42.0 Lsize= 1981KiB time=00:00:03.95
bitrate=4107.6kbits/s dup=945 drop=0 speed=0.468x
Exiting normally, received signal 2.
}}}
And then using VQAnalyzer you see that Annex B has wrong SPS. As you can
see because flag is not set the motion_vectors_over_pic_boundaries_flag
and log2_max_mv_length_horizontal and **max_num_reorder_frames** are not
populated.
Patches should be submitted to the ffmpeg-devel mailing list and not this
bug tracker.
--
--
Ticket URL: <https://trac.ffmpeg.org/ticket/11131#comment:4>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list