[FFmpeg-trac] #11143(undetermined:new): Vulkan filters fail when engaging NVDEC / NVENC

[FFmpeg]

#11143: Vulkan filters fail when engaging NVDEC / NVENC
-------------------------------------+-------------------------------------
             Reporter:  rnbwdrk      |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:  vulkan       |               Blocked By:
  NVDEC NVENC cuda                   |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:

 Vulkan filters fail when engaging NVDEC / NVENC.

 How to reproduce:

 {{{

 The following works fine:

 ffmpeg -init_hw_device cuda=cu:0 -init_hw_device vulkan=vk at cu -i
 IMG_8057.MOV -vf:v
 "hwupload=derive_device=vulkan,libplacebo=tonemapping=auto:colorspace=bt709:color_primaries=bt709:color_trc=bt709:format=yuv420p:upscaler=none:downscaler=none:peak_detect=0,hwupload=derive_device=cuda,scale_cuda=format=yuv420p"
 -f null -

 Attempting to add NVDEC / NVENC into the mix fails:

 ffmpeg -init_hw_device cuda=cu:0 -init_hw_device vulkan=vk at cu -hwaccel
 cuda -hwaccel_output_format cuda -i IMG_8057.MOV -vf:v
 "hwupload=derive_device=vulkan,libplacebo=tonemapping=auto:colorspace=bt709:color_primaries=bt709:color_trc=bt709:format=yuv420p:upscaler=none:downscaler=none:peak_detect=0,hwupload=derive_device=cuda,scale_cuda=format=yuv420p"
 -f null -

 BtbN's latest git master (2024-08-16 13:46) Win64 GPL build was used for
 this test in
 Windows 10 22H2, although it has has been extensively and exhaustively
 reproduced with
 various compiles and builds of ffmpeg 7.0, 7.0.1, and 7.0.2 in windows and
 in linux.
 Different versions of the Vulkan and Cuda SDK, NVIDIA Drivers, libplacebo,
 etc were also
 tried. Hardware-wise, the bug has been reproduced on both a NVIDIA A2000
 and an RTX 2080 TI.

 AFAIK the only time this has ever worked was with ffmpeg ~5.1.3 via
 a patch to libavutil/hwcontext_vulkan.c and a very specific version of
 libplacebo,
 as described here:

 https://github.com/jellyfin/jellyfin-ffmpeg/issues/215

 All attempted input files have been NVDEC compatible so the issue is not
 the file. In fact,
 everything works fine with the aforementioned jellyfin fix and ffmpeg
 5.1.3.

 "summary" output (ffmpeg_output_summary.txt)

 and

 "-v 9 -loglevel 99" output (ffmpeg_output.txt)

 and

 "-report" output (ffmpeg-20240816-130347.log)

 Have been attached with metadata redacted.

 }}}
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/11143>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker