[FFmpeg-trac] #9055(undetermined:new): HW Decoded VAAPI video gets unexpected bars when mapped to vulkan

[FFmpeg]

#9055: HW Decoded VAAPI video gets unexpected bars when mapped to vulkan
-------------------------------------+-------------------------------------
             Reporter:  aphysically  |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:  vulkan       |               Blocked By:
  vaapi                              |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 How to reproduce: You need a system capable of VAAPI hardware decoding.
 I'm using an i9-9900K (coffee lake refresh).

 Then, run the following commands using the test-img.jpg file from
 [https://stream.gably.net/images/test-img.jpg] or attached to the bug.

 We generate a test video from the image, encode it in a format that vaapi
 can hardware decode, then encode it after doing a hwmap to a noop vulkan
 filter. The video output from this final encode can be compared to the
 intermediate to show extraneous green bars were added. This bug was
 discovered when trying to do fully hardware vaapi encoding with a vulkan
 filter which was resulting in unexpected bars in videos. So far I've only
 tested this on git master. I'm attaching the full debug log of the 3
 encodes as well.

 {{{
 #!/bin/bash
 shopt -s expand_aliases
 alias ffmpeg="LD_LIBRARY_PATH=$HOME/apps/ffmpeg/build/lib
 /home/gregory/apps/ffmpeg/bin/ffmpeg"
 alias ffprobe="LD_LIBRARY_PATH=$HOME/apps/ffmpeg/build/lib
 /home/gregory/apps/ffmpeg/bin/ffprobe"

 vulkanvaapi_flags=(
   -hwaccel vaapi
   -init_hw_device vaapi=va:/dev/dri/renderD128
   -hwaccel_device va

   -hwaccel_output_format vaapi
   -init_hw_device vulkan=vulk
   -filter_hw_device vulk
 )
 vaapi_flags=(
   -init_hw_device vaapi=foo:/dev/dri/renderD128
   -filter_hw_device foo
 )

 # gen test encode
 ffmpeg -loglevel debug -stream_loop 1 -i test-img.jpg -t 5 -vf
 "pad=ceil(iw/2)*2:ceil(ih/2)*2,format=yuv420p" -c:v libx264 -pix_fmt
 yuv420p -crf 0 -y vtest.nut

 # put in hw-decodable format
 # no black bars
 LIBVA_DRIVER_NAME=i965 ffmpeg -loglevel debug -hide_banner
 "${vaapi_flags[@]}" -i vtest.nut -c:v h264_vaapi -r:v 30 -profile:v high
 -qp:v 18 -bf:v 2 -vf format=nv12,hwupload -map 0 -y vaapiencode.mkv

 # test, sw encode, hw decode, has green bars
 LIBVA_DRIVER_NAME=i965 ffmpeg -loglevel debug -hide_banner
 "${vulkanvaapi_flags[@]}" -i vaapiencode.mkv -c:v libx264 -r:v 30
 -profile:v high -preset veryfast -vf
 "hwmap,chromaber_vulkan=0:0,hwdownload,format=nv12" -map 0 -y
 vaapiouts.mkv
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/9055>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker