[FFmpeg-trac] #8512(undetermined:new): The Vulkan based filter scale_vulkan does not work on NVIDIA hardware.

[FFmpeg]

#8512: The Vulkan based filter scale_vulkan does not work on NVIDIA hardware.
-------------------------------------+-------------------------------------
             Reporter:  Brainiarc7   |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:
                                     |  undetermined
              Version:  git-master   |               Resolution:
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------

Comment (by Brainiarc7):

 Hello @richardpl ,

 Your advice above works very well.


 Here are two samples that work, one with NVDEC and the other without:

 (a). With nvdec:

 {{{
    ffmpeg -threads 1 -loglevel info -nostdin -y \
    -fflags +genpts-fastseek \
    -init_hw_device vulkan=gpu:0.0 -filter_hw_device gpu \
    -hwaccel nvdec -hwaccel_device 0 -extra_hw_frames 2 \
    -reinit_filter 1 -vsync 1 -async 1 -filter_threads 2
 -filter_complex_threads 2 \
    -i feeds.mp4 -filter_complex \
   "[0:v]format=nv12,hwupload,split=2[s0][s1]; \
    [s0]scale_vulkan=w=1920:h=1080:scaler=0,hwdownload,format=nv12[v0]; \
    [s1]scale_vulkan=w=1280:h=720:scaler=0,hwdownload,format=nv12[v1]" \
   -map "[v0]" -b:v:0 5800k -minrate:v:0 5800k -maxrate:v:0 5800k
 -bufsize:v:0 5800k -c:v:0 h264_nvenc -r:v:0 ntsc \
   -profile:v:0 high -preset:v:0 llhp -rc:v:0 cbr_ld_hq -g:v:0 60 -gpu:v:0
 0 -strict_gop:v:0 1 -bf:v:0 0 \
   -map "[v1]"  -b:v:1 4000k -minrate:v:1 4000k -maxrate:v:1 4000k
 -bufsize:v:1 4000k -c:v:1 h264_nvenc -r:v:1 ntsc \
   -profile:v:1 high -preset:v:1 llhp -rc:v:1 cbr_ld_hq -g:v:1 60 -gpu:v:1
 0 -strict_gop:v:1 1 -bf:v:1 0 \
   -map 0:a -c:a libfdk_aac -ac 2 -ar 48000 -b:a 128k \
   -flags +global_header+cgop \
   -max_muxing_queue_size 9000000 -threads 2 -f tee  \
   "[select=\'v:0,a\':f=mp4]'hq.mp4'| \
    [select=\'v:1,a\':f=mp4]'med.mp4'"
 }}}

 (b). No hwaccel decode:

 {{{
    ffmpeg -threads 1 -loglevel info -nostdin -y \
    -fflags +genpts-fastseek \
    -init_hw_device vulkan=gpu:0.0 -filter_hw_device gpu \
    -reinit_filter 1 -vsync 1 -async 1 -filter_threads 2
 -filter_complex_threads 2 \
    -i feeds.mp4 -filter_complex \
   "[0:v]format=nv12,hwupload,split=2[s0][s1]; \
    [s0]scale_vulkan=w=1920:h=1080:scaler=0,hwdownload,format=nv12[v0]; \
    [s1]scale_vulkan=w=1280:h=720:scaler=0,hwdownload,format=nv12[v1]" \
   -map "[v0]" -b:v:0 5800k -minrate:v:0 5800k -maxrate:v:0 5800k
 -bufsize:v:0 5800k -c:v:0 h264_nvenc -r:v:0 ntsc \
   -profile:v:0 high -preset:v:0 llhp -rc:v:0 cbr_ld_hq -g:v:0 60 -gpu:v:0
 0 -strict_gop:v:0 1 -bf:v:0 0 \
   -map "[v1]"  -b:v:1 4000k -minrate:v:1 4000k -maxrate:v:1 4000k
 -bufsize:v:1 4000k -c:v:1 h264_nvenc -r:v:1 ntsc \
   -profile:v:1 high -preset:v:1 llhp -rc:v:1 cbr_ld_hq -g:v:1 60 -gpu:v:1
 0 -strict_gop:v:1 1 -bf:v:1 0 \
   -map 0:a -c:a libfdk_aac -ac 2 -ar 48000 -b:a 128k \
   -flags +global_header+cgop \
   -max_muxing_queue_size 9000000 -threads 2 -f tee  \
   "[select=\'v:0,a\':f=mp4]'hq.mp4'| \
    [select=\'v:1,a\':f=mp4]'med.mp4'"
 }}}

 See the repeated use of the hwdownload filters trailing each scale_vulcan
 instance.

 Where it all falls apart is if you attempt device derivation via hwupload.

 The example above with NVDEC will fail if you attempt device derivation in
 hwupload so as to re-use the same context for hwaccel via nvdec:

 {{{

    ffmpeg -threads 1 -loglevel info -nostdin -y \
    -fflags +genpts-fastseek \
    -init_hw_device vulkan=gpu:0.0 -filter_hw_device gpu \
    -hwaccel nvdec -hwaccel_device cuda -extra_hw_frames 2 \
    -reinit_filter 1 -vsync 1 -async 1 -filter_threads 2
 -filter_complex_threads 2 \
    -i feeds.mp4 -filter_complex \
   "[0:v]format=nv12,hwupload=derive_device=cuda,split=2[s0][s1]; \
    [s0]scale_vulkan=w=1920:h=1080:scaler=0,hwdownload,format=nv12[v0]; \
    [s1]scale_vulkan=w=1280:h=720:scaler=0,hwdownload,format=nv12[v1]" \
   -map "[v0]" -b:v:0 5800k -minrate:v:0 5800k -maxrate:v:0 5800k
 -bufsize:v:0 5800k -c:v:0 h264_nvenc -r:v:0 ntsc \
   -profile:v:0 high -preset:v:0 llhp -rc:v:0 cbr_ld_hq -g:v:0 60 -gpu:v:0
 0 -strict_gop:v:0 1 -bf:v:0 0 \
   -map "[v1]"  -b:v:1 4000k -minrate:v:1 4000k -maxrate:v:1 4000k
 -bufsize:v:1 4000k -c:v:1 h264_nvenc -r:v:1 ntsc \
   -profile:v:1 high -preset:v:1 llhp -rc:v:1 cbr_ld_hq -g:v:1 60 -gpu:v:1
 0 -strict_gop:v:1 1 -bf:v:1 0 \
   -map 0:a -c:a libfdk_aac -ac 2 -ar 48000 -b:a 128k \
   -flags +global_header+cgop \
   -max_muxing_queue_size 9000000 -threads 2 -f tee  \
   "[select=\'v:0,a\':f=mp4]'hq.mp4'| \
    [select=\'v:1,a\':f=mp4]'med.mp4'"

 }}}

 Error message:

 {{{
 Impossible to convert between the formats supported by the filter
 'Parsed_split_2' and the filter 'auto_scaler_0'
 Error reinitializing filters!
 Failed to inject frame into filter network: Function not implemented
 Error while processing the decoded data for stream #0:0
 Conversion failed!
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8512#comment:2>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker