October 2024 - FFmpeg-trac - lists.ffmpeg.org

#11219(avcodec:new): On the state of Vulkan video encoding on Intel and NVIDIA Hardware
by FFmpeg 08 Oct '24

08 Oct '24

#11219: On the state of Vulkan video encoding on Intel and NVIDIA Hardware -------------------------------------+------------------------------------- Reporter: Dennis E. | Type: defect Mungai | Status: new | Priority: normal Component: avcodec | Version: git- Keywords: | master hevc_vulkan,h264_vulkan,libplacebo,vulkan| Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Hello there, I have had the chance to test the Vulkan video encoding stack on the current ffmpeg git tip, on both Intel and NVIDIA platforms, as shown below. So far, basic pipelines work as expected. However, more common use cases that involve handling of Vulcan-based decoding, HDR signalling and filtering (with the likes of {{{libplacebo}}}, etc, (as advertised to be near-feature parity with VAAPI), often fail, as shown in the examples below. **Platform info:** **(a). Intel:** {{{ ffmpeg -v verbose -init_hw_device vulkan ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable- libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags ='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable- libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable- zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable- liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable- pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir /Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 [AVHWDeviceContext @ 0x589b038cdf00] Supported layers: [AVHWDeviceContext @ 0x589b038cdf00] VK_LAYER_MESA_device_select [AVHWDeviceContext @ 0x589b038cdf00] VK_LAYER_MESA_overlay [AVHWDeviceContext @ 0x589b038cdf00] Using instance extension VK_KHR_portability_enumeration [AVHWDeviceContext @ 0x589b038cdf00] GPU listing: [AVHWDeviceContext @ 0x589b038cdf00] 0: Intel(R) Graphics (RPL-P) (integrated) (0xa7a0) [AVHWDeviceContext @ 0x589b038cdf00] Device 0 selected: Intel(R) Graphics (RPL-P) (integrated) (0xa7a0) [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_push_descriptor [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_descriptor_buffer [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_physical_device_drm [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_shader_atomic_float [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_cooperative_matrix [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_maintenance1 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_external_memory_fd [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_external_memory_dma_buf [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_image_drm_format_modifier [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_external_semaphore_fd [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_external_memory_host [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_queue [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_encode_queue [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_decode_queue [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_encode_h264 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_decode_h264 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_encode_h265 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_decode_h265 [AVHWDeviceContext @ 0x589b038cdf00] Queue families: [AVHWDeviceContext @ 0x589b038cdf00] 0: graphics compute transfer sparse protected (queues: 1) [AVHWDeviceContext @ 0x589b038cdf00] 1: encode decode (queues: 2) [AVHWDeviceContext @ 0x589b038cdf00] Using device: Intel(R) Graphics (RPL-P) [AVHWDeviceContext @ 0x589b038cdf00] Alignments: [AVHWDeviceContext @ 0x589b038cdf00] optimalBufferCopyRowPitchAlignment: 128 [AVHWDeviceContext @ 0x589b038cdf00] minMemoryMapAlignment: 4096 [AVHWDeviceContext @ 0x589b038cdf00] nonCoherentAtomSize: 64 [AVHWDeviceContext @ 0x589b038cdf00] minImportedHostPointerAlignment: 4096 [AVHWDeviceContext @ 0x589b038cdf00] Using queue family 0 (queues: 1) for graphics compute transfers [AVHWDeviceContext @ 0x589b038cdf00] Using queue family 1 (queues: 2) for encode decode }}} I've tested this implementation on Intel's ANV driver. This was built against the mesa main branch, which now has the enablement bits for {{{KHR_video_maintenance1}}} on anv. See the related MR https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31436 Also, the environment variables: {{{ ANV_VIDEO_ENCODE=1 ANV_VIDEO_DECODE=1 }}} Are also set. **i. With hwaccel:** {{{ ffmpeg \ -init_hw_device vaapi=intel:/dev/dri/renderD128 \ -init_hw_device vulkan=vk@intel \ -hwaccel vulkan -hwaccel_output_format vulkan \ -hwaccel_device vk -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "libplacebo=w=1920:h=1080" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket0.ts" }}} **Output:** {{{ ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable- libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags ='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable- libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable- zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable- liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable- pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir /Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2016-01-30T17:46:24.000000Z com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : H.264 Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan)) Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac)) Press [q] to stop, [?] for help MESA-INTEL: warning: ../src/intel/vulkan/anv_formats.c:811: FINISHME: support more multi-planar formats with DRM modifiers Output #0, mpegts, to 'ticket0.ts': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 encoder : Lavf61.9.100 Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default) Metadata: encoder : Lavc61.20.100 hevc_vulkan creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default) Metadata: encoder : Lavc61.20.100 libfdk_aac creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] frame= 1 fps=0.0 q=-0.0 size= 256KiB time=00:00:00.01 bitraframe= 1 fps=1.0 q=-0.0 size= 256KiB time=00:00:00.01 bitra[hevc_vulkan @ 0x5720d9a16280] Unable to perform query: VK_ERROR_DEVICE_LOST! [hevc_vulkan @ 0x5720d9a16280] Output failed: -542398533. [vost#0:0/hevc_vulkan @ 0x5720d9a190c0] [enc:hevc_vulkan @ 0x5720d9a12c80] Error submitting video frame to the encoder [vost#0:0/hevc_vulkan @ 0x5720d9a190c0] [enc:hevc_vulkan @ 0x5720d9a12c80] Error encoding a frame: Generic error in an external library [vost#0:0/hevc_vulkan @ 0x5720d9a190c0] Task finished with error code: -542398533 (Generic error in an external library) [vost#0:0/hevc_vulkan @ 0x5720d9a190c0] Terminating thread with return code -542398533 (Generic error in an external library) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Attempting to release an unheld image? Last message repeated 1 times [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [h264 @ 0x5720d9a130c0] [vk @ 0x79643417da00] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [h264 @ 0x5720d9a130c0] hardware accelerator failed to decode picture [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) [libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105) [libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504) Last message repeated 1 times [out#0/mpegts @ 0x5720d9a46dc0] video:260KiB audio:291KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 12.472680% frame= 1 fps=0.8 q=-0.0 Lsize= 620KiB time=00:00:00.01 bitrate=304553.9kbits/s dup=8 drop=0 speed=0.0139x }}} **ii. And without H/W accelerated decoding, ie -hwaccel vulkan is disabled:** {{{ ffmpeg -v debug \ -init_hw_device vaapi=intel:/dev/dri/renderD128 \ -init_hw_device vulkan=vk@intel \ -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "hwupload,libplacebo=w=1920:h=1080:format=nv12" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -noautoscale -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket1.ts" }}} **Output:** {{{ ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable- libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags ='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable- libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable- zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable- liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable- pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir /Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2016-01-30T17:46:24.000000Z com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : H.264 Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan)) Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac)) Press [q] to stop, [?] for help MESA-INTEL: warning: ../src/intel/vulkan/anv_formats.c:811: FINISHME: support more multi-planar formats with DRM modifiers Output #0, mpegts, to 'ticket1.ts': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 encoder : Lavf61.9.100 Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default) Metadata: encoder : Lavc61.20.100 hevc_vulkan creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default) Metadata: encoder : Lavc61.20.100 libfdk_aac creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] [hevc_vulkan @ 0x56d91fd19480] Encode failed: -1073731200. [vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] [enc:hevc_vulkan @ 0x56d91fdaa140] Error submitting video frame to the encoder [vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] [enc:hevc_vulkan @ 0x56d91fdaa140] Error encoding a frame: Error number -1073731200 occurred [vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] Task finished with error code: -1073731200 (Error number -1073731200 occurred) [vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] Terminating thread with return code -1073731200 (Error number -1073731200 occurred) [out#0/mpegts @ 0x56d91fda12c0] video:0KiB audio:291KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 21.645238% frame= 0 fps=0.0 q=0.0 Lsize= 354KiB time=N/A bitrate=N/A dup=2 drop=0 speed=N/A Conversion failed! }}} Both encoding scenarios fail. Now, on to testing with NVIDIA. **(b). NVIDIA:** **Platform info:** {{{ ffmpeg -v verbose -init_hw_device vulkan ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable- libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags ='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable- libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable- zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable- liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable- pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir /Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 [AVHWDeviceContext @ 0x5fd8a3820f00] Supported layers: [AVHWDeviceContext @ 0x5fd8a3820f00] VK_LAYER_NV_optimus [AVHWDeviceContext @ 0x5fd8a3820f00] VK_LAYER_MESA_device_select [AVHWDeviceContext @ 0x5fd8a3820f00] VK_LAYER_MESA_overlay [AVHWDeviceContext @ 0x5fd8a3820f00] Using instance extension VK_KHR_portability_enumeration [AVHWDeviceContext @ 0x5fd8a3820f00] GPU listing: [AVHWDeviceContext @ 0x5fd8a3820f00] 0: NVIDIA GeForce RTX 4060 Laptop GPU (discrete) (0x28a0) [AVHWDeviceContext @ 0x5fd8a3820f00] Device 0 selected: NVIDIA GeForce RTX 4060 Laptop GPU (discrete) (0x28a0) [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_push_descriptor [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_descriptor_buffer [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_physical_device_drm [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_shader_atomic_float [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_cooperative_matrix [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_NV_optical_flow [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_shader_object [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_maintenance1 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_external_memory_fd [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_external_memory_dma_buf [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_image_drm_format_modifier [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_external_semaphore_fd [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_external_memory_host [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_queue [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_encode_queue [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_queue [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_encode_h264 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_h264 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_encode_h265 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_h265 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_av1 [AVHWDeviceContext @ 0x5fd8a3820f00] Queue families: [AVHWDeviceContext @ 0x5fd8a3820f00] 0: graphics compute transfer sparse (queues: 16) [AVHWDeviceContext @ 0x5fd8a3820f00] 1: transfer sparse (queues: 2) [AVHWDeviceContext @ 0x5fd8a3820f00] 2: compute transfer sparse (queues: 8) [AVHWDeviceContext @ 0x5fd8a3820f00] 3: transfer decode sparse (queues: 1) [AVHWDeviceContext @ 0x5fd8a3820f00] 4: transfer encode sparse (queues: 1) [AVHWDeviceContext @ 0x5fd8a3820f00] 5: transfer sparse optical_flow (queues: 1) [AVHWDeviceContext @ 0x5fd8a3820f00] Using device: NVIDIA GeForce RTX 4060 Laptop GPU [AVHWDeviceContext @ 0x5fd8a3820f00] Alignments: [AVHWDeviceContext @ 0x5fd8a3820f00] optimalBufferCopyRowPitchAlignment: 1 [AVHWDeviceContext @ 0x5fd8a3820f00] minMemoryMapAlignment: 64 [AVHWDeviceContext @ 0x5fd8a3820f00] nonCoherentAtomSize: 64 [AVHWDeviceContext @ 0x5fd8a3820f00] minImportedHostPointerAlignment: 4096 [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 0 (queues: 16) for graphics [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 2 (queues: 8) for compute [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 1 (queues: 2) for transfers [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 3 (queues: 1) for decode [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 4 (queues: 1) for encode }}} **And with Vulkan hwaccel enabled:** {{{ ffmpeg \ -init_hw_device vulkan=vk:0 \ -hwaccel vulkan -hwaccel_output_format vulkan \ -hwaccel_device vk -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "libplacebo=w=1920:h=1080" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket3.ts" }}} ****Output:**** {{{ ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable- libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags ='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable- libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable- zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable- liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable- pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir /Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2016-01-30T17:46:24.000000Z com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : H.264 Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan)) Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac)) Press [q] to stop, [?] for help Output #0, mpegts, to 'ticket3.ts': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 encoder : Lavf61.9.100 Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default) Metadata: encoder : Lavc61.20.100 hevc_vulkan creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default) Metadata: encoder : Lavc61.20.100 libfdk_aac creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] [hevc_vulkan @ 0x61c807f06040] Unable to finish command buffer: VK_ERROR_INITIALIZATION_FAILED [hevc_vulkan @ 0x61c804be8c40] Encode failed: -542398533. [vost#0:0/hevc_vulkan @ 0x61c807d90340] [enc:hevc_vulkan @ 0x61c807db2540] Error submitting video frame to the encoder [vost#0:0/hevc_vulkan @ 0x61c807d90340] [enc:hevc_vulkan @ 0x61c807db2540] Error encoding a frame: Generic error in an external library [vost#0:0/hevc_vulkan @ 0x61c807d90340] Task finished with error code: -542398533 (Generic error in an external library) [vost#0:0/hevc_vulkan @ 0x61c807d90340] Terminating thread with return code -542398533 (Generic error in an external library) [out#0/mpegts @ 0x61c807db48c0] video:0KiB audio:291KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 21.645238% frame= 0 fps=0.0 q=0.0 Lsize= 354KiB time=N/A bitrate=N/A dup=4 drop=0 speed=N/A }}} The ffmpeg process hangs indefinitely. **ii. With Vulkan H/W Accelerated decode disabled:** {{{ ffmpeg \ -init_hw_device vulkan=vk:0 \ -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "hwupload,libplacebo=w=1920:h=1080:format=nv12" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -noautoscale -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket4.ts" }}} **Output:** {{{ ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable- libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags ='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable- libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable- zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable- liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable- pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir /Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2016-01-30T17:46:24.000000Z com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : H.264 Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Metadata Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan)) Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac)) Press [q] to stop, [?] for help Output #0, mpegts, to 'ticket4.ts': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500 com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 6s com.apple.quicktime.software: 9.2.1 encoder : Lavf61.9.100 Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default) Metadata: encoder : Lavc61.20.100 hevc_vulkan creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default) Metadata: encoder : Lavc61.20.100 libfdk_aac creation_time : 2016-01-30T17:46:24.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] [hevc_vulkan @ 0x5cdf853901c0] Encode failed: -939502976. [vost#0:0/hevc_vulkan @ 0x5cdf821dd440] [enc:hevc_vulkan @ 0x5cdf821e1100] Error submitting video frame to the encoder [vost#0:0/hevc_vulkan @ 0x5cdf821dd440] [enc:hevc_vulkan @ 0x5cdf821e1100] Error encoding a frame: Error number -939502976 occurred [vost#0:0/hevc_vulkan @ 0x5cdf821dd440] Task finished with error code: -939502976 (Error number -939502976 occurred) [vost#0:0/hevc_vulkan @ 0x5cdf821dd440] Terminating thread with return code -939502976 (Error number -939502976 occurred) [out#0/mpegts @ 0x5cdf8539c500] video:0KiB audio:196KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 21.917399% frame= 0 fps=0.0 q=0.0 Lsize= 239KiB time=N/A bitrate=N/A dup=2 drop=0 speed=N/A Conversion failed! }}} Attempting to strip out HDR and a53cc with {{{-units:v -hdr-a53_cc}}} also results in the same failure states above. H.264 encoding pipelines from the media samples I've tested with also fail/die the same way, with the exception of lavfi sources like this: (a). HEVC: {{{ ffmpeg -v verbose -init_hw_device vulkan=vk:0 \ -filter_hw_device vk \ -f lavfi -i testsrc=duration=3600:size=1920x1080:rate=60,format=nv12 -vf "hwupload" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -profile:v main -r:v 60 -g:v 120 -tune:v hq -usage:v transcode -content:v rendered \ -async_depth:v 4 -refs:v 8 -b_depth:v 1 -bf:v 0 -strict -1 \ -f null /dev/null -y }}} That passes with NVIDIA and **FAILS** on Intel, and; (b). H.264/AVC: {{{ ffmpeg -v verbose -init_hw_device vulkan=vk:0 \ -filter_hw_device vk \ -f lavfi -i testsrc=duration=3600:size=1920x1080:rate=60,format=nv12 -vf "hwupload" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v h264_vulkan \ -profile:v high -r:v 60 -g:v 120 -tune:v hq -usage:v transcode -content:v rendered \ -async_depth:v 4 -refs:v 8 -b_depth:v 1 -bf:v 0 -strict -1 \ -f null /dev/null -y }}} Which passes on **BOTH** NVIDIA and Intel. The flag {{{-strict 1}}} has no impact on the results. -- Ticket URL: <https://trac.ffmpeg.org/ticket/11219> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 1

#11233(ffmpeg:new): heap-buffer-overflow occurred when running program ffmpeg in av_crc at crc.c
by FFmpeg 08 Oct '24

08 Oct '24

#11233: heap-buffer-overflow occurred when running program ffmpeg in av_crc at crc.c ------------------------------------+-------------------------------------- Reporter: Du4t | Type: defect Status: new | Priority: important Component: ffmpeg | Version: git-master Keywords: crash bugs | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+-------------------------------------- Summary of the bug: heap-buffer-overflow occurred when running program ffmpeg in av_crc at crc.c:403 How to reproduce: {{{ % git log commit 73b3344edd39468cb3f729d613949f52dbcba84e (HEAD -> master, origin/master, origin/HEAD) Author: Niklas Haas <git(a)haasn.dev> Date: Tue Oct 1 12:32:29 2024 +0200 % ffmpeg -avioflags direct -i PoC /dev/null ffmpeg version N-117413-g73b3344edd Copyright (c) 2000-2024 the FFmpeg developers built with Ubuntu clang version 14.0.0-1ubuntu1.1 configuration: --cc=clang-14 --cxx=clang++-14 --prefix=/home/du4t/target/FFmpeg/asan-release --toolchain=clang-asan --disable-shared --disable-stripping libavutil 59. 41.100 / 59. 41.100 libavcodec 61. 21.100 / 61. 21.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 [ogg @ 0x617000000080] CRC mismatch! ================================================================= ==1185968==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62d000012400 at pc 0x5acc7d21a0d5 bp 0x7fffa6cb2170 sp 0x7fffa6cb2168 READ of size 4 at 0x62d000012400 thread T0 #0 0x5acc7d21a0d4 in av_crcFFmpeg/libavutil/crc.c:403:20 #1 0x5acc7a365df6 in ff_crc04C11DB7_updateFFmpeg/libavformat/aviobuf.c:568:12 #2 0x5acc7a365ece in ffio_get_checksumFFmpeg/libavformat/aviobuf.c:585:19 #3 0x5acc7a612610 in ogg_read_pageFFmpeg/libavformat/oggdec.c:378:15 #4 0x5acc7a610e4d in ogg_packetFFmpeg/libavformat/oggdec.c:518:19 #5 0x5acc7a60e610 in ogg_read_headerFFmpeg/libavformat/oggdec.c:737:15 #6 0x5acc7a3a45c8 in avformat_open_inputFFmpeg/libavformat/demux.c:305:20 #7 0x5acc7997aee3 in ifile_openFFmpeg/fftools/ffmpeg_demux.c:1727:11 #8 0x5acc799c072f in open_filesFFmpeg/fftools/ffmpeg_opt.c:1334:15 #9 0x5acc799c072f in ffmpeg_parse_optionsFFmpeg/fftools/ffmpeg_opt.c:1374:11 #10 0x5acc799ee79f in mainFFmpeg/fftools/ffmpeg.c:974:11 #11 0x7de66c429d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #12 0x7de66c429e3f in __libc_start_main csu/../csu/libc-start.c:392:3 #13 0x5acc798af994 in _start (FFmpeg/asan-release/bin/ffmpeg+0x8bf994) (BuildId: c49a31b3f5b8cb19958f8bbbbf47bfaf1f9a9139) }}} Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker. -- Ticket URL: <https://trac.ffmpeg.org/ticket/11233> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 2

#11230(ffprobe:new): Feature request: A file header search function
by FFmpeg 06 Oct '24

06 Oct '24

#11230: Feature request: A file header search function -------------------------------------+------------------------------------- Reporter: Damole-wer | Type: | enhancement Status: new | Priority: normal Component: ffprobe | Version: git- Keywords: header | master search metadata | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Greetings, FFmpeg Team. I'm requesting this feature because, currently, ffmpeg treats files that have some data before the correct file header as invalid. Therefore it can also skip multiple files, that are embedded into a single file. == Here are two examples: **First example:** A PNG from a Godot project has **GDIM png** text before the valid **‰PNG** signature, if i try to open it with [https://github.com/zhongfly /mpv-winbuild/releases mpv-master], i get an ffmpeg error: **"Invalid PNG signature 0x4744494D03000000"**. So, if i remove the **GDIM png**, image opened successfully and no ffmpeg errors are logged. It should mean that ffprobe couldn't find the file signature because it's expected to be only at the start of the file. **Second example:** Some game stores its audio files inside a single **.wav** file for each level. So, an [https://github.com/user-attachments/files/16146001/archive.zip example] contains 3 wave audios, each starts with a **RIFF** signature. When i fully play the first audio using [https://github.com/zhongfly/mpv- winbuild/releases mpv-master], ffmpeg gives no errors and playback is finished. But that means 2 remaining audios weren't even probed. ---- Keeping examples above in mind, An implemantation of a file header search function with 4 optional command-line arguments is requested: 1. **-header-search-[file-format]**: this argument will enable search function for headers that are associated with the file format specified within the argument. Offset for each header will be recorded to a list that can be output using a **-header-search-output** command-line argument. When ffmpeg has the list, it will probe the whole file at each offset specified. Example: **-header-search-wav** will try to find every instance of **RIFF** in the whole file and record the results. From there ffmpeg can use the data as if there are multiple wave audios being input. 2. **-header-search-ext**: same as previous, but it will automatically search based on the file extension. Example: **-i landscape.jpg** with this argument will work the same as **-header-search-jpg** 3. **-header-search-all**: this argument will enable search function for all supported file headers. **The search should work like this:** there is a list of **"supported headers"** -> the function chooses the first header -> searches the whole file for every instance and appends offset to the **"found headers"** list. The process repeats for every other instance in **"supported headers"** list. This is obviously a very slow process, but it ensures that no useful data is missed and no manual searching has to be done. 4. **-header-search-output**: this argument will enable search function to output to a file. Example: **-header-search-output "path\to\output.txt"** will record search function output to the specified location. ---- In conclusion, I belive that the requested feature will provide a robust file probing functionality that ensures all useful data can be extracted from any type of file using a simple command-line argument with no manual searching needed. I hope that i made everything clear. Thank you for considering. -- Ticket URL: <https://trac.ffmpeg.org/ticket/11230> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 3

#11194(avformat:new): image2 frame_pts integer overflow
by FFmpeg 06 Oct '24

06 Oct '24

#11194: image2 frame_pts integer overflow -------------------------------------+------------------------------------- Reporter: Filip | Type: defect Status: new | Priority: normal Component: avformat | Version: git- Keywords: image2 | master frame_pts | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- **Summary of the bug:** The image2 muxer has a frame_pts option, which forwards the pts of every frame into the filename. When using microsecond precision (which is the default setting of the drawtext video filter pts template for comparison), the numbers in the filenames of the output images overflow when the timestamps go beyond 35m47s. These calculated pts numbers are being coerced into a signed int32 variable, used for sequence numbers when frame_pts is off, before being converted into text for the filename. Solutions would be to expand this variable to int64 or to reroute frame_pts to calculate the pts values as doubles and then printf them with a float format. **How to reproduce:** Substitute video.mp4 for any video longer than 35m48s. {{{ % ffmpeg -ss 35:47 -t 1 -i video.mp4 -copyts -fps_mode passthrough -enc_time_base 0.000001 -frame_pts 1 %08d.png }}} Reproduced on build 2024-09-16-git-76ff97cef5-essentials_build- www.gyan.dev -- Ticket URL: <https://trac.ffmpeg.org/ticket/11194> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 8

#11228(undetermined:new): SEGV bug at libavfilter/drawutils.c:172:27 in ff_draw_color in FFmpeg7.1
by FFmpeg 06 Oct '24

06 Oct '24

#11228: SEGV bug at libavfilter/drawutils.c:172:27 in ff_draw_color in FFmpeg7.1 -------------------------------------+------------------------------------- Reporter: | Type: defect ZengYunxiang | Status: new | Priority: important Component: | Version: 7.1 undetermined | Keywords: bugs,fuzz | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Summary of the bug: Dear developers, We found the following SEGV bug on FFmpeg(version 7.1) , please confirm. This may be due to a Segmentation violation caused by dereferencing a null pointer, which can sometimes lead to a crash. The poc file(poc24ffmpeg) will be attached to this ticket. {{{ 157 void ff_draw_color(FFDrawContext *draw, FFDrawColor *color, const uint8_t rgba[4]) 158 { 159 unsigned i; 160 double yuvad[4]; 161 double rgbad[4]; 162 const AVPixFmtDescriptor *desc = draw->desc; 163 164 if (rgba != color->rgba) 165 memcpy(color->rgba, rgba, sizeof(color->rgba)); 166 167 memset(color->comp, 0, sizeof(color->comp)); 168 169 for (int i = 0; i < 4; i++) 170 rgbad[i] = color->rgba[i] / 255.; 171 172 if (draw->desc->flags & AV_PIX_FMT_FLAG_RGB) 173 memcpy(yuvad, rgbad, sizeof(double) * 3); 174 else }}} How to reproduce: {{{ tar -xvf ffmpeg-7.1.tar.xz cd ffmpeg-7.1 ./configure --cc=clang --cxx=clang++ --ld=clang --enable-debug --toolchain =clang-asan make -j30 ./ffmpeg_g -y -i poc24ffmpeg -filter_complex pad tmp.mp4 }}} ASAN Log: {{{ AddressSanitizer:DEADLYSIGNAL ================================================================= ==4008132==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x55a0c4422a34 bp 0x7f565491aad0 sp 0x7f565491a9a0 T19) ==4008132==The signal is caused by a READ memory access. ==4008132==Hint: address points to the zero page. #0 0x55a0c4422a34 in ff_draw_color /afltest/ffmpeg-7.1/libavfilter/drawutils.c:172:27 #1 0x55a0c3f00004 in config_input /afltest/ffmpeg-7.1/libavfilter/vf_pad.c:115:5 #2 0x55a0c39887f3 in ff_filter_config_links /afltest/ffmpeg-7.1/libavfilter/avfilter.c:432:28 #3 0x55a0c39880c6 in ff_filter_config_links /afltest/ffmpeg-7.1/libavfilter/avfilter.c:365:24 #4 0x55a0c39880c6 in ff_filter_config_links /afltest/ffmpeg-7.1/libavfilter/avfilter.c:365:24 #5 0x55a0c399e407 in graph_config_links /afltest/ffmpeg-7.1/libavfilter/avfiltergraph.c:255:24 #6 0x55a0c399e407 in avfilter_graph_config /afltest/ffmpeg-7.1/libavfilter/avfiltergraph.c:1302:16 #7 0x55a0c3852d82 in configure_filtergraph /afltest/ffmpeg-7.1/fftools/ffmpeg_filter.c:1951:16 #8 0x55a0c3848fdf in send_eof /afltest/ffmpeg-7.1/fftools/ffmpeg_filter.c:2736:23 #9 0x55a0c3848fdf in filter_thread /afltest/ffmpeg-7.1/fftools/ffmpeg_filter.c:3023:19 #10 0x55a0c389e586 in task_wrapper /afltest/ffmpeg-7.1/fftools/ffmpeg_sched.c:2514:11 #11 0x7f5660d1fac2 (/lib/x86_64-linux-gnu/libc.so.6+0x94ac2) (BuildId: 490fef8403240c91833978d494d39e537409b92e) #12 0x7f5660db184f (/lib/x86_64-linux-gnu/libc.so.6+0x12684f) (BuildId: 490fef8403240c91833978d494d39e537409b92e) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /afltest/ffmpeg-7.1/libavfilter/drawutils.c:172:27 in ff_draw_color Thread T19 (fc0) created by T0 here: #0 0x55a0c37c998c in __interceptor_pthread_create (/afltest/ffmpeg-7.1/ffmpeg_g+0x99498c) (BuildId: 8ca1265ed5a8b6b91f520daf8c6156ed184d52f0) #1 0x55a0c3896853 in task_start /afltest/ffmpeg-7.1/fftools/ffmpeg_sched.c:422:11 ==4008132==ABORTING }}} ffmpeg version: {{{ # ./ffmpeg -version ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers built with Ubuntu clang version 14.0.0-1ubuntu1.1 configuration: --cc=clang --cxx=clang++ --ld=clang --enable-debug --toolchain=clang-asan libavutil 59. 39.100 / 59. 39.100 libavcodec 61. 19.100 / 61. 19.100 libavformat 61. 7.100 / 61. 7.100 libavdevice 61. 3.100 / 61. 3.100 libavfilter 10. 4.100 / 10. 4.100 libswscale 8. 3.100 / 8. 3.100 libswresample 5. 3.100 / 5. 3.100 }}} Thanks for your time! -- Ticket URL: <https://trac.ffmpeg.org/ticket/11228> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 1

#9574(undetermined:new): -profile option only accepts numeric values for aac_at encoder
by FFmpeg 05 Oct '24

05 Oct '24

#9574: -profile option only accepts numeric values for aac_at encoder -------------------------------------+------------------------------------- Reporter: Markus | Type: defect Status: new | Priority: normal Component: | Version: undetermined | unspecified Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- The definition of {{{ static const AVOption options[] = { }}} misses the line {{{ FF_AAC_PROFILE_OPTS }}} so profile names cannot be mapped to numbers. When specified as numeric values, the profiles work just fine, e.g. 4 works as aac_he and 28 as aac_he_v2 -- Ticket URL: <https://trac.ffmpeg.org/ticket/9574> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 4

#11226(ffmpeg:new): Add Track Names to OP1a MXF files
by FFmpeg 04 Oct '24

04 Oct '24

#11226: Add Track Names to OP1a MXF files -------------------------------------+------------------------------------- Reporter: Isaac T. | Type: defect Status: new | Priority: normal Component: ffmpeg | Version: Keywords: MXF mxf | unspecified avid op1a tracks metadata | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Hello ffmpeg Team, My name is Isaac. Thank you for all your work on ffmpeg, but I think I found a bug worth addressing. I am trying to add Track Names to existing OP1a MXF files with ffmpeg. For context, I want to use these OP1a MXFs in Avid Media Composer as editorial media, not for exports or deliverables. If Track Names are available and uniquely named in an OP1a MXF, those MXFs become more useful in the conform stage of post-production between Media Composer and Blackmagic Design’s DaVinci Resolve. Given an existing OP1a MXF was transcoded from something like EditReady by Hedge, this is the exact command line I used to add track names to an OP1a MXF: `~/Downloads/ffmpeg/ffmpeg -i ~/Movies/EditReady/P1810409.mxf -map_metadata 0:g -metadata material_package_name="Isaac's Material Package Name" -metadata file_package_name="Isaac's File Package Name" -metadata reel_name="Isaac's Reel Name" -metadata:s:0 track_name="Track 1" -metadata:s:1 track_name="Track 2" -c copy ~/Movies/EditReady/test.mxf` Although ffmpeg reports successfully adding a “track_name” to each stream, the resulting OP1a MXFs are unusable in Avid Media Composer. Adding the OP1a MXFs to one of Media Composer managed media folders results in an error – CM_LABEL_NOT_UNIQUE, tt:2, lnum:768 – the Bin Column labeled “Tracks” displays an inaccurate number of tracks, and the associated Clip won’t play. (See attached screenshots.) Also, using ffprobe on the resulting OP1a MXFs shows no “track_name” metadata was added to any of the streams. Attached is the full, uncut console output from “ffmpeg -v 9 -loglevel 99 -i” I used with my OP1a MXF from EditReady. A few last bits on OP1a MXFs you may already know: Attached is a screenshot from ffprobe of a usable OP1a MXF transcoded with Avid Media Composer 2024.6. Based on this screenshot, all of these metadata items will exist in an OP1a MXF when used by Avid Media Composer: - `material_package_name` & `material_package_umid` - `file_package_name` & `file_package_umid` (per stream) - `track_name` (per stream) - `reel_name` & `reel_name_umid` (per stream) - `timecode` Here are my observations on how Media Composer treats OP1a MXFs with and without certain metadata: 1. If you don't define `material_package_name`, `file_package_name`, and `reel_name`, MC successfully indexes, opens, and plays back an OP1a MXF. 2. If you define, `material_package_name` and `file_package_name`, MC successfully indexes, opens, and plays back an OP1a MXF. 3. If you define `material_package_name`, `file_package_name`, and `reel_name`, MC fails to index an OP1a MXF with that error. MC will open that Clip without crashing, but won't playback anything. Apparently, MC expects `track_name` to be explicitly defined per stream. And if `reel_name` isn’t present in an OP1a MXF, DaVinci Resolve users have to find another way to match media files when doing a conform and round-trip back to Media Composer. I know Media Composer is difficult to use, so if needed, these articles I wrote may help with testing: - Prepping Dailies for Avid Media Composer Without… Buying Media Composer? - https://isaact.micro.blog/2024/08/26/prepping-dailies-for.html - What’s the Deal With Reel Names in Avid Media Composer Clips - https://isaact.micro.blog/2024/09/10/inspired-by-some.html - In Praise of OP1a MXFs for Editorial Use in Avid Media Composer - https://isaact.micro.blog/2024/09/17/in-praise-of.html Thank you for considering! -Isaac T.- -- Ticket URL: <https://trac.ffmpeg.org/ticket/11226> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 9

#11227(undetermined:new): V-Nova appears to violate LGPL
by FFmpeg 04 Oct '24

04 Oct '24

#11227: V-Nova appears to violate LGPL -------------------------------------+------------------------------------- Reporter: vriska | Type: license | violation Status: new | Priority: normal Component: | Version: undetermined | unspecified Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- The company V-Nova advertises commercial availability of an FFmpeg build with their proprietary codecs added: https://docs.v-nova.com/v-nova/lcevc /reference-applications/ffmpeg While downloads do not appear to be available for non-customers, I feel it's a safe assumption that they do not provide source code as required by the LGPL considering the quote "Please note, LCEVC is proprietary to V-Nova and subject to V-Nova's proprietary licence. Therefore, distribution of any pre-compiled subsystem is strictly prohibited, even between group companies." -- Ticket URL: <https://trac.ffmpeg.org/ticket/11227> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 0

#10390(avfilter:new): Feature request: Add Vulkan-based deinterlace filter(s) to FFmpeg project
by FFmpeg 04 Oct '24

04 Oct '24

#10390: Feature request: Add Vulkan-based deinterlace filter(s) to FFmpeg project -------------------------------------+------------------------------------- Reporter: Dennis E. | Type: Mungai | enhancement Status: new | Priority: wish Component: avfilter | Version: Keywords: | unspecified vulkan,yadif,deinterlace | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- This is a feature request to add and enable Vulkan-based deinterlacing filters in ffmpeg. Some hardware-accelerated encoder implementations and their wrappers, such as NVENC on Turing and newer generation GPUs only support progressive encoding, which *will* implicitly break encoding workflows with *some* sources when a Vulkan-based filter pipeline is in use without a corresponding deinterlacer. Auto-inserting the {{{yadif}}} deinterlacer as a work-around to this could be sub-optimal (for performance reasons). The lack of a Vulkan-based deinterlacer would also break such Vulkan-based video encoding scenarios (when implemented) on such hardware. -- Ticket URL: <https://trac.ffmpeg.org/ticket/10390> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 4

#11224(undetermined:new): -filter_complex "[0:a] ..." does not skip disabled tracks
by FFmpeg 03 Oct '24

03 Oct '24

#11224: -filter_complex "[0:a] ..." does not skip disabled tracks -------------------------------------+------------------------------------- Reporter: markshw2222 | Type: defect Status: new | Priority: normal Component: | Version: undetermined | unspecified Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- IPhone 16 introduced spatial audio. They use multiple audio tracks to carry audio. One track is enabled the other is disabled. When specifying filter_complex "[0:a]..." the first audio track is selected, even it is disabled In this case, the disable track also have no audio codec set, so conversion fails ./ffmpeg -y -i ~/Downloads/spatial_audio_test_swapped_short.MOV -map 0:v -c:v copy -filter_complex "[0:a] asetpts=PTS-STARTPTS [output_audio]" -map '[output_audio]' -c:a aac ~/multiaudio.mp4 [aist#0:1/none @ 0x122609b40] Decoding requested, but no decoder found for: none -- Ticket URL: <https://trac.ffmpeg.org/ticket/11224> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 4