Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
I'm testing out an Intel Flex 140 card and I am attempting to transcode 4 inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so: /tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]" \ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bits=10528" The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out: [h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to the encoder Error while filtering: Internal bug, should not have happened I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command: /tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bits=10528" When that stream changes resolution I see these come out in the log, and the stream carries on without dying: [h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to transcode 4 inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]" \ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@ 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bits=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to the encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@ 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@ 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bits=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one.
I have created a bug and linked a sample video that changes resolutions on the fly: https://trac.ffmpeg.org/ticket/10762 Get back to me if you need anything else I can help with. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Wednesday, December 27, 2023 11:38 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: Haihao <haihao.xiang@intel.com>; artem.galin@gmail.com <artem.galin@gmail.com>; fei.w.wang@intel.com <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened [You don't often get email from dmngaie@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe. On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to transcode 4 inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]" \ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@ 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bits=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to the encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@ 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@ 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bits=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one. _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmpeg-user&data=05%7C02%7Cshanew%40innovsys.com%7C845c76c6fb5043f1d7b508dc0702bfbb%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638392955596088321%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zOOg0JRDiaJ5MpLbWQBjapaeFEyXsr32NsL2DLFNlUw%3D&reserved=0<https://ffmpeg.org/mailman/listinfo/ffmpeg-user> To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
I have created a bug and linked a sample video that changes resolutions on the fly:
https://trac.ffmpeg.org/ticket/10762
Get back to me if you need anything else I can help with. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Wednesday, December 27, 2023 11:38 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: Haihao <haihao.xiang@intel.com>; artem.galin@gmail.com <artem.galin@gmail.com>; fei.w.wang@intel.com <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
[You don't often get email from dmngaie@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to transcode 4 inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
Please use a render node instead, and you should use the key of child_device if you want to use a non-default render node: qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 - filter_hw_device card1
-fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun _nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru n_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun _nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@ 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun _nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow- iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
-c:v h264_qsv encoder can accept data in system memory, you may remove 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph. If you want to use hwupload filter to upload data from system memory to video memory before -c:v h264_qsv encoder, could you try with https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 which added the support for dynamic frame pool in qsv (Note a newer version of oneVPL-intel-gpu is required, you may download oneVPL-intel-gpu from https://github.com/oneapi-src/oneVPL-intel-gpu/tags , and should not use extra_hw_frame=64 anymore ) Thanks Haihao
\ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@ 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit s=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to the encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@ 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun _nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@ 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit s=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one. _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user<; https://ffmpeg.org/mailman/listinfo/ffmpeg-user>
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
On Thu, 28 Dec 2023, 05:15 Xiang, Haihao, < haihao.xiang-at-intel.com@ffmpeg.org> wrote:
I have created a bug and linked a sample video that changes resolutions on the fly:
https://trac.ffmpeg.org/ticket/10762
Get back to me if you need anything else I can help with. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Wednesday, December 27, 2023 11:38 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: Haihao <haihao.xiang@intel.com>; artem.galin@gmail.com <artem.galin@gmail.com>; fei.w.wang@intel.com <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
[You don't often get email from dmngaie@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to
transcode 4
inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
Please use a render node instead, and you should use the key of child_device if you want to use a non-default render node:
qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 - filter_hw_device card1
-fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
n_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow- iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
-c:v h264_qsv encoder can accept data in system memory, you may remove 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph.
If you want to use hwupload filter to upload data from system memory to video memory before -c:v h264_qsv encoder, could you try with https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 which added the support for dynamic frame pool in qsv (Note a newer version of oneVPL-intel-gpu is required, you may download oneVPL-intel-gpu from https://github.com/oneapi-src/oneVPL-intel-gpu/tags , and should not use extra_hw_frame=64 anymore )
Thanks Haihao
\ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
s=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit the
encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
s=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one.
Is this patchwork https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 already present in ffmpeg-cartwheel?
It must be in the latest ffmpeg because I took all the hwupload and hwdownload out of my command and everything still works. One question I have is: Why is hwdownload not needed before the pad filter and hwupload needed after the pad filter. Before trying out intel's hw transcoding I used nvidia's cards and as far as I remember one always had to hwdownload when doing cpu filters and hwupload when going back to gpu filters or gpu encoding. Anyway, thanks for the help, updated command survives resolution changes. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Thursday, December 28, 2023 1:23 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: artem.galin@gmail.com <artem.galin@gmail.com>; Wang, Fei W <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened On Thu, 28 Dec 2023, 05:15 Xiang, Haihao, < haihao.xiang-at-intel.com@ffmpeg.org> wrote:
I have created a bug and linked a sample video that changes resolutions on the fly:
Get back to me if you need anything else I can help with. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Wednesday, December 27, 2023 11:38 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: Haihao <haihao.xiang@intel.com>; artem.galin@gmail.com <artem.galin@gmail.com>; fei.w.wang@intel.com <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
[You don't often get email from dmngaie@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to
transcode 4
inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
Please use a render node instead, and you should use the key of child_device if you want to use a non-default render node:
qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 - filter_hw_device card1
-fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
n_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow- iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
-c:v h264_qsv encoder can accept data in system memory, you may remove 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph.
If you want to use hwupload filter to upload data from system memory to video memory before -c:v h264_qsv encoder, could you try with https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ffmpeg.org%2Fproject%2Fffmpeg%2Flist%2F%3Fseries%3D10318&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZlVkTZJm%2BEZLHToo5dKMfK70Pc1AgixiiH5UQiuf4aI%3D&reserved=0<https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318> which added the support for dynamic frame pool in qsv (Note a newer version of oneVPL-intel-gpu is required, you may download oneVPL-intel-gpu from https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foneapi-src%2FoneVPL-intel-gpu%2Ftags&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DzItmcdB1xbz%2BrnChvp7%2B%2FG%2F8vmNzD9bbB8boVvNW10%3D&reserved=0<https://github.com/oneapi-src/oneVPL-intel-gpu/tags> , and should not use extra_hw_frame=64 anymore )
Thanks Haihao
\ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
s=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit the
encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
s=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one.
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmpeg-user&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dQ914ybkb%2FTpul8SqZ0KMm3jOUejwz1tSTUpRt%2BRwU8%3D&reserved=0<https://ffmpeg.org/mailman/listinfo/ffmpeg-user> To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
On Do, 2023-12-28 at 14:46 +0000, Shane Warren wrote:
It must be in the latest ffmpeg because I took all the hwupload and hwdownload out of my command and everything still works.
One question I have is:
Why is hwdownload not needed before the pad filter and hwupload needed after the pad filter. Before trying out intel's hw transcoding I used nvidia's cards and as far as I remember one always had to hwdownload when doing cpu filters and hwupload when going back to gpu filters or gpu encoding.
vpp_qsv filter supports output in system memory and h264_qsv encoder supports input in system memory too, so you may use or not use hwdownload and hwupload in your case. [...] [Parsed_vpp_qsv_0 @ 0x7f07b00065d0] VPP: input is video memory surface [Parsed_vpp_qsv_0 @ 0x7f07b00065d0] VPP: output is video memory surface [...] [h264_qsv @ 0x5636855b0f80] Encoder: input is system memory surface Thanks Haihao
Anyway, thanks for the help, updated command survives resolution changes.
________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Thursday, December 28, 2023 1:23 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: artem.galin@gmail.com <artem.galin@gmail.com>; Wang, Fei W <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
On Thu, 28 Dec 2023, 05:15 Xiang, Haihao, < haihao.xiang-at-intel.com@ffmpeg.org> wrote:
I have created a bug and linked a sample video that changes resolutions on the fly:
https://trac.ffmpeg.org/ticket/10762<https://trac.ffmpeg.org/ticket/10762>
Get back to me if you need anything else I can help with. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Wednesday, December 27, 2023 11:38 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: Haihao <haihao.xiang@intel.com>; artem.galin@gmail.com <artem.galin@gmail.com>; fei.w.wang@intel.com <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
[You don't often get email from dmngaie@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to
transcode 4
inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
Please use a render node instead, and you should use the key of child_device if you want to use a non-default render node:
qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 - filter_hw_device card1
-fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
n_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow- iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
-c:v h264_qsv encoder can accept data in system memory, you may remove 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph.
If you want to use hwupload filter to upload data from system memory to video memory before -c:v h264_qsv encoder, could you try with https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318<; https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318> which added the support for dynamic frame pool in qsv (Note a newer version of oneVPL-intel-gpu is required, you may download oneVPL-intel-gpu from https://github.com/oneapi-src/oneVPL-intel-gpu/tags<; https://github.com/oneapi-src/oneVPL-intel-gpu/tags> , and should not use extra_hw_frame=64 anymore )
Thanks Haihao
\ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
s=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit the
encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
s=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one.
Is this patchwork https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318<; https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318> already present in ffmpeg-cartwheel?
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user<; https://ffmpeg.org/mailman/listinfo/ffmpeg-user>
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
On Do, 2023-12-28 at 10:23 +0300, Dennis Mungai wrote:
On Thu, 28 Dec 2023, 05:15 Xiang, Haihao, < haihao.xiang-at-intel.com@ffmpeg.org> wrote:
I have created a bug and linked a sample video that changes resolutions on the fly:
https://trac.ffmpeg.org/ticket/10762
Get back to me if you need anything else I can help with. ________________________________ From: ffmpeg-user <ffmpeg-user-bounces@ffmpeg.org> on behalf of Dennis Mungai <dmngaie@gmail.com> Sent: Wednesday, December 27, 2023 11:38 AM To: FFmpeg user questions <ffmpeg-user@ffmpeg.org> Cc: Haihao <haihao.xiang@intel.com>; artem.galin@gmail.com <artem.galin@gmail.com>; fei.w.wang@intel.com <fei.w.wang@intel.com> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened
[You don't often get email from dmngaie@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew@innovsys.com> wrote:
I'm testing out an Intel Flex 140 card and I am attempting to
transcode 4
inputs to 1 output using the xstack_qsv filter. My command takes in 4 udp multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp transport stream. My command looks like so:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
Please use a render node instead, and you should use the key of child_device if you want to use a non-default render node:
qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 - filter_hw_device card1
-fflags discardcorrupt -fflags +genpts -fflags discardcorrupt \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
n_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096 -async_depth 1 -i "udp://@
225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" \ -noautoscale -filter_complex "\ [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \ [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \ [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \ [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \ [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \ [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow- iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
-c:v h264_qsv encoder can accept data in system memory, you may remove 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph.
If you want to use hwupload filter to upload data from system memory to video memory before -c:v h264_qsv encoder, could you try with https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 which added the support for dynamic frame pool in qsv (Note a newer version of oneVPL-intel-gpu is required, you may download oneVPL-intel-gpu from https://github.com/oneapi-src/oneVPL-intel-gpu/tags , and should not use extra_hw_frame=64 anymore )
Thanks Haihao
\ -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \ -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k -threads 1 -profile:v high -bf:v 0 -g:v 15 \ -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto \ -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
s=10528 "
The issue is eventually stream 1 changes resolution from 1280x720 (progressive) to 1920x1080 (interlaced). The stream dies with the following logs coming out:
[h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding. [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit the
encoder Error while filtering: Internal bug, should not have happened
I can transcode the single stream that changes resolution on its own and it can survive a resolution change using this command:
/tmp/ffmpeg_g -nostats -loglevel verbose -init_hw_device qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1 -fflags discardcorrupt -fflags +genpts -fflags discardcorrupt -hwaccel qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
_nonfatal=1" -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v" -map 0:a:0 -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0 -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528 "udp://@
229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
s=10528 "
When that stream changes resolution I see these come out in the log, and the stream carries on without dying:
[h264_qsv @ 0x561bc472f8c0] Video parameter change [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.2.4 (). [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard list, using standard behaviour. [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
My quesiton is why does the stream die when using xstack_qsv but works fine on its own.
This is definitely a bug, something to do with frame pool setting(s) in that filter chain. You may want to open a ticket for this on ffmpeg trac. The xstack_qsv may need a dynamic frame pool workaround to address this anomaly on video parameter changes. Adding in the Intel guys on this one.
Is this patchwork https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 already present in ffmpeg-cartwheel?
Yes, BRs Haihao
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
participants (3)
-
Dennis Mungai -
Shane Warren -
Xiang, Haihao