[FFmpeg-trac] #10274(avfilter:open): ffmpeg-qsv: 6.0 regression - Error running VPP: undefined behavior (-16)

[FFmpeg]

#10274: ffmpeg-qsv: 6.0 regression - Error running VPP: undefined behavior (-16)
------------------------------------+------------------------------------
             Reporter:  nyanmisaka  |                    Owner:  haihao
                 Type:  defect      |                   Status:  open
             Priority:  normal      |                Component:  avfilter
              Version:  6.0         |               Resolution:
             Keywords:  QSV         |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+------------------------------------
Description changed by nyanmisaka:

Old description:

> Summary of the bug:
>
> Hybrid VA-API->QSV transcoding fails with {{{VPP: undefined behavior
> (-16)}}}.
>
> Worked fine in 5.1.2 so it should be a 6.0 regression caused by the
> recent {scale,deinterlace,overlay,vpp}_qsv refactor. The issue happens in
> both libmfx and libvpl builds.
>
> Device context:
>
> GPU: Arc A380 and A770 (DG2)
> libva: 2.17.0
> gmmlib: 22.3.4
> media-driver: 23.1.3
> onevpl-gpu: 23.1.3
> mediasdk: 23.1.3
>
> How to reproduce:
> {{{
> ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device
> qsv=qs at va -filter_hw_device qs \
> -hwaccel vaapi -hwaccel_output_format vaapi -i test-clip.mkv -an -sn \
> -vf hwmap=derive_device=qsv,format=qsv,scale_qsv=format=nv12 -f null -
> }}}
>
> Error messages:
>
> {{{MFXVideoVPP_RunFrameVPPAsync}}} or {{{MFXVideoCORE_SyncOperation}}}
> failed in
> https://github.com/FFmpeg/FFmpeg/blob/ea3d24bbe3c58b171e55fe2151fc7ffaca3ab3d2/libavfilter/qsvvpp.c#L932
>
> {{{
> [Parsed_scale_qsv_1 @ 0x55d2b97f7480] Error running VPP: undefined
> behavior (-16)0x
> Error while filtering: Internal bug, should not have happened
> Failed to inject frame into filter network: Internal bug, should not have
> happened
> }}}
>
> Video stream info:
>
> MPEG2 progressive 720x480 yuv420p
>
> {{{
> Stream #0:0(eng): Video: mpeg2video (Main), 1 reference frame,
> yuv420p(tv, progressive, left), 720x480 [SAR 32:27 DAR 16:9], SAR 853:720
> DAR 853:480, 29.97 fps, 29.97 tbr, 1k tbn
>     Metadata:
>       BPS-eng         : 4353639
>       DURATION-eng    : 00:43:03.914666666
>       NUMBER_OF_FRAMES-eng: 61957
>       NUMBER_OF_BYTES-eng: 1406178722
>       SOURCE_ID-eng   : 0100E0
>       _STATISTICS_WRITING_APP-eng: MakeMKV v1.17.3 darwin(arm64-release)
>       _STATISTICS_WRITING_DATE_UTC-eng: 2023-03-19 04:25:37
>       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
> SOURCE_ID
>       DURATION        : 00:02:00.070000000
>     Side data:
>       cpb: bitrate max/min/avg: 9800000/0/0 buffer size: 1835008
> vbv_delay: N/A
> }}}
>
> Video clip:
>
> https://f.losno.co/v/test-clip.mkv

New description:

 Summary of the bug:

 Hybrid VA-API->QSV and software->VPP transcoding fails with {{{VPP:
 undefined behavior (-16)}}}.

 Worked fine in 5.1.2 so it should be a 6.0 regression caused by the recent
 {scale,deinterlace,overlay,vpp}_qsv refactor. The issue happens in both
 libmfx and libvpl builds.

 Device context:

 GPU: Arc A380 and A770 (DG2)
 libva: 2.17.0
 gmmlib: 22.3.4
 media-driver: 23.1.3
 onevpl-gpu: 23.1.3
 mediasdk: 23.1.3

 How to reproduce:
 {{{
 ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device
 qsv=qs at va -filter_hw_device qs \
 -hwaccel vaapi -hwaccel_output_format vaapi -i test-clip.mkv -an -sn \
 -vf hwmap=derive_device=qsv,format=qsv,scale_qsv=format=nv12 -f null -
 }}}

 Also fails with software decoding + VPP.

 {{{
 ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device
 qsv=qs at va -filter_hw_device qs \
 -i test-clip.mkv -an -sn \
 -vf hwupload=extra_hw_frames=64,format=qsv,scale_qsv=format=nv12 -f null -
 }}}

 Error messages:

 {{{MFXVideoVPP_RunFrameVPPAsync}}} or {{{MFXVideoCORE_SyncOperation}}}
 failed in
 https://github.com/FFmpeg/FFmpeg/blob/ea3d24bbe3c58b171e55fe2151fc7ffaca3ab3d2/libavfilter/qsvvpp.c#L932

 {{{
 [Parsed_scale_qsv_1 @ 0x55d2b97f7480] Error running VPP: undefined
 behavior (-16)0x
 Error while filtering: Internal bug, should not have happened
 Failed to inject frame into filter network: Internal bug, should not have
 happened
 }}}

 Video stream info:

 MPEG2 progressive 720x480 yuv420p

 {{{
 Stream #0:0(eng): Video: mpeg2video (Main), 1 reference frame, yuv420p(tv,
 progressive, left), 720x480 [SAR 32:27 DAR 16:9], SAR 853:720 DAR 853:480,
 29.97 fps, 29.97 tbr, 1k tbn
     Metadata:
       BPS-eng         : 4353639
       DURATION-eng    : 00:43:03.914666666
       NUMBER_OF_FRAMES-eng: 61957
       NUMBER_OF_BYTES-eng: 1406178722
       SOURCE_ID-eng   : 0100E0
       _STATISTICS_WRITING_APP-eng: MakeMKV v1.17.3 darwin(arm64-release)
       _STATISTICS_WRITING_DATE_UTC-eng: 2023-03-19 04:25:37
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
 SOURCE_ID
       DURATION        : 00:02:00.070000000
     Side data:
       cpb: bitrate max/min/avg: 9800000/0/0 buffer size: 1835008
 vbv_delay: N/A
 }}}

 Video clip:

 https://f.losno.co/v/test-clip.mkv

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10274#comment:2>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker