[FFmpeg-trac] #10274(avfilter:new): ffmpeg-qsv: 6.0 regression - Error running VPP: undefined behavior (-16)

[FFmpeg]

#10274: ffmpeg-qsv: 6.0 regression - Error running VPP: undefined behavior (-16)
------------------------------------+----------------------------------
             Reporter:  nyanmisaka  |                     Type:  defect
               Status:  new         |                 Priority:  normal
            Component:  avfilter    |                  Version:  6.0
             Keywords:  QSV         |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+----------------------------------
 Summary of the bug:

 Hybrid VA-API->QSV transcoding fails with {{{VPP: undefined behavior
 (-16)}}}.

 Worked fine in 5.1.2 so it should be a 6.0 regression caused by the recent
 {scale,deinterlace,overlay,vpp}_qsv refactor. The issue happens in both
 libmfx and libvpl builds.

 Device context:

 GPU: Arc A380 and A770 (DG2)
 libva: 2.17.0
 gmmlib: 22.3.4
 media-driver: 23.1.3
 onevpl-gpu: 23.1.3
 mediasdk: 23.1.3

 How to reproduce:
 {{{
 ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device
 qsv=qs at va -filter_hw_device qs \
 -hwaccel vaapi -hwaccel_output_format vaapi -i test-clip.mkv -an -sn \
 -vf hwmap=derive_device=qsv,format=qsv,scale_qsv=format=nv12 -f null -
 }}}

 Error messages:

 {{{MFXVideoVPP_RunFrameVPPAsync}}} or {{{MFXVideoCORE_SyncOperation}}}
 failed in
 https://github.com/FFmpeg/FFmpeg/blob/ea3d24bbe3c58b171e55fe2151fc7ffaca3ab3d2/libavfilter/qsvvpp.c#L932

 {{{
 [Parsed_scale_qsv_1 @ 0x55d2b97f7480] Error running VPP: undefined
 behavior (-16)0x
 Error while filtering: Internal bug, should not have happened
 Failed to inject frame into filter network: Internal bug, should not have
 happened
 }}}

 Video stream info:

 MPEG2 progressive 720x480 yuv420p

 {{{
 Stream #0:0(eng): Video: mpeg2video (Main), 1 reference frame, yuv420p(tv,
 progressive, left), 720x480 [SAR 32:27 DAR 16:9], SAR 853:720 DAR 853:480,
 29.97 fps, 29.97 tbr, 1k tbn
     Metadata:
       BPS-eng         : 4353639
       DURATION-eng    : 00:43:03.914666666
       NUMBER_OF_FRAMES-eng: 61957
       NUMBER_OF_BYTES-eng: 1406178722
       SOURCE_ID-eng   : 0100E0
       _STATISTICS_WRITING_APP-eng: MakeMKV v1.17.3 darwin(arm64-release)
       _STATISTICS_WRITING_DATE_UTC-eng: 2023-03-19 04:25:37
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
 SOURCE_ID
       DURATION        : 00:02:00.070000000
     Side data:
       cpb: bitrate max/min/avg: 9800000/0/0 buffer size: 1835008
 vbv_delay: N/A
 }}}

 Video clip:

 https://f.losno.co/v/test-clip.mkv
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10274>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker