[FFmpeg-trac] #10236(avcodec:new): av_hwframe_transfer_data fails from QSV decoded frame on Intel 4th gen hardware

[FFmpeg]

#10236: av_hwframe_transfer_data fails from QSV decoded frame on Intel 4th gen
hardware
--------------------------------------+----------------------------------
             Reporter:  Steve Browne  |                     Type:  defect
               Status:  new           |                 Priority:  normal
            Component:  avcodec       |                  Version:  5.1.2
             Keywords:                |               Blocked By:
             Blocking:                |  Reproduced by developer:  0
Analyzed by developer:  0             |
--------------------------------------+----------------------------------
 Summary of the bug:
 On Intel 4th gen (probably older as well) hardware decode with an h264_qsv
 decoder then try to transfer the frame to system memory with
 av_hwframe_transfer_data.

 For reference the hwdevice_ctx was created as follows:
 {{{
 av_dict_set(&dict, "child_device_type", "d3d11va", 0);
 av_hwdevice_ctx_create(&ctx, AV_HWDEVICE_TYPE_QSV, "auto", dict, 0);
 }}}

 Then try to copy the frame back to system memory:
 {{{
 av_hwframe_transfer_data(dstFrame, frame,
 AV_HWFRAME_TRANSFER_DIRECTION_FROM);
 }}}

 This will fail with:
 {{{
 Error synchronizing the operation: -17
 }}}

 Error -17 corresponds to MFX_ERR_DEVICE_FAILED. I assume the problem is
 simply that MFXVideoVPP_RunFrameVPPAsync doesn't support copying to system
 memory on the older hardware. I updated to the latest Intel drivers etc
 and it still didn't work. On newer Intel hardware this works just fine.

 Luckily this was a result of a recent ffmpeg upgrade of our code where I
 was trying to ditch our existing code that worked with qsv/mfx directly so
 using our existing code I could see that we were creating a staging
 texture and mapping/unmapping that. So the workaround is to avoid
 av_hwframe_transfer_data and instead use av_hwframe_map to map to an
 AV_PIX_FMT_D3D11 frame. Then create the D3D11 staging texture using the
 device I get from the ID3D11Texture2D from the mapped frame, lock the
 staging texture, put those buffers on an intermediate frame then
 av_frame_copy that to the final output frame.

 I've timed the code using av_hwframe_transfer_data vs the workaround on
 newer Intel hardware and there's not much of a performance difference. If
 anything the workaround is ~0.1ms faster on average.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10236>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker