[FFmpeg-trac] #10119(avcodec:new): h264_amf and hevc_amf is extremely slow when use D3D11 hwaccel as input

[FFmpeg]

#10119: h264_amf and hevc_amf is extremely slow when use D3D11 hwaccel as input
-------------------------------------+-------------------------------------
             Reporter:  Reito        |                     Type:  defect
               Status:  new          |                 Priority:  critical
            Component:  avcodec      |                  Version:  git-
             Keywords:  AMF amf      |  master
  h264_amf                           |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 How to reproduce:

 Initialize an encoding codec using h264_amf or hevc_amf, create an D3D11
 device and initialize the hw_device and hw_frame, feed D3D11Texture2D to
 AVFrame and then avcodec_send_frame, It takes about 12ms for each frame.

 After doing a dig into the amfenc.c source, I find out these code cost
 about 10ms every frame and literally did nothing useful:

 Line 673

 {{{
         if (hw_surface) {
             AMFBuffer *frame_ref_storage_buffer;

             // input HW surfaces can be vertically aligned by 16; tell AMF
 the real size
             surface->pVtbl->SetCrop(surface, 0, 0, frame->width,
 frame->height);

             frame_ref_storage_buffer =
 amf_create_buffer_with_frame_ref(frame, ctx->context);
             AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL,
 AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n");

             res = amf_set_property_buffer(surface, L"av_frame_ref",
 frame_ref_storage_buffer);
             AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN,
 "SetProperty failed for \"av_frame_ref\" with error %d\n", res);
             ctx->hwsurfaces_in_queue++;
 frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer);
         }
 }}}

 Line 733

 {{{
             if (data->pVtbl->HasProperty(data, L"av_frame_ref")) {
                 AMFBuffer *frame_ref_storage_buffer;
                 res = amf_get_property_buffer(data, L"av_frame_ref",
 &frame_ref_storage_buffer);
                 AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN,
 "GetProperty failed for \"av_frame_ref\" with error %d\n", res);
 amf_release_buffer_with_frame_ref(frame_ref_storage_buffer);
                 ctx->hwsurfaces_in_queue--;
             }
 }}}

 After remove these code the encode speed is normal (about 0.5ms per
 frame).
 In my understanding, these code is requesting an buffer from amf, clone an
 AVFrame, copy AVFrame to it, pass it to amf. After frame is submitted and
 encoded, it retrives the buffer, copy AVFrame out and release everything.
 In summary, neither amf nor ffmpeg use anything from this procedure, it is
 totally useless code.

 ffmpeg version
 built on master, 5.1.2, 4.4.3
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10119>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker