[FFmpeg-devel] [RFC 2/6] Add common V4L2 request API code

[Jonas Karlman]

On 2019-04-09 01:32, Jonas Karlman wrote:
> On 2019-04-09 00:35, James Almer wrote:
>> On 4/8/2019 5:12 PM, Jonas Karlman wrote:
>>> Signed-off-by: Jonas Karlman <jonas at kwiboo.se>
>>> ---
>>>  configure                 |   8 +
>>>  libavcodec/Makefile       |   1 +
>>>  libavcodec/hwaccel.h      |   2 +
>>>  libavcodec/v4l2_request.c | 885 ++++++++++++++++++++++++++++++++++++++
>>>  libavcodec/v4l2_request.h |  65 +++
>>>  5 files changed, 961 insertions(+)
>>>  create mode 100644 libavcodec/v4l2_request.c
>>>  create mode 100644 libavcodec/v4l2_request.h
>>> +int ff_v4l2_request_uninit(AVCodecContext *avctx)
>>> +{
>>> +    V4L2RequestContext *ctx = avctx->internal->hwaccel_priv_data;
>>> +    int ret;
>>> +
>>> +    av_log(avctx, AV_LOG_DEBUG, "%s: avctx=%p ctx=%p\n", __func__, avctx, ctx);
>>> +
>>> +    if (ctx->video_fd >= 0) {
>>> +        ret = ioctl(ctx->video_fd, VIDIOC_STREAMOFF, &ctx->output_type);
>>> +        if (ret < 0)
>>> +            av_log(avctx, AV_LOG_ERROR, "%s: output stream off failed, %s (%d)\n", __func__, strerror(errno), errno);
>>> +
>>> +        ret = ioctl(ctx->video_fd, VIDIOC_STREAMOFF, &ctx->format.type);
>>> +        if (ret < 0)
>>> +            av_log(avctx, AV_LOG_ERROR, "%s: capture stream off failed, %s (%d)\n", __func__, strerror(errno), errno);
>>> +    }
>>> +
>>> +    if (avctx->hw_frames_ctx) {
>>> +        AVHWFramesContext *hwfc = (AVHWFramesContext*)avctx->hw_frames_ctx->data;
>>> +        av_buffer_pool_reclaim(hwfc->pool);
>> What's the idea behind this? Calling this function will free all the
>> unused buffers currently residing in the pool, but will do nothing with
>> those already in assigned to some AVBufferRef created with
>> av_buffer_pool_get(). Those will in fact go back to the pool once they
>> are not needed anymore.
>>
>> Other than immediately freeing up some memory, the new function you
>> added is not going to do much. It wont be until av_buffer_pool_uninit()
>> is called and all the assigned buffers if any returned to the pool that
>> everything will be effectively freed.
> This was added to solve an issue with seeking h264 files, during seeking a new hwaccel instance / buffer pool would be created (pps/sps changed)
> and since we need to pre-allocate output/capture buffers during hwaccel init there was situations where too much CMA memory got consumed
> while there was multiple hwaccel instances / buffer pools active.
>
> This may have been a bigger issue in the beginning when we pre-allocated 20 buffers similar to vaapi hwaccel and when we had a bug in Kodi that
> would hold reference on two instead of one AVFrame while waiting on next frame being decoded, in some situations (seeking too fast) this bug
> resulted in Kodi having reference on AVFrame's from three different buffer pools requiring enough CMA memory for 60 capture and output buffers.
>
> We have since then fixed the bug in Kodi and minimized the number of buffers that is pre-allocated in hwaccel.
> Immediately freeing up unused buffers helps the likelihood of successful initializing a new hwaccel instance / buffer pool, especially useful for 4K content.
>
> Regards,
> Jonas

I captured debug logs while playing a h264 video and seeking a few times, see http://ix.io/1FJ0 for the log.
There we can see that with the first call to ff_v4l2_request_uninit() there is 9 unused buffers that immediately can be freed (around 90MiB)
while a new hwframes context is being created and the player is waiting on next decoded frame from seek point to be presented on screen.

Regards,
Jonas