[FFmpeg-devel] [PATCHv6 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

[Jorge Ramirez]

On 08/25/2017 05:35 PM, wm4 wrote:
>> +static inline int buffer_ops_v4l2buf_to_bufref(V4L2Buffer *in, int plane, AVBufferRef **buf)
>> +{
>> +    if (plane >= in->num_planes)
>> +        return AVERROR(EINVAL);
>> +
>> +    /* even though most encoders return 0 in data_offset encoding vp8 does require this value*/
>> +    *buf = av_buffer_create((char *)in->plane_info[plane].mm_addr + in->planes[plane].data_offset,
>> +                            in->plane_info[plane].lengths, free_v4l2buf_cb, in, 0);
>> +    if (!*buf)
>> +        return AVERROR(ENOMEM);
>> +
>> +    in->status = V4L2BUF_RET_USER;
>> +
>> +    return 0;
>> +}
> This looks like it would trigger massive UB if you keep a frame after
> the decoder is closed.  This should not happen, an AVBufferRef must
> stay valid forever.

Yes you are right.

v4l2 uses a limited number of buffers in physical memory that it reuses 
during the video operation.
these buffers must be mapped to the process address space before they 
can be used.

for decoding, ffmpeg maps the V4L2 physical buffers, memcpies the input 
data to the output queue and uses the _references_ from the capture 
queue to create AVBufferRefs (these references are mmaped addresses of 
physical memory buffers).

In the current design buffers are not be reused until their 
corresponding AVBufferRefs are released (notice that freeing an 
AVBufferRef executes the callback that sets the V4L2BUF_AVAILABLE flag 
making it ready to be enqueued again).

So:

1. would it be acceptable if I stopped the codec from ummaping its 
buffers until all AVBufferRefs have been freed?
 From what you are saying I think this has to be done.

2. also notice that if ffmpeg keeps all the AVBufferRefs for 'too long' 
this would cause the v4l2 pipeline to starve.

I have not seen this case (I have tested all decoders with ffplay) but 
from what you are saying it could be possible (typical hardware can 
handle 32 physical buffers although v4l2 applications rarely request so 
many).
Is there any way in ffmpeg to catch this condition and force the release 
of some of the AVBuffers?