[FFmpeg-devel] [PATCH] avfilter/framepool: use av_image_fill_plane_sizes() to calculate pool sizes

[James Almer]

On 3/3/2022 1:46 PM, Paul B Mahol wrote:
> On 3/3/22, James Almer <jamrial at gmail.com> wrote:
>>
>>
>> On 3/3/2022 10:56 AM, Paul B Mahol wrote:
>>> On 3/3/22, James Almer <jamrial at gmail.com> wrote:
>>>>
>>>>
>>>> On 3/1/2022 11:23 AM, James Almer wrote:
>>>>> Signed-off-by: James Almer <jamrial at gmail.com>
>>>>> ---
>>>>>     libavfilter/framepool.c | 20 ++++++++++++++------
>>>>>     1 file changed, 14 insertions(+), 6 deletions(-)
>>>>>
>>>>> diff --git a/libavfilter/framepool.c b/libavfilter/framepool.c
>>>>> index 5b510c9af9..cf6a1d0ea0 100644
>>>>> --- a/libavfilter/framepool.c
>>>>> +++ b/libavfilter/framepool.c
>>>>> @@ -57,6 +57,8 @@ FFFramePool *ff_frame_pool_video_init(AVBufferRef*
>>>>> (*alloc)(size_t size),
>>>>>         int i, ret;
>>>>>         FFFramePool *pool;
>>>>>         const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(format);
>>>>> +    ptrdiff_t linesizes[4];
>>>>> +    size_t sizes[4];
>>>>>
>>>>>         if (!desc)
>>>>>             return NULL;
>>>>> @@ -89,13 +91,19 @@ FFFramePool *ff_frame_pool_video_init(AVBufferRef*
>>>>> (*alloc)(size_t size),
>>>>>             }
>>>>>         }
>>>>>
>>>>> -    for (i = 0; i < 4 && pool->linesize[i]; i++) {
>>>>> -        int h = pool->height;
>>>>> -        if (i == 1 || i == 2)
>>>>> -            h = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
>>>>> +    for (i = 0; i < 4; i++)
>>>>> +        linesizes[i] = pool->linesize[i];
>>>>>
>>>>> -        pool->pools[i] = av_buffer_pool_init(pool->linesize[i] * h +
>>>>> align,
>>>>> -                                             alloc);
>>>>> +    if (av_image_fill_plane_sizes(sizes, pool->format,
>>>>> +                                  FFALIGN(pool->height, align),
>>>>> +                                  linesizes) < 0) {
>>>>> +        goto fail;
>>>>> +    }
>>>>> +
>>>>> +    for (i = 0; i < 4 && sizes[i]; i++) {
>>>>> +        if (sizes[i] > SIZE_MAX - align)
>>>>> +            goto fail;
>>>>> +        pool->pools[i] = av_buffer_pool_init(sizes[i] + align, alloc);
>>>>>             if (!pool->pools[i])
>>>>>                 goto fail;
>>>>>         }
>>>>
>>>> Ping. I can also remove the height padding if preferred.
>>>
>>> It does not work. No?
>>
>> It doesn't fix the mpeg encoder issue with yuv422p streams and 64 stride
>> alignment, no, but it's not strictly about that either. It's a
>> simplification using existing helpers.
> 
> Ok, if you need it.

Applied without the vertical padding, to not change the behavior your 
introduced in 17a59a634c.