[FFmpeg-devel] [PATCH] vp9: uses ff_set_dimensions (which sets coded_width/height).

[Ronald S. Bultje]

Hi,

On Fri, Oct 23, 2015 at 9:40 PM, Ronald S. Bultje <rsbultje at gmail.com>
wrote:

> Fixes ticket 4935.
> ---
>  libavcodec/vp9.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)


I can give a slightly longer explanation here, I think it's worth it.
Without the above change, coded_width/height are not initialized in vp9.c,
but the values are not uninitialized. I'm guessing that perhaps they are
initialized from the stream (webm) instead of codec (vp9), I didn't
investigate very much because I don't think it matters. Regardless, for
this stream, vp9 says the file is 512x33k, and doesn't initialize
coded_w/h, which at that point have values of 33kx288. This code in
ff_get_buffer_internal is brilliantly innocent (and I can't for the life of
me figure out why it does what it does):

        if (frame->width <= 0 || frame->height <= 0) {
            frame->width  = FFMAX(avctx->width,
 FF_CEIL_RSHIFT(avctx->coded_width,  avctx->lowres));
            frame->height = FFMAX(avctx->height,
FF_CEIL_RSHIFT(avctx->coded_height, avctx->lowres));
            override_dimensions = 0;
        }

And for the given situation indeed leads to a 33kx33k frame->width/height,
which is used for bufferpool allocation. 33kx33k=1.1GB Y plane + 275MB
chroma planes (x2), so 1.65GB per frame, for 7 frames (3 refs + 4 threads)
= 11.5GB, which is the reported max memory use by massif. Valgrind reports
no leaks and it all works fine on my system. Note that for a slightly
differently corrupted vp9 bitstream, one where both size fields were
changed to 33k, this would still happen after the patch and there's nothing
we can do against it, the stream could be perfectly valid (vp9 supports up
to 64kx64k).

Ronald