[FFmpeg-devel] [PATCH] Fix quadratic memory use in ff_h2645_extract_rbsp() when multiple NALUs exist in packet.

[Kieran Kunhya]

>
> This reduces memory consumption to linear from qudratic but i think
> it still can be made to allocate very large amounts of memory.
> That is with many small NAL units MAX_MBPAIR_SIZE would be allocated
> for each.in worst case.
> So this does fix the qudratic issue but not the OOM issue.
> Using the same buffer for all would fix it unless iam missing something.
> Using the same buffer avoids the padding needs for all but the last.
> So its alot less memory for many small nal units
>

This wouldn't fix the issue because the nals_allocated cache could still
grow with a crafted stream.

Kieran