[FFmpeg-devel] Output buffer size in ffmpeg.c

[Måns Rullgård]

In ffmpeg.c, line 1926, the size of the output buffer for compressed
data is calculated as 6*width*height+200.  The subsequent malloc is
failing on my 64MB systems for some inputs, and is the cause of the
redcode-demux FATE failure on AVR32.  This particular test passes if I
enable memory overcommit.  This will not help on blackfin, however,
since it has no MMU and hence overcommit is impossible.

Does the output buffer really need to be this big?  The only case I
can think of that would need it is 48-bit raw.  Perhaps we should use
a more intelligent estimate taking the codec into account.

At the very least, we should print an error message when this
allocation fails.  It would have saved me some debugging time.  Patch
for this sent separately.

-- 
M?ns Rullg?rd
mans at mansr.com