[FFmpeg-devel] Memory leak in h264

[Ian McIntosh]

Hi All

Using the latest source from svn (updated this morning) I think there is 
a memory leak with decoding h264. When opening, h264 will call the 
function MPV_common_init() which in turn calls init_duplicate_context() 
in mpegvideo.c. This allocates a buffer and stores the pointer in 
s->allocated_edge_emu_buffer. Following this, h264 will then do its own 
context initialisation in context_init() (from h264.c) which will again 
allocate a buffer and overwrite the current s->allocated_edge_emu_buffer 
pointer and so memory is lost. It seems to me the fix is to check  
s->allocated_edge_emu_buffer!=NULL in context_init() before allocating - 
but I am not certain whether this is 100% correct. I include snippets of 
the source to help locate the relevant source lines within the files.

//--------------------------------------------------------------------//
from mpegvideo.c

static int init_duplicate_context(MpegEncContext *s, MpegEncContext *base){
    int i;
.......
    // edge emu needs blocksize + filter length - 1 (=17x17 for halfpel 
/ 21x21 for h264)
    CHECKED_ALLOCZ(s->allocated_edge_emu_buffer, (s->width+64)*2*21*2); 
//(width + edge + align)*interlaced*MBsize*tolerance

    s->edge_emu_buffer= s->allocated_edge_emu_buffer + (s->width+64)*2*21;
........

//--------------------------------------------------------------------//
from h264.c

static int context_init(H264Context *h){
    MpegEncContext * const s = &h->s;
......

     // edge emu needs blocksize + filter length - 1 (=17x17 for halfpel 
/ 21x21 for h264)
       CHECKED_ALLOCZ(s->allocated_edge_emu_buffer,
                                                 (s->width+64)*2*21*2); 
//(width + edge + align)*interlaced*MBsize*tolerance
                    s->edge_emu_buffer= s->allocated_edge_emu_buffer + 
(s->width+64)*2*21;
.......


Regards,

Ian McIntosh