[FFmpeg-devel] Memory leak using bitstream filters with shared libs

[Michael Niedermayer]

On Wed, Apr 02, 2008 at 10:50:20AM +0200, Baptiste Coudurier wrote:
> Hi guys,
> 
> M?ns Rullg?rd wrote:
> > Baptiste discovered a rather nasty memory leak when using a bitstream
> > filter with shared libraries.
> > 
> > In ffmpeg.c:write_frame() the output of bitstream filters is stored in
> > an AVPacket, and its destruct function is set like this (line 417):
> > 
> >             new_pkt.destruct= av_destruct_packet;
> > 
> > Later on, in av_interleave_packet_per_dts(), AVPacket.destruct is
> > compared against av_destruct_packet, like this (utils.c:2439):
> > 
> >         if(pkt->destruct == av_destruct_packet)
> > 
> > The trouble here is that with shared libraries, the address of
> > av_destruct_packet when used in the main executable or libs other than
> > the one defining it resolves to a trampoline that calls the real
> > function using a GOT entry.  As a result, this comparison is always
> > false when the destruct pointer was assigned outside libavformat.
> > 
> > Does anyone have an idea for fixing this?
> > 
> 
> Sorry, but could we try to find an acceptable solution for this issue ?

as said elsewhere in this thread:
 static inline void av_free_packet(AVPacket *pkt)
 {
-    if (pkt && pkt->destruct) {
-        pkt->destruct(pkt);
+    if(pkt){
+        if(pkt->destruct)
+            pkt->destruct(pkt);
+        pkt->data = NULL; pkt->size = 0;
     }
 }
----
 void av_destruct_packet(AVPacket *pkt)
 {
     av_free(pkt->data);
-    pkt->data = NULL; pkt->size = 0;
 }
----
-void av_destruct_packet_nofree(AVPacket *pkt)
-{
-    pkt->data = NULL; pkt->size = 0;
-}

and use NULL instead of av_destruct_packet_nofree
-----------------
... I dont use libav shared libs ... full patch welcome.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When you are offended at any man's fault, turn to yourself and study your
own failings. Then you will forget your anger. -- Epictetus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080402/6521d910/attachment.pgp>