[FFmpeg-devel] [PATCH 1/2] avcodec, avformat: deprecate anything related to side data merging

wm4 nfxjfg at googlemail.com
Sat Mar 18 04:38:58 EET 2017

On Fri, 17 Mar 2017 13:59:40 +0100
Hendrik Leppkes <h.leppkes at gmail.com> wrote:

> On Fri, Mar 17, 2017 at 12:32 AM, Michael Niedermayer
> <michael at niedermayer.cc> wrote:
> > On Thu, Mar 16, 2017 at 09:20:55PM +0100, Nicolas George wrote:  
> >> Le sextidi 26 ventôse, an CCXXV, Michael Niedermayer a écrit :  
> >> > Applications which depend on the current default would need  
> >>
> >> ... to implement a correct structure to carry the data from their
> >> demuxer to the lavc decoders.  
> >
> > Is this possible for every application using every framework ?
> >  
> If you are using a multimedia framework, then putting non-media data
> into the media buffers is even worse then doing it internally in
> avformat->avcodec only.
> There could be all sorts of other components in play in such a
> framework which have no idea what this sidedata is, and as someone
> actually working with such a framework daily (DirectShow), it can and
> has broken playback when non-avcodec decoders were used and such data
> was in the packets.

This was how I actually found out about this. I was feeding
libavformat data to other non-libavcodec decoders, and there were
mysterious failures about corrupt data in the packets. I wasn't very
amused when finding out libavformat deliberately put crap there.

Another time it was because libavformat apparently failed to add side
data to an AVPacket that should have been there. Again, libavformat's
"helpful" hack caused trouble.

Not to mention that we sometimes go length to remove minor
inefficiencies like avoiding copying packets (such as requiring padding
for bitstream readers), but then copy and reallocate the packet _twice_
because of the side data merging thing.

> So in short, the "merge" solution has never solved anything for me
> using a media framework, luckily it was only used very sparingly
> (until recently when every packet out of the mpegts demuxer suddenly
> had pointless sidedata and broke all the things).
> If you use a multimedia framework that does in no way allow you to
> attach side data to the packets, then the person working with the
> framework should decide what to do about that, because only they know
> how their framework may work - merging it into the data has potential
> for way more breakage.

That gives us another nice argument: users of different media
frameworks might generally not know about this hack, because side data
is rare. But when a demuxer suddenly adds side data, their code would
suddenly break for no apparent reason. It would probably take them
quite some debugging to find out what happens here.

This whole side data merging thing is so god damn ridiculous, and even
more so because it's actually defended.

More information about the ffmpeg-devel mailing list