[Ffmpeg-devel] Metrics for compressed video

[Jon Burgess]

Hi,

I am currently doing some work with the libavcodec library to extract some
metrics from a MPEG-4 Part 2 ASP stream.

By metrics in this case I am talking about determining the magnitude of
motion in each frame.  This then enables a "waveform" view of the stream to
be shown on a GUI (something like the waveform view in a sound editor for
example), making it obvious where there is lots of motion, and making it
easy for a user to jump to a time in the stream where there might be
something interesting happening.

As an aside for those who care, I am using the ratio of intra-coded blocks
to total blocks in each P or B frame to indicate motion (the more intra
coded blocks, then the more motion).  This might not work very well for all
MPEG-4 Part 2 implementations, but it seems to work quite well for my stream
(which is coming from an Axis 241QA video server).

Anyway, my question comes down to the following: I have two options for how
to actually implement this "metric extraction" into libavcodec and our
company's product.
1) Maintain our own copy of the libavcodec source; or
2) Add some sort of generic metric API into libavcodec itself.

I would prefer option 2, since it is obviously better from my point of view
not to have to keep up to-date with changes to libavcodec, as would be the
case for option 1. Secondly, a metric API might be useful for other people,
so why not put it back into open source? Thirdly, making it
generic/extensible means that other devs might come up with some better
metrics, other than I have described.

So basically what I am after is some advice from anyone who knows the
libavcodec well, i.e. the best way to add in this sort of functionality
without polluting the current code (since metrics might not be used by many
people, you wouldn't want to complicate the code for 99% of devs). And you
would also need to be able to switch metric extraction on or off, since it
could potentially be computationally expensive (not for the technique I am
using incidentally).

Any ideas of where to start? I am obviously going to be doing the work for
this, but am prepared to share with everyone else if the libavcodec people
would like it added into the source.

Thanks in advance,
Jono Burgess