[FFmpeg-devel] [PATCH] lavfi/select/WIP: add scene detection.

[David Rice]

On May 26, 2012, at 3:17 PM, Clément Bœsch wrote:

> On Sat, May 26, 2012 at 06:45:09PM +0200, Michael Niedermayer wrote:
> [...]
>>> +static void set_scene_values(AVFilterContext *ctx, AVFilterBufferRef *picref)
>>> +{
>>> +    SelectContext *select = ctx->priv;
>>> +    AVFilterBufferRef *prev_picref = select->prev_picref;
>>> +
>>> +    if (prev_picref &&
>>> +        picref->video->h == prev_picref->video->h &&
>>> +        picref->video->w == prev_picref->video->w) {
>>> +        int64_t score = 0, diff;
>>> +        int x, y;
>>> +
>> 
>>> +#define ABSSC(a) abs((int)p[a] - (int)q[a])
>>> +        for (y = 0; y < picref->video->h; y++) {
>>> +            for (x = 0; x < picref->video->w; x++) {
>>> +                const uint8_t *p =      picref->data[0] + y *      picref->linesize[0] + x*3;
>>> +                const uint8_t *q = prev_picref->data[0] + y * prev_picref->linesize[0] + x*3;
>>> +                score += ABSSC(0) + ABSSC(1) + ABSSC(2);
>>> +            }
>> 
>> theres some optimized code in dsputil to calculate sum of abs diff
>> not sure how easy it is to use from lavfi but above looks pretty
>> inefficient so a TODO/FIXME should be added at least
>> 
> 
> It seems some DSP stuff is going to move to lavu. I added a "TODO: use
> DSPContext.sad[]".
> 
> Also, I'm looking for a better algorithm ATM; I'd like to avoid having the
> need of a user threshold (there was a linked paper mentioning another
> method), so users would just have to use select=scene, with scene being 0
> or 1 (and eventually in the whole [0,1] so it could be used like
> select=gt(scene\,0.8))

Here's a link to the scene detection filter I've been using, http://www.silicontrip.net/~mark/view.php/lavtools/20111103-yuvdiff. In additions to sequential frame comparison, it can look at sequential fields to detect scene changes that are produced with pulldown.
Dave Rice