[FFmpeg-devel] [PATCH] lavf/mov.c: Guess video codec delay based on PTS while parsing MOV header.

[Sasi Inguva]

I was just shooting for a heuristic which works for most of the cases.
Most GOP structures are like this in decoding order
PTS:              Pmin ,  Pmax , (Pmin + k) , (Pmin + k -1) ... (Pmin + 1)
,  (Pmin + 2k) , (Pmin + 2k -1) ... (Pmin + k + 1) .... (Pmax + 1)
PICT_TYPE:     I0         I1               Bk               Bk-1
    B0                  B2k                 B2k-1                  Bk+1
                   I2

The min. buffer size is actually the number of frames that came in between
Pmin and Pmin + 1 ,since the buffer should be able to hold all those frames
(and sort them) , before Pmin + 1 comes along and is output. So here, the
buffer size is "k".

The algo computes the number of decreasing steps from Pmax ->  (Pmin + 1)
=  k . Hence the algo works for this case.

However I realize now that my algo will break,
when Pmin and Pmax are not adjacent
Pmin , *(Pmin + k -1)* , Pmax , (Pmin + k) ,  ... (Pmin + 1) , ...   - The
buffer size is still k but the algo will compute as (k - 1) . e.g.  0, 2,
4, 1, 3

or the path from Pmax -> (Pmin + 1) not strictly decreasing .
Pmin ,  Pmax , (Pmin + k) , (Pmin + k -1), *(Pmin + k +1)*   ... (Pmin +
1)  - The buffer size is now (k + 1) but the algo will still compute as  k
e.g. 0, 4, 2, 3, 1

I can build a better heuristic by finding the  frame with next min. PTS
after Pmin and computing the distance between that frame and Pmin.  However
that will still fail for this example,
 0, 3, 5, 1, 4, 2 . The delay computed will be 2 (because 2 frames between
0 and 1 ) but we need a buffer size of 3 .

On Wed, Nov 22, 2017 at 8:29 AM, Michael Niedermayer <michael at niedermayer.cc
> wrote:

> On Mon, Nov 20, 2017 at 08:27:05PM -0800, Sasi Inguva wrote:
> > Signed-off-by: Sasi Inguva <isasi at google.com>
> > ---
> >  libavformat/mov.c                | 50 ++++++++++++++++++++++++++++++
> ++++++++++
> >  tests/fate/mov.mak               |  7 ++++++
> >  tests/ref/fate/mov-guess-delay-1 |  3 +++
> >  tests/ref/fate/mov-guess-delay-2 |  3 +++
> >  tests/ref/fate/mov-guess-delay-3 |  3 +++
> >  5 files changed, 66 insertions(+)
> >  create mode 100644 tests/ref/fate/mov-guess-delay-1
> >  create mode 100644 tests/ref/fate/mov-guess-delay-2
> >  create mode 100644 tests/ref/fate/mov-guess-delay-3
> >
> > diff --git a/libavformat/mov.c b/libavformat/mov.c
> > index fd170baa57..afb0d4ca5c 100644
> > --- a/libavformat/mov.c
> > +++ b/libavformat/mov.c
> > @@ -3213,6 +3213,54 @@ static int64_t add_ctts_entry(MOVStts**
> ctts_data, unsigned int* ctts_count, uns
> >      return *ctts_count;
> >  }
> >
> > +static void mov_guess_video_delay(MOVContext *c, AVStream* st) {
> > +    MOVStreamContext *msc = st->priv_data;
> > +    int ind;
> > +    int ctts_ind = 0;
> > +    int ctts_sample = 0;
> > +    int64_t curr_pts = AV_NOPTS_VALUE;
> > +    int64_t min_prev_pts = AV_NOPTS_VALUE;
> > +    int64_t prev_max_pts = AV_NOPTS_VALUE;
> > +    int num_steps = 0;
> > +
> > +    if (st->codecpar->video_delay <= 0 && msc->ctts_data &&
> > +        st->codecpar->codec_id == AV_CODEC_ID_H264) {
> > +        st->codecpar->video_delay = 0;
> > +        for(ind = 0; ind < st->nb_index_entries && ctts_ind <
> msc->ctts_count; ++ind) {
> > +            curr_pts = st->index_entries[ind].timestamp +
> msc->ctts_data[ctts_ind].duration;
> > +
> > +            // Everytime we encounter a new max_pts we reset num_steps
> and compute again.
> > +            if (curr_pts > prev_max_pts) {
> > +                st->codecpar->video_delay = FFMIN(FFMAX(st->codecpar->video_delay,
> num_steps), 16);
> > +                num_steps = 0;
> > +                prev_max_pts = curr_pts;
> > +                min_prev_pts = curr_pts;
> > +            } else {
> > +                // Compute delay as the length of the path from max PTS
> to min PTS.
> > +                // Frames: I0 I1 B0 B1 B2
> > +                // PTS:     0  4  1  2  3 -> num_steps = delay = 1
> (4->1)
> > +                //
> > +                // Frames: I0 I1 B1 B0 B2
> > +                // PTS:     0  4  2  1  3 -> num_steps = delay = 2
> (4->2, 2->1)
> > +                if (min_prev_pts != AV_NOPTS_VALUE) {
> > +                    if (curr_pts < min_prev_pts) {
> > +                        ++num_steps;
> > +                        min_prev_pts = curr_pts;
> > +                    }
> > +                }
> > +            }
>
> Can you explain why this algorithm is correct ?
> (iam asking as i suspect it is not correct, but i may be wrong)
>
> What this should do is find the minimum buffer size to sort the stream
> of frame timestamps.
>
>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Its not that you shouldnt use gotos but rather that you should write
> readable code and code with gotos often but not always is less readable
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>