[FFmpeg-devel] [PATCH 4/5] Prefer pts over dts in timestamp correction

Alexander Strange astrange
Mon Sep 13 05:06:34 CEST 2010


On Tue, Jul 27, 2010 at 9:10 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Mon, Jul 26, 2010 at 01:16:09PM -0700, Alexander Strange wrote:
>> Fixes decoder delay incorrectly causing the output picture timestamps
>> to change.
>> ---
>> ?cmdutils.c | ? ?2 +-
>> ?1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> diff --git a/cmdutils.c b/cmdutils.c
>> index cd0b194..f86a182 100644
>> --- a/cmdutils.c
>> +++ b/cmdutils.c
>> @@ -682,7 +682,7 @@ int64_t guess_correct_pts(PtsCorrectionContext *ctx, int64_t reordered_pts, int6
>> ? ? ? ? ?ctx->num_faulty_pts += reordered_pts <= ctx->last_pts;
>> ? ? ? ? ?ctx->last_pts = reordered_pts;
>> ? ? ?}
>> - ? ?if ((ctx->num_faulty_pts<ctx->num_faulty_dts || dts == AV_NOPTS_VALUE)
>> + ? ?if ((ctx->num_faulty_pts<=ctx->num_faulty_dts || dts == AV_NOPTS_VALUE)
>> ? ? ? ? && reordered_pts != AV_NOPTS_VALUE)
>> ? ? ? ? ?pts = reordered_pts;
>> ? ? ?else
>
> This looks like your code doesnt work with dts.
> A file can require the use of dts by not having pts or having incorrect
> pts, this code is there to select which of dts/pts is better.
> Biasing slightly to pts isnt going to help if there are no pts or wrong
> pts.
>

Here's the specific problem + some other related ones.

dts can be in-order and valid when it was written but still wrong. The
problem I found is that if the initial decoder delay is underspecified
(like h264 without num_reorder_frames set), there can be some dts
value in the file and it will use that, but all the times will be
assigned to the wrong pictures.

Using a logging patch (attached) + this file which doesn't have
num_reorder_frames + ffplay -strict 1 with this file:
http://astrange.ithinksw.net/ffmpeg/camp_mot_mbaff0_full.mp4

Before:
pts - 0, pkt dts - 0, chose - 0 (no picture)
pts - 0, pkt dts - 3003, chose - 3003 (no picture)
pts - 0, pkt dts - 6006, chose - 6006 (no picture)
pts - 0, pkt dts - 9009, chose - 9009 (no picture)
pts - 0, pkt dts - 12012, chose - 12012 (no picture)
pts - 0, pkt dts - 15015, chose - 15015 (no picture)
pts - 0, pkt dts - 18018, chose - 18018 (no picture)
pts - 0, pkt dts - 21021, chose - 21021 (no picture)
pts - 0, pkt dts - 24024, chose - 24024 (no picture)
pts - 0, pkt dts - 27027, chose - 27027 (no picture)
pts - 0, pkt dts - 30030, chose - 30030 (no picture)
pts - 0, pkt dts - 33033, chose - 33033 (no picture)
pts - 0, pkt dts - 36036, chose - 36036 (no picture)
pts - 0, pkt dts - 39039, chose - 39039 (no picture)
pts - 0, pkt dts - 42042, chose - 42042 (no picture)
pts - 0, pkt dts - 45045, chose - 45045 (no picture)
pts - 3003, pkt dts - 48048, chose - 48048
pts - 6006, pkt dts - 51051, chose - 51051
pts - 9009, pkt dts - 54054, chose - 54054

After:
pts - 0, pkt dts - 0, chose - 0 (no picture)
pts - 0, pkt dts - 3003, chose - 0 (no picture)
pts - 0, pkt dts - 6006, chose - 0 (no picture)
pts - 0, pkt dts - 9009, chose - 0 (no picture)
pts - 0, pkt dts - 12012, chose - 0 (no picture)
pts - 0, pkt dts - 15015, chose - 0 (no picture)
pts - 0, pkt dts - 18018, chose - 0 (no picture)
pts - 0, pkt dts - 21021, chose - 0 (no picture)
pts - 0, pkt dts - 24024, chose - 0 (no picture)
pts - 0, pkt dts - 27027, chose - 0 (no picture)
pts - 0, pkt dts - 30030, chose - 0 (no picture)
pts - 0, pkt dts - 33033, chose - 0 (no picture)
pts - 0, pkt dts - 36036, chose - 0 (no picture)
pts - 0, pkt dts - 39039, chose - 0 (no picture)
pts - 0, pkt dts - 42042, chose - 0 (no picture)
pts - 0, pkt dts - 45045, chose - 0 (no picture)
pts - 3003, pkt dts - 48048, chose - 3003
pts - 6006, pkt dts - 51051, chose - 6006
pts - 9009, pkt dts - 54054, chose - 9009

Using pts by default works because pts values are reordered through
the decoder.
I could see another way to fix this (by buffering dts and reading from
the buffer once the first frame comes out) but this seems like less of
a hack to me.

Related problems:
- When ffmpeg.c is flushing output pictures it generates fake dts by
adding one timebase unit to next_pts. In that mp4 file, that seems to
be the wrong unit; the dts starts incrementing by 5 at the end where
all the real times increment by 30000 or so. With this patch, it works
after the patch to ffmpeg.c is applied.
An alternate fix would be to just ignore next_pts when flushing output
pictures, or a better approximation (r_frame_rate?) for next_pts.
- IIRC when playing an h264 PAFF file with field packets, if dts is
used then the timestamp of the second field is used for the output
picture instead of the first field. I can't find a sample for that,
though.
- and, of course, ffmpeg-mt adds extra delay to all streams, breaking
dts even if it was accurate. I could add more API to deal with that -
the delay is predictable, so there could be a value in AVCodecContext
that you subtract from each dts value to get an accurate one - but
that would be more complex than just using pts.

Some of these problem can be fixed, but I haven't found any problems
while testing this patch, and it's much simpler (1 character) than the
alternatives for working with decoder delay. And the faulty pts
detection still works just as well (though it could always be
better/more complicated) for switching back to dts.

(It took me a month to write this email?)



More information about the ffmpeg-devel mailing list