[FFmpeg-user] Can't get a/v synchronisation right with mpeg transport stream that has gaps in it

[Erik Slagter]

[ ffmpeg not syncing a/v properly with an mpeg transport stream, that
has gaps in it, as input ]

It seems nobody experiences the same issues. As said before it's not
specific to a certain file, it happens with all such streams. So
uploading such a huge stream is imho pointless.

I've done some experimentation in the meantime, the results may or may
not help others, who knows.

This is the exact command I'm using. Note the vaapi hwaccel, yes I am
aware and it does not have any impact on the synchronisation (as it should).

ffmpeg -y -frame_drop_threshold -0.1 -hwaccel vaapi -vaapi_device
/dev/dri/card0 -i test.ts -sn -dn -codec:v mjpeg -q:v 3 -codec:a
pcm_s16le -ac 2 -ar 48000 -af
'aresample=och=2:async=1:osr=48000:min_comp=0.02:min_hard_comp=0.02:max_soft_comp=10000:first_pts=0'
-vsync cfr -shortest -map 0:v,0:a -map 0:a -f matroska test.mkv

The "-af aresample..." is similar to -async with some extra user
settable thresholds. They don't seem to make any difference.

The -vsync cfr makes sure the audio and video streams are physically
aligned within the container (i.e. when some was to demux the container
stream, the audio and video stream would still align to each other
without a skew).

My experience:

- most gaps are recovered from quite well, although a tiny bit of desync
remains, usually about 100 ms that the video lags behind the audio, just
enough to be noticable
- sometimes the desync is considerately larger, up to a few seconds; my
guess that is around a mpeg transport stream timestamp wrapround (every
27 hours), it may be that ffmpeg doesn't handle that too well.

What really surprises me is that mplayer plays these streams perfectly,
as do my DVB settopboxes. So it's not something that's impossible.