[FFmpeg-trac] #2445(undetermined:new): Live audio/video gets out of sync when the output (file, network, pipe) lags or is too slow.
FFmpeg
trac at avcodec.org
Sun Apr 7 21:55:19 CEST 2013
#2445: Live audio/video gets out of sync when the output (file, network, pipe)
lags or is too slow.
-------------------------------------+-------------------------------------
Reporter: Max-P | Type: defect
Status: new | Priority: normal
Component: | Version: git-
undetermined | master
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Hello everyone, first please excuse my lenghty bug report. I'm trying to
be as presise and helpful as possible, please correct me if I miss
anything here.
Context: I was trying to stream a game from my desktop with the audio that
comes with it, but always got a random delay with the audio. I couldn't
get it working right, so I searched Google a lot and kind of figured the
problem by taking apart each step of the encoding.
Summary of the bug: When converting live sources (here x11grab and pulse
sources), the audio gets out of sync with the video no matter which
settings are used (-vsync, -af aresample, -async, -isync, -af asetpts=PTS-
STARTPTS, -vf setpts=PTS-STARTPTS, none of them change anything). I found
that this is due to the output being unresponsive or too slow at start,
because it stays in sync when I output to a file, and I found a way to
reliably tell it's not the tcp:// or rtmp:// protocol of FFmpeg that
causes problems.
How to reproduce:
In order to simulate the buggy or laggy output, I made three C programs
(attached with the ffmpeg -report log to this ticket), and all examples
were ran with this command. I understand it's not as good, but it allows
me to rule out any FFmpeg internal protocol bugs:
{{{
% ffmpeg -report \
-f x11grab -s 1280x720 -r 30 -show_region 1 -i $DISPLAY \
-f pulse -ac 2 -ar 44100 -i default \
-vcodec libx264 -b:v 500k -preset veryfast \
-pix_fmt yuv420p -threads 4 -r 30 -s 1280x720 \
-acodec libmp3lame -ab 192k \
-f flv -y - | tee test.flv | ./slow
}}}
So, to explain why I do this: I tell the first ffmpeg instance to output
to stdout, so I can pipe it to any other program I want. I did this,
because I noticed local files were always in perfect sync without any
trouble, but as soon as I tried to pipe a ffmpeg to another ffmpeg to get
it to my server via RTMP, it got out of sync again.
So I pipe it to tee, which will output the result to a local file on disk,
and pipe it again to another program.
1- I pipe it to ./void, which is the program in void.c. It reads STDIN as
fast as it can. The resulting test.flv file is in perfect sync for as long
as I tried it (5 minutes). It works the exact same was as just letting
ffmpeg outputting to test.flv directly. This is the expected behavior.
2- I pipe it to ./slow, which is a variant of the void.c program, but it
wait 3 seconds before reading the data, reads it all and continue reading
it a bit, then wait 3 seconds again and repeat over and over forever. I
was lazy, so it doesn't do proper rate limiting, but it's good enough for
the tests. It simulated periodic lags in the output, which seems to happen
a lot when streaming to slow networks or servers. The resulting test.flv
file gets progressively out of sync. After only 20 seconds, the audio is
already a second off from the video, and it gets worse and worse. '''The
ffmpeg command is left untouched the whole time, only the piped command is
changed.''' This lets me suppose that when the output buffer is completely
filled, the input buffers will get filled, but one of the video or audio
buffers can hold more data than the other, so when the output clears out,
both sources are now out of sync.
3- Just for testing sake, I piped it to my last program, idelay.c, which
simply waits 10 seconds before reading as fast as it can like the void.c
program. It simulates an initial network setup delay, which is possibly
what rtmp:// does for me and gets my stream all out of sync. The output
test.flv file is out of sync by a second or two, but delay seems constant
afterward.
4- Of course, piping to "ffmpeg -i - -codec copy -f flv
rtmp://my.server.tld" causes the same initial delay, which breaks my live
stream just as much as using rtmp://server directly in the original ffmpeg
command.
I seriously hope this is clear enough, I'm doing my best :) If someone can
give me pointers on where to look at, I could also do some test, C is not
my best language but I sure can dig into it if needed.
--
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2445>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list