[Ffmpeg-devel] Audio/Video Syncing...

Ryan Martell rdm4
Tue Oct 17 23:10:07 CEST 2006

On Oct 14, 2006, at 3:53 AM, Michael Niedermayer wrote:

> On Fri, Oct 13, 2006 at 09:15:20PM -0500, Ryan Martell wrote:
>> On Oct 13, 2006, at 7:06 PM, Michael Niedermayer wrote:
>>> On Fri, Oct 13, 2006 at 05:58:57PM -0500, Ryan Martell wrote:
>>>> I have implented the rtsp streaming over h264.  Everything works
>>>> against Darwin Streaming Server (packetization mode 1 & 2).  The  
>>>> only
>>>> problem is that I can't figure out what I'm doing wrong on getting
>>>> the audio/video to sync.
>>>> I have called this on the stream:
>>>>    av_set_pts_info(stream, 33, 1, 90000)
>>>> And 90kHz is the standard for h264 streaming.
>>>> When I let ffmpeg figure out the frame rate, sometimes it shows  
>>>> up as
>>>> 10 fps(r), and other times it shows up at 29.97 fps(r).
>>>> If I hardcode the fps (num=30000, and den= 1001), it will still
>>>> sometimes playback at 10.
>>> well the question here is what is the real framerate of the stream
>>> or in other words how many frames do you pass to libavformat per
>>> second? and what is the difference between the pts values of the
>>> frames you pass to lavf
>> Well, I'm passing them as fast as I get them, which I suspect is
>> close to 30fps.  The source movie is 30fps, and occasionally
>> (probably due to ffplay's a/v syncing stuff) it plays faster than 30
>> fps.
>> I'm wondering if the frame rate randomness is some sort of artifact
>> of when the rtcp packet shows up?  Again, the stream info (with the
>> fps) doesn't show up before some delay (which I haven't fully
>> investigated yet.  Maybe my frame rate is affected by the arrival of
>> the rtcp packet?  (I'm going to try to discard all packets that
>> arrive before the rtcp packet shows up, and see what happens).
>> Here's an output with DEBUG_SYNC defined, at the point that it
> well, my question was more, what dts/pts do you pass to libavformat
> not what ffplay contains internally, ffplay buffers packets and stuff
> so its internal timestamps might not be exactly what you passed to it
> also does ffmpeg show the same AVsync issue when transcoding to some
> random fileformat? what if you specify some -async ?
> [...]
>>>> When I enable DEBUG_SYNC in ffplay, the pts value I'm giving it
>>>> starts off around -9000 (this is because there is no rtcp packet  
>>>> yet
>>>> to sync the timebase to, I think), then jumps to 0, and proceeds to
>>>> go up from there.  The A-V delay will end up around 11 seconds at
>>>> times, other times it will be closer to zero.
>>> does the delay stay constant or does it increase?
>>> and can you elaborate on the rtcp packet sync thing ...
>> The delay tends to decrease, albeit slowly.
>> The rtcp packet sync thing is this code:
>>     if (s->last_rtcp_ntp_time != AV_NOPTS_VALUE) {
>>         int64_t addend;
>>         int32_t delta_timestamp;
>> /*	
>> 	ntp format represents seconds and fraction as a 64-bit unsigned
>> fixed-point integer with decimal point
>> 	to the left of bit 32 numbered from the left. The 32-bit seconds
>> field spans about 136 years, while the
>> 	32-bit fraction field precision is about 232 picoseconds.
>> */
>>         //      fprintf(stderr, " (has timestamp) ");
>>         /* XXX: is it really necessary to unify the timestamp  
>> base ? */
>>         /* compute pts from timestamp with received ntp_time */
>>         delta_timestamp = timestamp - s->last_rtcp_timestamp;
>> //      if(delta_timestamp<0) fprintf(stderr, "Backwards in time: %d
>> \n", delta_timestamp);
>>         /* convert to 90 kHz without overflow */
>>         addend = (s->last_rtcp_ntp_time - s->first_rtcp_ntp_time) >>
>> 14;
>>         addend = (addend * 5625) >> 14;
>>         adjusted_timestamp = addend + delta_timestamp;
>>     }
>> This code comes from rtp.c.
> note, patches with code duplication will not be accepted unless  
> removing
> the duplication would be very difficult, but thats not the case  
> here, it
> needs a mere addition of a function which does this timestamp adjust

This is just me trying to make it work before I clean it up to  
submit. ;-)

>> The aac stuff doesn't adjust pts
>> anywhere in the rtp code (that I can see), so I'm not sure how it's
>> generating it's pts.
> maybe thats why it doesnt work ...

Maybe that is part of my problem.  I figured that at some point that  
aspect of it had been tested, so I didn't want to screw up someone  
else's working code.  I was going on the assumption that it was  
working for someone else, so they had already sorted out that issue.

>> Essentially, because of above, before the first rtcp packet, i have
>> pts values that aren't necessarily scaled correctly.  The Stream Info
>> won't show up (ffplay -stats) until the first rtcp packet shows up,
>> so there is no audio or video playing, but the packets are being
>> cached.  I guess I could pitch them until we get the first packet,
>> and see what that does....
> sounds like a good idea, even if it doesnt fix the issue, packets with
> random timestamps arent good ...

Still doesn't sync, but I like the behavior better too.

> [...]
>> PS: Did you get a chance to look over my code on the other thread to
>> see if it was getting close to acceptable?
> its 80k looking through that takes time, be patient

No problem; just wanted to make sure it wasn't forgotten; I know  
you're busy. ;-)


More information about the ffmpeg-devel mailing list