[FFmpeg-devel] rtp streaming x264+audio issues (and some ideas to fix them)

[Martin Storsjö]

On Sat, 6 Feb 2010, Timo Ter?s wrote:

> Looking at the code it looks like the RTCP packet timestamps
> are calculated relative to the first time they are sent which
> is not really correct.
> 
> As a quick test, I did the following to ensure that the RTCP
> packets were using same base time.
> 
> --- libavformat/rtpenc.c	(revision 21651)
> +++ libavformat/rtpenc.c	(working copy)
> @@ -91,7 +91,8 @@
>     s->cur_timestamp = 0;
>     s->ssrc = 0; /* FIXME: was random(), what should this be? */
>     s->first_packet = 1;
> -    s->first_rtcp_ntp_time = AV_NOPTS_VALUE;
> +    //s->first_rtcp_ntp_time = AV_NOPTS_VALUE;
> +    s->first_rtcp_ntp_time = ntp_time();
> 
>     max_packet_size = url_fget_max_packet_size(s1->pb);
>     if (max_packet_size <= 12)
> 
> What would be the proper way to fix the synchronization issue?
> So that the RTP packets sent out are approximately (<1s difference)
> from the original stream.

I've also been looking at this same issue, and this is IMO the first step 
towards getting it right. This way, the sync error between the two streams 
is reduced to the duration between the calls to rtp_write_header - 
currently it is the duration between the first sent RTP packets for both 
streams.

Ideally, the first_rtcp_ntp_time field should be set to the exact same 
ntp_time() value for all RTP sessions within the same stream, but I'm not 
sure what the best way to synchronize it between them would be. Setting 
it in some metadata field?

// Martin