[FFmpeg-devel] [RFC] Synchronize the start times of multiple RTP streams

[Ronald S. Bultje]

Hi,

On Fri, Mar 26, 2010 at 5:52 PM, Martin Storsj? <martin at martin.st> wrote:
> This patch implements my suggestion on how to solve the synchronization of
> multiple RTP streams, the problem that Ronald had that originally lead to
> SVN rev 21857, but without making packets start at a random timestamp.
[..]
> -    if (ret < 0)
> +    if (ret < 0) {
> +        /* Either bad packet, or a RTCP packet. Check whether the
> +         * first_rtcp_ntp_time field was initialized. */
> +        RTPDemuxContext *rtpctx = rtsp_st->transport_priv;
> +        if (rtpctx->first_rtcp_ntp_time != AV_NOPTS_VALUE) {
> +            /* first_rtcp_ntp_time has been initialized for this stream,
> +             * copy the same value to all other uninitialized streams,
> +             * in order to map their timestamp origin to the same ntp time
> +             * as this one. */
> +            int i;
> +            for (i = 0; i < rt->nb_rtsp_streams; i++) {
> +                RTPDemuxContext *rtpctx2 = rtsp_st->transport_priv;
> +                if (rtpctx2 &&
> +                    rtpctx2->first_rtcp_ntp_time == AV_NOPTS_VALUE)
> +                    rtpctx2->first_rtcp_ntp_time = rtpctx->first_rtcp_ntp_time;
> +            }
> +        }
>          goto redo;

One problem here is that it doesn't necessarily take "lowest start
time", but "first start time". Imagine that we have h264+aac, h264
coming in first at a time of 100 (99 start + 1 time), and then aac at
50 (49 + 1). This code will set first_rtcp to 99 for each, and thus
set the timestamp of the first aac data to -50.

What's more worrying is that I think this shouldn't be needed. There's
several functions in utils.c that do this automatically (no action
needed), but they don't work because of overflows because of the crazy
randomness in starttime here. We should fix (imo) that, rather than
trying to workaround it in the RTSP demuxer...

Ronald