[FFmpeg-devel] rtp streaming x264+audio issues (and some ideas to fix them)

[Luca Abeni]

Hi Michael,

Michael Niedermayer wrote:
[...]
>> On Feb 9, 2010, at 8:14 AM, Luca Abeni <lucabe72 at email.it> wrote:
>>> If you want to avoid large buffering in the client, the buffering
>>> must be in the streamer. This can be done at muxer level, by using
>>> something like Martin's RTSP muxer (it does not implement buffering
>>> right now, but this feature can be implemented later), or at
>>> application level (by using avstreamer instead of ffmpeg).
>> Doesn't lavf do that for you when using av_write_frame_interleaved()?
> 
> that is only correct when each frame is encoded to the same number of bits
> (much much stricter than cbr and very low quality, noone does this)
> 
> If your streams are CBR/VBR and transmitted over a channel of constant maximum
> rate then your "streamer" must before high bitrate parts transmit data ahead
> of time to fill the decoders buffers (which are mandatory anyway).

Yes, I agree with this. This kind of "bandwidth smoothing" should be
implemented (either at application level or in the RTSP muxer). But I
believe it should be configurable/optional, because if the network
provides enough bandwidth (for example, when streaming over a local
ethernet) this mechanism is useless and only increases the delay.


> I dont know how RT*P handles these things or if its just undefined random
> "guess and do what works 98% of the time"

AFAIR, the RTP/RTSP standards do not say anything about buffering leaving
these issues to other standards (ISMA, 3GPP, etc... - For example, ISMA
requires some information about the buffer sizes to be present in the SDP.
But I've already seen some "ISMA compliant" streamers filling the SDP with
some random buffering information, which are completely wrong) or to
applications (the network can introduce some random additional delays, so
RT*P clients generally add a lot of buffering).


> but in mpeg there is an exact buffer model 
> and the muxer is completely aware of how much the decoder has
> in its buffers and how large these buffers are.

When streaming over DVB, this is possible. But when streaming over a
generic network the server can hardly know how much data the client
already has in its buffers ;-)
Maybe this is why (AFAIR) the RT*P standards do not say much about
buffering.


> You should maybe read our mpeg PS muxer code or some text related to high
> quality VBR mpeg muxing or VBV.

Yes, I think implementing a similar mechanism is useful... It's on my
TODO list since a long time :)
BTW, if anyone has suggestions about good papers or texts to read...



			Thanks,
				Luca