[FFmpeg-devel] high cpu usage and frame dropping
donmoir at comcast.net
Wed May 2 22:08:34 CEST 2012
> On Wed, May 02, 2012 at 02:31:34PM -0400, Don Moir wrote:
>> All players I tested fail badly when I push them. They try to drop
>> frames, but they can stop displaying all together and never recover,
>> audio is no longer serviced, and worse.
>> A sad state of affairs for players that have been out for awhile.
> I don't think you can have tested many. MPlayer will just desync, or you
> can tell it to skip even decoding, though that has a good chance of
> Anyway I can't see why it is a sad state, in 90% of cases the video will
> be unwatchable anyway, why putting a lot of effort into what just ends
> up prolonging the users' pain.
I tested WMP, QuickTime, VLC, SMPlayer, and player built into firefox. I
would say the most used players on windows are WMP, QuickTime, and VLC.
SMPlayer uses mplayer.
All these players failed badly when pushed. When I say badly, I mean
It's a sad state mostly because it's handled poorly to begin with and then
they can't even recover from it. Again stop playing altogether, infinite
short audio loops (audio stops getting serviced), and more. WMP seemed to
handle it better but only because it did nothing to deal with it and audio
and video totally out of sync.
>> I have tried various flags etc to speed things up and that helps a little
>> but not enough and they don't really work anyway. There could be
>> something in the flags or callbacks that would help alot but I have not
>> found them yet.
> You're too vague. skiploopfilter always helps a lot for files using the
> For high level H.264 skipping nonref frames will cut the processing time
> by at least half.
I have tried skip_frame and skip_loop_filter. These do help. The thing is
you need to turn these on and off on the fly depending on CPU usage etc. and
that is ok but still in some cases not enough. Also, you can't use these
blindly as you may not get any frame at all depending on demuxer and if
there are going to be future keyframes etc. I tried AVDISCARD_NOREF and
AVDISCARD_NONKEY with some improvement.
I think using skip_frame and skip_loop_filter can have some side effects
such as distortion also, when changing on the fly, so that has to be dealt
In our applications, we have the abilitily to play any number of videos at
the same time. This is not to say we do this in an abitrary way. The most
use case of this is to fade in one video while the other is fading out.
There was a request that I do this and that led me down the path of beating
the hell out of things related to CPU usage. On my fast machine I played 7
high res videos all at the same time in one container with no real impact
and no frame drops.
In my test app, I can control when I need to start dropping frames depending
on desired CPU usage. I can set this to allow anywhere from 1 to 100 percent
usage. For much of testing with many different files, I set it to 1 or 2
percent so I can see the effect it might have.
So the thing that works absolutely the best so far is to stop decoding and
wait but I will continue to dig.
Is there any possible way to know if a future keyframe will exist on a
general basis when index entries may not exist ? If it means I have to look
as every possible codec and make a best guess that would probably be ok.
More information about the ffmpeg-devel