[FFmpeg-user] Dual Xeon VS Haswell-E, maximize fps

Ryan M rymerrick at gmail.com
Thu Oct 2 18:06:32 CEST 2014 

"The sum of several single threaded encodings will result in more total-fps
then 1 multithreaded encoding."
This is interesting.  So whatever route I take (many separate boxes or one
many core system) feeding it with several jobs at once and limiting it to
one core is the way to go.
The issue with what your suggesting then is that if there is only one or
two jobs to run, a lot of the boxes are going to be sitting there doing
nothing.  Maximizing the throughput on your system requires it always being
feed the maximum # of jobs.   The dual Xeon (or lesser but still many core)
is going to be more flexible in that regard (so the high price tag is for
that flexibility).

On Wed, Oct 1, 2014 at 12:21 PM, Henk D. Schoneveld <belcampo at zonnet.nl>
wrote:

> Some additional things to consider.
>
> Perofrm some short tests, with a 1000 frames file, try with threads 1,
> threads 2, threads 3, etc
> Threads 1 gives you 100% of a single core say 4fps. Threads 2 doesn’t
> perform 8fps, because of overhead, and adding another thread adds less then
> the 2nd core. The sum of several single threaded encodings will result in
> more total-fps then 1 multithreaded encoding.
>
> If you want 40fps you can easily add as many DualCelerons as mentioned
> before to get the desired result, you only need to add the ‘missing’ fps to
> the CPU you already have.
>
> If you could send a 1000frames file, I’m happy to test te results for the
> mentioned Celeron.
>
> Henk
>
> On 01 Oct 2014, at 04:07, Ryan M <rymerrick at gmail.com> wrote:
>
> > It is definitely something to think of/consider. Thanks for the input.
> >
> > On Mon, Sep 29, 2014 at 2:14 PM, Henk D. Schoneveld <belcampo at zonnet.nl>
> > wrote:
> >
> >> Size of 1 system 8 * 8.5 * 2.25 inches, so they are very small just a
> >> little bigger then an Apple Mini.
> >> So 8 of them doen’t take that much space, you can stack them.
> >>
> >> On 29 Sep 2014, at 18:37, Ryan M <rymerrick at gmail.com> wrote:
> >>
> >>> Yeah really the actual space/room is the issue there, not sure where
> i'd
> >>> put that.
> >>> With PPSS, it looks like it distributes the encoding of different files
> >>> over the available resources, it wouldn't spread a single encoding job
> >> over
> >>> them correct?
> >> Yes that’s correct. If 1 resource becomes free, job done, it takes the
> >> next file to be done.
> >>> I do need a way to automate ffmpeg instances so more jobs
> >>> can run at the same time.
> >> You can run as many jobs as you do have CPU cores, with 8 systems being
> >> DualCore, 16 jobs at once.
> >>> My jobs aren't just  a bunch of files sitting in
> >>> a directory, it is querying a db for files that need to be encoded in
> >>> various ways.
> >> You could run multiple instances of PPSS, with different encoding
> >> parameters. You could query your DB for files with ffmpeg parameters X,
> >> make symlinks to Directory X
> >> and
> >> another query for files with parameters Y and make  symlinks to
> directory Y
> >> One PPSS instance runs the X and another instance runs the Y symlinks
> >> directory.
> >>>
> >>>
> >>> On Mon, Sep 29, 2014 at 11:15 AM, Henk D. Schoneveld <
> belcampo at zonnet.nl
> >>>
> >>> wrote:
> >>>
> >>>> Why wouldn’t it be realistic ?
> >>>> Have a look at https://code.google.com/p/ppss/ to distribute jobs
> over
> >>>> the amount of systems you have and works like a charm.
> >>>> On 29 Sep 2014, at 17:05, Ryan M <rymerrick at gmail.com> wrote:
> >>>>
> >>>>> I don't know that it is realistic to have 8 systems running though.
> >> If I
> >>>>> can pay more and have the approx same amount of throughput with one
> >>>> system,
> >>>>> that would be better.
> >>>> In what way would it be better, less room, yes but that’s very
> expensive
> >>>> room you pay for in my opinion.
> >>>>> I just don't want to pay like $4k  more (for the
> >>>>> Xeon vs Haswell-EP) and only get a few more fps out of it.  If
> stepping
> >>>> up
> >>>>> to Xeon allows ~40fps like Sean said then it is probably worth it.
> >>>> Thank
> >>>>> you very much for the input.
> >>>>>
> >>>>> On Sun, Sep 28, 2014 at 7:23 AM, Henk D. Schoneveld <
> >> belcampo at zonnet.nl>
> >>>>> wrote:
> >>>>>
> >>>>>>
> >>>>>> On 28 Sep 2014, at 13:13, Henk D. Schoneveld <belcampo at zonnet.nl>
> >>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>> On 26 Sep 2014, at 18:36, Ryan M <rymerrick at gmail.com> wrote:
> >>>>>>>
> >>>>>>>> I am putting together hardware to encode a large and ever growing
> >>>>>> catalog
> >>>>>>>> of video using ffmpeg/x264.  Much of the source video is 1080p
> >>>> ProResHQ.
> >>>>>>>> Currently I have a box with a Haswell 4770k CPU which gets around
> >>>> 12-15
> >>>>>>>> fps, I am of course looking to increase that as much as possible.
> >>>>>>> Have a look at
> >>>>>>
> >>>>
> >>
> http://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-5960X+%40+3.00GHz&id=2332
> >>>>>>> and compare with
> >>>>>>>
> >>>>>>
> >>>>
> >>
> http://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-4770K+%40+3.50GHz&id=1919
> >>>>>>> or in 1 view
> >>>>>>> http://www.cpubenchmark.net/compare.php?cmp[]=1919&cmp[]=2364
> >>>>>>> 7 times the price for less then 2 times the performance.
> >>>>>>> Spreading the task over more much cheaper CPU’s gives you more bang
> >> for
> >>>>>> the buck if realtime encoding isn’t needed.
> >>>>>>> You could use 8 Dual Celeron systems to get the same performance
> as 1
> >>>>>> E5-2690 where CPU cost would be 8x52.45 vs. 1x2299.99 Of course
> >>>>>> motherboards etc are needed and housing but a low cost MB and
> Housing
> >>>> can
> >>>>>> be bought at < 100.00 a piece. 8*(100+52.45) ~ 1220.00
> >>>>>>> Saved more then $1.000 And the Xeon still needs a motherboard and
> >> case
> >>>>>> of course.
> >>>>>> The CPU Mark and single thread performance are good indicators for
> >>>>>> relative ffmpeg encoding speeds. I do have i5 i7 and the named
> >> Celerons
> >>>>>> which I compared extensively with the same source files and encoding
> >>>>>> parameters.
> >>>>>> Disk I/O isn’t any problem, think of it, 5 times BluRay bitrates
> >> results
> >>>>>> in 90Kb/s < 12MB/s. Every modern disk has no problem with that and
> >> even
> >>>>>> old-fashioned nics, 100Mb/s can cope with that.
> >>>>>>>>
> >>>>>>>> Trying to determine if getting the latest dual CPU Xeon (such as
> >>>> E5-2690
> >>>>>>>> v3) setup is going to be worth the significant additional cost
> over
> >>>> the
> >>>>>>>> best Haswell-E (Core i7-5960X).
> >>>>>>>>
> >>>>>>>> I've read lots of articles/posts and it is not clear to me.  I
> know
> >>>>>> worth
> >>>>>>>> is subjective but looking to know if there'll be significant
> >> increase
> >>>> in
> >>>>>>>> fps using 2 Xeons.  I need to justify the $6-7k price tag.
> >>>>>>>>
> >>>>>>>> Any insight/experience would be appreciated.
> >>>>>>>>
> >>>>>>>> thanks
> >>>>>>>> Ryan
> >>>>>>>> _______________________________________________
> >>>>>>>> ffmpeg-user mailing list
> >>>>>>>> ffmpeg-user at ffmpeg.org
> >>>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> ffmpeg-user mailing list
> >>>>>>> ffmpeg-user at ffmpeg.org
> >>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> ffmpeg-user mailing list
> >>>>>> ffmpeg-user at ffmpeg.org
> >>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>>>>>
> >>>>> _______________________________________________
> >>>>> ffmpeg-user mailing list
> >>>>> ffmpeg-user at ffmpeg.org
> >>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>>>
> >>>> _______________________________________________
> >>>> ffmpeg-user mailing list
> >>>> ffmpeg-user at ffmpeg.org
> >>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>>>
> >>> _______________________________________________
> >>> ffmpeg-user mailing list
> >>> ffmpeg-user at ffmpeg.org
> >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>
> >> _______________________________________________
> >> ffmpeg-user mailing list
> >> ffmpeg-user at ffmpeg.org
> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>

More information about the ffmpeg-user mailing list