[FFmpeg-user] CPU and GPU

[Ted Park]

Hi,

> What is implemented in FPGAs is not solely so-called software-in-silicon. Sometimes it is software-in-silicon -- marginal speedup -- but usually software algorithms are realized as systems of FSMs (finite state machines) -- huge speedup -- that simply mimic the original software processes.

FSMs are pretty abstract in my head, but I meant that simplifying the logic so they mimic original behavior closely enough can’t be a trivial one. I think that with the scale of human effort it would involve, by using the libraries, it would end up contributing indirectly at least.


But I may have misunderstood it seems…

> That is not what I meant. What I'm getting at is that it appears that the latest streaming protocols & methods are being developed via ffmpeg, then being tested-debugged-retested-etc. via ffmpeg users, then implemented in FPGAs by software-in-silicon vendors like NGCodec, MainConcept, Beamr, etc., and finally cast into FSMs in GPUs.


Which protocols, do you mean like its API?

> So codec engineering companies like NGCodec, MainConcept, Beamr and MulticoreWare turn open source-based, ffmpeg workflows into FPGAs that, when mature firmware implementations, chip companies like Intel & NVIDIA turn into real hardware: masked GPUs. Do I have that right?


If they market standalone hardware encoders they probably write firmware for it, but I guess that could be some build of linux with ffmpeg. But I/O isn’t going to be handled by GPUs and FPGAs, not sure what streaming protocols you mean.

Software development being driven by advances in hardware is just normal for codecs I feel; I don’t know if FPGAs started being used before new codecs like h.265 were standardized and bitstreams frozen, but there was existing hardware.

> I presume that what is implemented in GPUs (CUDA cores, for example) are the FSMs with custom hardware tool blocks (transform processors and the like) operating as slaved coprocessors shared by the cores -- even bigger speedup.

Individual cuda cores could actually be described as a FSM, I think, but it is managed in a group, so they are like the coprocessors with shared scheduler and memory.

Regards,
Ted Park