[FFmpeg-devel] request for feedback on video codec idea

[Roger Pack]

On 10/14/15, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
> On Wed, Oct 14, 2015 at 8:08 PM, Roger Pack <rogerdpack2 at gmail.com> wrote:
>> Lacking a better place to debate this, I would like to ask some
>> questions on a video codec idea...
>>
>> The goal is basically to create a very fast lossless screen capture
>> codec (i.e. in the input there will be lots of repeated "colors" of
>> neighboring pixels, not a lot of dynamic content between frames).
>>
>> I have become aware of some "fast" compression tools like LZO, LZ4,
>> density, etc.  It seems like they all basically compress "the first
>> 64KB then the next 64KB" or something like that [1].
>>
>> My idea is to basically put pixels of the same position, from multiple
>> frames, "together" in a stream, then apply normal (fast) compression
>> algorithms to the stream.  The hope being that if the pixels are the
>> "same" between frames (presumed to be so because of not much dynamic
>> content), the compression will be able to detect the similarity and
>> compress it well.
>>
>> For instance, given 3 frames of video ("one after another" from the
>> incoming video stream), "combine them" into one stream like:
>> pixel 1 frame 1, pixel 1 frame 2, pixel 1 frame 3, pixel 2 frame 2,
>> pixel 2 frame 2, pixel 2 frame 3 ...
>>
>> then basically apply LZ4 or density algorithm to those bytes.
>>
>> The theory being that if there is a lot of repeated content between
>> frames, it will compress well.
>>
>> The basic need/desire for this was that huffyuv, though super fast at
>> encoding (it basically zips the frame), seems to create *huge* files,
>> I assume because "each frame is an I-frame" so it has to re encode
>> everything each frame.   And also the egotistical desire to create the
>> "fastest video codec in existence" in case the same were useful in
>> other situations (i.e. use very little cpu--even huffyuv uses quite a
>> bit of cpu) :)
>>
>> Any feedback on the concept?
>> Also does anything similar to this already exist? (though should I
>> create my new codec, it would be open source of course, which is
>> already different than many [probably efficient] screen capture codecs
>> out there).
>>
>> Thanks.
>> -roger-
>>
>
> I can't really comment on the merits of this compression scheme, but
> note that you might have trouble with the ffmpeg API when handling
> such a codec, since every single data packet would result in X output
> frames (3 in your example) - this is not a scheme that avcodec can
> really represent well.
> On top of that, containers might have troubles timestamping this properly.

True, I had hoped that FFmpeg would have some "b frame" type
functionality that I could leverage somehow...or perhaps I can fake it
by using null frames or something along those lines.
Cheers!