[FFmpeg-devel] Mixture of homographies video stabilization

Paul B Mahol onemda at gmail.com
Thu Aug 2 17:31:29 EEST 2018


On 8/2/18, Matthew Lai <matthewlai-at-google.com at ffmpeg.org> wrote:
> Hello!
>
> I want to write a more advanced video stabilizer for libavfilter (*),
> implementing the algorithm described here -
> https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37744.pdf
> The focus of the paper is rolling shutter removal, but it builds on top of
> another algorithm that does full frame stabilization, and the new algorithm
> does that as well.
>
> This is the algorithm used in YouTube's stabilizing filter, and is state of
> the art. Adobe calls it Warp Stabilizer (it's the same thing as far as I
> can tell from public information anyways).
>
> 3 questions:
> 1. Is there a linear algebra library already in use? I didn't see anything
> in configure, but would be surprised if none of the existing filters work
> with matrices?

There is no such library here used. There are indeed video/audio
filters that work with matrices.

> 2. Is there anything to watch out for re. a high frame delay (say a few
> hundred frames)? Looking at the API, I don't see a callback to flush out
> remaining frames when input frames are finished? Is doing it in two passes
> the only option?

It is handled internally, there are two internal APIs, activate one and legacy.
With legacy you can flush frames when you receive last frame from input.
With newer, activate API, its similar.

> 3. doc/writing_filters.txt says only slice threading is available. That's
> not really possible with this filter, but frame threading is. Can I just
> buffer frames internally (which I need to do anyways to smooth out motion),
> and do my own threading?

You could do it.

>
> * vid.stab is good for what it does, but it only does rotation and
> translation, and doesn't handle zoom, perspective distortion, or rolling
> shutter. This means it's limited when it comes to things like scuba diving
> videos, where the camera is filming the seabed at a small distance and at
> an angle.
>
> Thanks!
> Matthew
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>


More information about the ffmpeg-devel mailing list