[FFmpeg-devel] [Patch][GSoC] Motion Estimation filter

[Michael Niedermayer]

On Wed, Apr 13, 2016 at 10:01:35PM +0200, Michael Niedermayer wrote:
> On Wed, Apr 13, 2016 at 07:24:42PM +0000, Davinder Singh wrote:
> > good vectors? how can i improve them? since it search for every possible
> > place, it should give best match. can you give more details, why
> > surrounding vectors need to be considered?
> 
> the goal is to find the true motion that is how the physical object
> moves, the best match pixel by pixel may not represent the true motion
> at all
> for example a block that contains a finely detailed high contrast
> object and that matches with a low difference to only one spot in the
> new image would likely represent a good/true motion vector
> a block that contains a sharp straight line and that matches well
> would likely be accurate in the direction normal to the line but may
> be somewhat inaccurate in the direction along the line
> a block containing nearly flat constant color will quite possibly
> match somewhere best that is very far from the true motion
> and blocks that on another frame move off screen or behind other objects
> will also match best somewhere pixel by pixel but wherever they will
> match it will not be the true motions direction
> 
> whats needed is to have the "strong/good" matching vectors pull any
> surrounding weak ones towards themselfs in some form.
> 
> Researching algorithms/papers that attempt to find true motion well and
> robustly, may be needed. Its liklely theres some existing research and
> experiments in that area. Of course we will have to experiment too
> but there should be some existing stuff
> 
> Its important to note though that this problem differs from motion
> estimation used in video encoding.
> But the methods used in video encoding still should perform much better
> than "flat" independant exhaustive search (they are also MUCH faster)
> For encoder targeted ME there is UMH and the older EPZS
> i dont know what algorithm is good for finding true motion
> but the encoder targeted ones at least have a term that minimizes
> vector differnces (as there is cost in a encoder to store the
> difference from the predicted)

one thing that could be tried in absence of other ideas
is to add a term that penalizes vectors which differ from the
surrounding vectors and just use exhaustive search as done, plus
repeating the search iteratively until it has converged and does
no change anymore.
Using a fast search like EPZS or UMH instead of exhaustive should
be alot more bearable speed wise though

also basic frame interpolation with the motion vectors should probably
be implemented better earlier than later as that would allow much
better judging (by eye) how good various motion estimatin algortithms
perform.

The good news for you is that ive already implemented motion
compensation so you only need to take my mcfps code which ive posted
a while ago and replace the ugly use of the libavcodec motion estimatin
in it by your code

that should be relatively easy and would provide a starting point
to finetune the motion estimation
first to match the iterative EPZS from libavcodec (which is designed
purely with encoding in mind)
and then hopefully to exceed it and make the filter actually work well
and of course anything can be changed in it to work well ...

ill try to update my mcfps code and put it in some branch on my repo
so you can take a look

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The bravest are surely those who have the clearest vision
of what is before them, glory and danger alike, and yet
notwithstanding go out to meet it. -- Thucydides
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160413/b5ed0111/attachment.sig>