[Ffmpeg-devel] assertion failure in epzs_motion_search_internal

[Michael Niedermayer]

Hi

On Tue, Dec 12, 2006 at 12:59:36AM -0800, Trent Piepho wrote:
> On Mon, 11 Dec 2006, Michael Niedermayer wrote:
> > On Thu, Dec 07, 2006 at 05:01:52PM -0800, Trent Piepho wrote:
> > > mencoder: motion_est_template.c:1044: epzs_motion_search_internal: Assertion `(P[3][0]>>shift) <= xmax' failed.
> > >
> > > This is when encoding some 1280x720 HD material with nearly latest svn,
> > > r7245.
> > >
> > > Options used with mencoder are:
> > >
> > > vcodec=mpeg4:qpel:v4mv:psnr:qprd:vbitrate=6950:vqmin=2:lmin=1:dc=8:trell:
> > > mbd=2:precmp=2:subcmp=2:cmp=2:dia=-4:predia=8:cbp:mv0:sc_factor=4:
> > > vmax_b_frames=2:vb_strategy=2:b_sensitivity=20:brd_scale=1:bidir_refine=1:autoaspect:vpass=1
> > >
> >
> > if "MV.x" is too large for the current macroblock then so it is for
> > the macroblock above and aboveright (P_TOP and P_TOPRIGHT)
> > so it _seems_ that an illegal motion vector has already been used in a
> > previous macroblock ...
> 
> Now that I'm able to repoduce this after only 10 minutes or so of encoding,
> I've tracked it back farther.
> 
> ff_estimate_b_frame_motion: estimate b forward
> ff_estimate_motion_b: 72,18 P_TOPRIGHT is <4092,-800> qpels
> mencoder: motion_est_template.c:1059: epzs_motion_search_internal: Assertion
> `(P[3][0]>>shift) <= xmax' failed.
> 
> Go back to look at macroblock 73,17:
> 
> ff_estimate_b_frame_motion: estimate b forward
> ff_estimate_motion_b: 73,17 P_TOPRIGHT is <-384,-288> qpels
> sab_diamond_search: minima 0 is 0,0 7371 from map eb800000 = 3ae 0 0, key eba00400 = 3ae 400 400
> sab_diamond_search: minima 1 is 1023,-200 4739 from map eb79c3ff = 3ad 738 3ff, key eb99c7ff = 3ae 338 7ff
> sab_diamond_search: minima 2 is -96,-73 6976 from map eb7db7a0 = 3ad 7b6 7a0, key eb9dbba0 = 3ae 3b7 3a0
> sab_diamond_search: minima 3 is -97,-72 6999 from map eb7dbf9f = 3ad 7b7 79f, key eb9dc39f = 3ae 3b8 39f
> sab_diamond_search: minima 4 is -96,-72 6991 from map eb7dbfa0 = 3ad 7b7 7a0, key eb9dc3a0 = 3ae 3b8 3a0
> sab_diamond_search: minima 5 is -95,-72 6983 from map eb7dbfa1 = 3ad 7b7 7a1, key eb9dc3a1 = 3ae 3b8 3a1
> sab_diamond_search: minima 6 is -89,-72 6935 from map eb7dbfa7 = 3ad 7b7 7a7, key eb9dc3a7 = 3ae 3b8 3a7
> sab_diamond_search: minima 7 is -96,-71 7018 from map eb7dc7a0 = 3ad 7b8 7a0, key eb9dcba0 = 3ae 3b9 3a0
> sab_diamond_search: result is <1023,-200> pels, allowed range is -1184..112, -288..448
> ff_estimate_motion_b: 73,17 got <1023,-200> pels from ezps
> ff_estimate_motion_b: 73,17 got <4092,-800> qpels from sub motion
> 
> I've yet to track back where the (MotionEstContext*)c->map[1] value of
> 0xeb79c3ff, which is out of range, comes from.  Is the code in
> sab_diamon_search supposed to detect out of range MV values and discard
> them?  It looks like there is code to do that, but obviously it isn't
> working.  Or should an out of range value like this never be one of the
> minima in sab_diamond_search in the first place?

IIRC none of the minima should be out of range ever
the problem likely is that map is filled with

(y<<ME_MAP_MV_BITS) + x + map_generation

in this each x and y have ME_MAP_MV_BITS bits space (11bit each) which is
enough for -1024 .. 1023, if the motion vector is larger it overflows into
the next field, and the sab extraction routine gets confused, simply
adding a clip() in the map -> minima initaization should fix this 
motion vectors >1023 fullpel units are useless anyway ...

ill add such a clip() ...

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is dangerous to be right in matters on which the established authorities
are wrong. -- Voltaire