[FFmpeg-devel] [PATCH 3/3] avcodec/x86/lossless_videoencdsp: Fix warning: signed dword value exceeds bounds

[Henrik Gramner]

On Sat, Sep 30, 2017 at 12:58 AM, Michael Niedermayer
<michael at niedermayer.cc> wrote:
> -    and                i, -2 * regsize
> +    and                i, -(2 * regsize)

regsize is defined to mmsize / 2 in the relevant case so the
expression resolves to -2 * 16 / 2

In nasm integers are 64-bit and / is unsigned division (// is signed
division). When you perform unsigned division on a negative number you
get a large number that doesn't fit inside a 32-bit immediate. E.g.
the warning is working as intended.

A better fix is to either use parentheses in the regsize definition
(e.g. what you'd normally do in an equivalent C define), or use
%assign instead of %define. Using // is kind of ugly so I rather avoid
that.