[FFmpeg-devel] swscale : add bitexact conv for grayf32 and gray16 to f32 conv

[Michael Niedermayer]

On Mon, Sep 10, 2018 at 07:57:42PM +0200, Martin Vignali wrote:
> >
> > does the LUT generation code produce different results on platforms ?
> > if so i would suggest to try to use double and to add a small offset if
> > needed
> >
> > a 8bit table has 256 entries, a 16bit table 65536
> > a difference would occur if a source value from 64bit floats gets rounded
> > differently to 32bit floats. If this occurs a small offset could be added
> > so that none of the 65536 cases end up close to being between 2 32bit
> > floats
> >
> > This would avoid teh rather complex code if it works
> >
> >
> Hello,
> 
> Can't test on other platform than x86_32 and x86_64, so i can't really
> answer to this question.

> It's the reason why, i write a code, which doesn't use float calc.

you can just use something like (possibly with more or less 0 or a magnitude
related value)
#define assert_stable_int(x) av_assert(llrintf(x+0.0001) == llrintf(x-0.0001))
#define assert_stable_float(x) av_assert((float)(x+0.000000000001) == (float)(x-0.000000000001))

and then place this where rounding happens, then you can see easily if
any test gets close to problematic values. No need for special HW


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Let us carefully observe those good qualities wherein our enemies excel us
and endeavor to excel them, by avoiding what is faulty, and imitating what
is excellent in them. -- Plutarch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20180911/df1a43c2/attachment.sig>