[FFmpeg-devel] swscale : add bitexact conv for grayf32 and gray16 to f32 conv

[Michael Niedermayer]

On Sun, Sep 16, 2018 at 09:25:14PM +0200, Martin Vignali wrote:
> > you can just use something like (possibly with more or less 0 or a
> > magnitude
> > related value)
> > #define assert_stable_int(x) av_assert(llrintf(x+0.0001) ==
> > llrintf(x-0.0001))
> > #define assert_stable_float(x) av_assert((float)(x+0.000000000001) ==
> > (float)(x-0.000000000001))
> >
> > and then place this where rounding happens, then you can see easily if
> > any test gets close to problematic values. No need for special HW
> >
> > Hello,
> 
> Do you mean something like that ? (which raise the assert for 8b and 16
> bits)
> 
> #define assert_stable_float(x)
> av_assert0((float)(x+0.0000000000000000000000000001) ==
> (float)(x-0.0000000000000000000000000001))
> 
> static void inline fill_uint_to_float_lut(SwsContext *c, int bitdepth) {
>     static const double float_mult8 = 1.0 / 255.0;
>     static const double float_mult16 = 1.0 / 65535.0;
>     int i;
>     double tmp;
> 
>     if (bitdepth == 8) { /*! fill uint8 to float lut */
>         for (i = 0; i < 256; ++i){
>             tmp = (double) i * float_mult8;
>             assert_stable_float(tmp);
>             c->uint2float_lut[i] = (float)tmp;
>         }
>     } else if (bitdepth == 16) { /*! fill uint16 to float lut */
>         for (i = 0; i < 65536; ++i){
>             tmp = (double) i * float_mult16;
>             assert_stable_float(tmp);
>             c->uint2float_lut[i] = (float)tmp;
>         }
>     } else { /*! unsupported bitdepth */
>         av_assert0(0);
>     }
> }

yes, for which values exactly does it fail ?
also, have you tried adding a small constant to tmp ? 
i would expect that this or a similar operation would allow moving
away from all "unstable" points without really changing the output in a
relevant way.

thanks


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Many things microsoft did are stupid, but not doing something just because
microsoft did it is even more stupid. If everything ms did were stupid they
would be bankrupt already.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20180917/c28b6ca7/attachment.sig>