[FFmpeg-devel] swscale : add bitexact conv for grayf32 and gray16 to f32 conv

Michael Niedermayer michael at niedermayer.cc
Tue Oct 9 12:41:23 EEST 2018


On Mon, Oct 08, 2018 at 05:04:43PM +0200, Martin Vignali wrote:
> >
> > also there are 2 divisions in this that you can trivially eliminate
> > /255 and /65535 (extra precission beyond IEEE float/double could change
> > these)
> >
> > also the whole could be done with fewer floats and no extra complexity
> > for example:
> > int64_t tmp2 = 16843009LL * i;
> > (float)((double)tmp2 / (1LL<<32))
> > and
> > int64_t tmp2 = 4295032833LL * i or uint64_t 281479271743489
> >
> >
> Still raise the assert for me.
> Maybe we can go back to the initial patch i propose.
> Let the simple initial code, for non bitexact conversion
> and use a dedicated calc for bit exact version, without float calc

does it fail on actual platforms to produce the same result ?

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20181009/af487560/attachment.sig>


More information about the ffmpeg-devel mailing list