[FFmpeg-devel] [PATCH 2/2] avutil/eval: change sqrt to hypot

[Ganesh Ajjanagadde]

On Sat, Nov 14, 2015 at 9:19 PM, Michael Niedermayer
<michael at niedermayer.cc> wrote:
> On Sat, Nov 14, 2015 at 12:11:20PM -0500, Ganesh Ajjanagadde wrote:
>> This improves the mathematical behavior of hypotenuse computation.
>>
>> Signed-off-by: Ganesh Ajjanagadde <gajjanagadde at gmail.com>
>> ---
>>  libavutil/eval.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> if a hypot() function fallback is added this is ok
> otherwise the code as it is before this should be fine too in practice
> doubles have a quite large exponent range, half of that is still quite
> large

It is not just an issue of overflow, but also accuracy more generally.
I gave overflow as the most easily understood issue.
Concrete example:
1e-1, 1e-8
actual: 0.100000000000000499999999...
hypot: 0.100000000000000505151476
naive: 0.100000000000000519029264

I am pretty sure that the error can be far worse, e.g when bad underflow occurs.

There is a fallback, albeit the naive one which Paul does not like,
though I myself don't see why in light of your comment above.

Rewrote and posted updated version that although not as "perfect" as
GNU libm's, is better than a naive sqrt(x*x + y*y).

I admit it is a minor thing, on the lines of the log10 stuff, but I
posted it as it actually has a greater significance.
avutil/eval was used as an illustrative example.

>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The real ebay dictionary, page 1
> "Used only once"    - "Some unspecified defect prevented a second use"
> "In good condition" - "Can be repaird by experienced expert"
> "As is" - "You wouldnt want it even if you were payed for it, if you knew ..."
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>