[PATCH]aacsbr: rounding
NINT as specified in 14496-3 is round away from zero which corresponds to lroundf not lrintf. I looked at all the places where it could actually affect the result and I found two places but it might be better to change them all to be safe. Thoughts?
On Wed, 2009-11-18 at 05:24 -0500, Alex Converse wrote:
NINT as specified in 14496-3 is round away from zero which corresponds to lroundf not lrintf. I looked at all the places where it could actually affect the result and I found two places but it might be better to change them all to be safe. Thoughts?
I'm generally inclined to do what the spec says unless we can prove it has no effect and alternatives are faster/cleaner or real world implementations do something different that is somehow better...? You can apply these two now and ignore it if you're confident there will be no alteration in other cases. Regards, Rob
On Wed, Nov 18, 2009 at 6:50 AM, Robert Swain <robert.swain@gmail.com> wrote:
On Wed, 2009-11-18 at 05:24 -0500, Alex Converse wrote:
NINT as specified in 14496-3 is round away from zero which corresponds to lroundf not lrintf. I looked at all the places where it could actually affect the result and I found two places but it might be better to change them all to be safe. Thoughts?
I'm generally inclined to do what the spec says unless we can prove it has no effect and alternatives are faster/cleaner or real world implementations do something different that is somehow better...?
Speaking of real world implementations my guess is we'll need an lroundf replacement (just like lrintf...) in internal.h for systems lacking C99 libm. *sigh*
On Wed, Nov 18, 2009 at 6:50 AM, Robert Swain <robert.swain@gmail.com>wrote:
On Wed, 2009-11-18 at 05:24 -0500, Alex Converse wrote:
NINT as specified in 14496-3 is round away from zero which corresponds to lroundf not lrintf. I looked at all the places where it could actually affect the result and I found two places but it might be better to change them all to be safe. Thoughts?
I'm generally inclined to do what the spec says unless we can prove it has no effect and alternatives are faster/cleaner or real world implementations do something different that is somehow better...?
You can apply these two now and ignore it if you're confident there will be no alteration in other cases.
Applied for all cases.
participants (2)
-
Alex Converse -
Robert Swain