[FFmpeg-devel] [PATCH 1/4] softfloat: handle INT32_MIN correctly in av_normalize1_sf

[Andreas Cadhalpun]

On 08.11.2015 01:06, Michael Niedermayer wrote:
> On Sun, Nov 08, 2015 at 12:07:08AM +0100, Andreas Cadhalpun wrote:
>> Otherwise a.mant=INT32_MIN triggers the av_assert2.
>>
>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
>> ---
>>  libavutil/softfloat.h | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/libavutil/softfloat.h b/libavutil/softfloat.h
>> index 00ff4a1..5fa5dc0 100644
>> --- a/libavutil/softfloat.h
>> +++ b/libavutil/softfloat.h
>> @@ -77,6 +77,10 @@ static inline av_const SoftFloat av_normalize1_sf(SoftFloat a){
>>      if((int32_t)(a.mant + 0x40000000U) <= 0){
>>          a.exp++;
>>          a.mant>>=1;
>> +        if(a.mant == -0x40000000){
>> +            a.exp++;
>> +            a.mant>>=1;
>> +        }
> 
> is  av_int2sf() the only way to reach this case ?

It seems that currently av_int2sf is always at least indirectly involved.

> if so it would be better to modify only av_int2sf() ans avoid this
> extra check in other cases (would slow them down)

OK, I'll send an updated patch 2. Consider this patch dropped.

Best regards,
Andreas