[FFmpeg-trac] #9049(avfilter:new): astats undefined behaviour on float pcm having peaks much greater than 1.0

[FFmpeg]

#9049: astats undefined behaviour on float pcm having peaks much greater than 1.0
--------------------------------------+------------------------------------
             Reporter:  danadam       |                    Owner:
                 Type:  defect        |                   Status:  new
             Priority:  important     |                Component:  avfilter
              Version:  git-master    |               Resolution:
             Keywords:  astats crash  |               Blocked By:
             Blocking:                |  Reproduced by developer:  0
Analyzed by developer:  0             |
--------------------------------------+------------------------------------

Comment (by danadam):

 > Fixed in 89c9c42c5b85b68eddf891e929cfdebd8c163547, I suppose ;)

 Now it crashes 2 lines later in:
 {{{
 p->histogram[av_clip(FFABS(drop) * HISTOGRAM_MAX, 0, HISTOGRAM_MAX)]--;
 }}}

 It is the same situation (gcc (Debian 10.2.1-1)):
 {{{
    0x00005555558d7d1f <filter_channel+5423>:    andpd  %xmm6,%xmm3
    0x00005555558d7d23 <filter_channel+5427>:    mulsd  %xmm7,%xmm3
    0x00005555558d7d27 <filter_channel+5431>:    cvttsd2si %xmm3,%eax
    0x00005555558d7d2b <filter_channel+5435>:    cmp    $0x1fff,%eax
    0x00005555558d7d30 <filter_channel+5440>:    cmovg  %ebx,%eax
    0x00005555558d7d33 <filter_channel+5443>:    add    $0x1,%ecx
    0x00005555558d7d36 <filter_channel+5446>:    cltq
 => 0x00005555558d7d38 <filter_channel+5448>:    subl
 $0x1,0xf8(%r13,%rax,4)
 }}}
 {{{
 rax            0xffffffff80000000  -2147483648
 rbx            0x1fff              8191
 rcx            0x2a                42
 r13            0x55555730eb00      93825023404800
 eflags         0x10202             [ IF RF ]

 xmm3  v2_double = {2152157349.40625, 0}
 xmm6  v2_double = {nan(0xfffffffffffff), 0}
 xmm7  v2_double = {8191, 0}
 }}}

 > In your opinion is that a bug in gcc that should be reported?

 Not sure if the question was for me, but AFAIK compilers work with the
 assumption that the program does not contain UB. Under that assumption
 this optimization is OK (again, AFAIK :-) ).

 Here's that situation distilled to minimum: https://godbolt.org/z/1j5ao8

--
Ticket URL: <https://trac.ffmpeg.org/ticket/9049#comment:8>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker