[FFmpeg-devel] swscale-test segfault with 64-bit icc 11.1

Måns Rullgård mans
Sat Jul 17 23:04:29 CEST 2010


Ramiro Polla <ramiro.polla at gmail.com> writes:

> Hi,
>
> swscale-test segfaults when built with 64-bit icc 11.1 (20100414). The
> function that fails is hyscale_fast_MMX2(). Here's a disassembly of
> the function:
> [...]
>
> Since no functions are being called in C inside hyscale_fast_MMX2(),
> the compiler decides it's ok to use -0x8(%rsp) instead of properly
> sub'ing rsp, as it supposedly won't get overwritten. But in this case
> we call the mmx2 code inside asm, overwriting -0x8(%rsp). The second
> callq goes to a522, and when run again, it tries to run some random
> code that was the next pointer on the stack. gcc does the same thing,
> but it seems it leaves -0x8(%rsp) alone and uses the stack -0x10(%rsp)
> and below.

This is similar to the problem I fixed with the Altivec FFT a while
ago...  I rewrote the entire thing in asm, thus avoiding the issue
entirely.

> Is this a compiler bug (as in should it detect a call inside asm)?

Apart from expanding %n operand references, the compiler passes the
asm text verbatim to the assembler.  There is no way it could detect a
call.

> Could (or should) we hint to the compiler that a call is being made
> inside the asm block (I don't even know if this is possible)?

There is, to my knowledge, no way of doing this.  Converting the whole
thing to yasm would of course fix it.

-- 
M?ns Rullg?rd
mans at mansr.com



More information about the ffmpeg-devel mailing list