[FFmpeg-devel] [PATCH] Hack around gcc 4.6 breaking asm using call.

[Yuriy Kaminskiy]

Reimar Döffinger wrote:
> gcc 4.6 no longer decrements esp to account for local variables.
> Thus using call will end up overwriting some local variable.
> So add an extra one it can safely clobber.
> This is a huge hack because it's basically pure chance it works,
> no idea how this is supposed to be done.
> 
> Fixes trac ticket #397.
> 
> Signed-off-by: Reimar Döffinger <Reimar.Doeffinger at gmx.de>
> ---
>  libswscale/x86/swscale_template.c |   10 ++++++++++
>  1 files changed, 10 insertions(+), 0 deletions(-)
> 
> diff --git a/libswscale/x86/swscale_template.c b/libswscale/x86/swscale_template.c
> index 6196e98..d65be48 100644
> --- a/libswscale/x86/swscale_template.c
> +++ b/libswscale/x86/swscale_template.c
> @@ -2283,6 +2283,10 @@ static void RENAME(hyscale_fast)(SwsContext *c, int16_t *dst,
>  #if defined(PIC)
>      DECLARE_ALIGNED(8, uint64_t, ebxsave);
>  #endif
> +    // HACK: gcc 4.6 no longer decrements esp,
> +    // use this to make it reserve space for the call
> +    // return address
> +    void *dummy;
>  
>      __asm__ volatile(
>  #if defined(PIC)
> @@ -2334,6 +2338,7 @@ static void RENAME(hyscale_fast)(SwsContext *c, int16_t *dst,
>  #if defined(PIC)
>            ,"m" (ebxsave)
>  #endif
> +          ,"m" (dummy)

Hmm, I'm not gcc/assembler expert, but here ebxsave is *input* parameter, but
uninitialized before, not marked as *output* parameter, and there are no
"memory" in clobbers (but it is actually modified/used in asm). So I won't even
consider this gcc bug (or even "surprising behavior"). And it won't surprise me
if your workaround will be broken again by next gcc version.

Note that one of recent commits (b4a224c5e4109cb2cca8bac38628673d685fe763) added
other code with input parameters modified in asm, but not marked as clobbered;
it won't surprise me if this also will be broken by some gcc version (if not
already).

And there are places in postproc where asm uses stack space below esp, that also
does not look future-proof (I posted patch more than year ago; it went nowhere).

>          : "%"REG_a, "%"REG_c, "%"REG_d, "%"REG_S, "%"REG_D
>  #if !defined(PIC)
>           ,"%"REG_b
> @@ -2355,6 +2360,10 @@ static void RENAME(hcscale_fast)(SwsContext *c, int16_t *dst1, int16_t *dst2,
>  #if defined(PIC)
>      DECLARE_ALIGNED(8, uint64_t, ebxsave);
>  #endif
> +    // HACK: gcc 4.6 no longer decrements esp,
> +    // use this to make it reserve space for the call
> +    // return address
> +    void *dummy;
>  
>      __asm__ volatile(
>  #if defined(PIC)
> @@ -2394,6 +2403,7 @@ static void RENAME(hcscale_fast)(SwsContext *c, int16_t *dst1, int16_t *dst2,
>  #if defined(PIC)
>            ,"m" (ebxsave)
>  #endif
> +          ,"m" (dummy)
>          : "%"REG_a, "%"REG_c, "%"REG_d, "%"REG_S, "%"REG_D
>  #if !defined(PIC)
>           ,"%"REG_b