[FFmpeg-devel] Using ASAN address sanitizer

[floyd]

Ok, maybe someone could instead confirm that it is this issue (seems
most likely):

https://savannah.gnu.org/patch/?8775

And if one of the main devs could comment on that bug that would
probably be helpful too (so it gets prioritized)

On 19/11/15 13:09, floyd wrote:
> Hi there
> 
> I had an issue when trying to use Address Sanitizer with ffmpeg, I'm
> using the git version with latest commit being
> 50ce510ac4e3ed093c051738242a9a75aeeb36ce. Although I had similar
> problems in the past for quiet a while. The configure script works fine
> without address sanitizer:
> 
> CFLAGS="-Wall -g -fno-omit-frame-pointer" ./configure
> --logfile=logfile.txt --cc=clang --cxx=clang++
> 
> However, as soon as I use the address sanitizer:
> 
> CFLAGS="-Wall -g -fno-omit-frame-pointer -fsanitize=address" ./configure
> --logfile=logfile.txt --cc=clang --cxx=clang++
> 
> I get an error that the "C compiler test failed". The logfile.txt shows
> pretty clearly where the issue is (linking will fail if the linker
> doesn't take into consideration that ASAN is used):
> 
> check_cc
> BEGIN /tmp/ffconf.IEqlUVIW.c
>     1   int main(void){ return 0; }
> END /tmp/ffconf.IEqlUVIW.c
> clang -Wall -g -fno-omit-frame-pointer -fsanitize=address -c -o
> /tmp/ffconf.YHgHqrKy.o /tmp/ffconf.IEqlUVIW.c
> clang -o /tmp/ffconf.HCBA8f8F /tmp/ffconf.YHgHqrKy.o
> /tmp/ffconf.YHgHqrKy.o: In function `main':
> /tmp/ffconf.IEqlUVIW.c:1: undefined reference to
> `__asan_option_detect_stack_use_after_return'
> /tmp/ffconf.IEqlUVIW.c:1: undefined reference to `__asan_stack_malloc_1'
> /tmp/ffconf.IEqlUVIW.c:1: undefined reference to `__asan_report_store4'
> /tmp/ffconf.YHgHqrKy.o: In function `asan.module_ctor':
> /tmp/ffconf.IEqlUVIW.c:1: undefined reference to `__asan_init_v3'
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
> C compiler test failed.
> 
> I then manually confirmed that setting the -fsanitize=address option
> with the second invocation of clang fixes the problem.
> 
> I tried to fix the problem in the configure script inside the shell
> function check_ld, although I'm not sure that's the correct location and
> if CFLAGS is the correct variable to use there. But it works for me. The
> patch is further below. Same problem with gcc.
> 
> Could someone have a look who is more familiar with the configure script
> setup? Or is there another way I can get around this? Thanks
> 
> Best regards,
> floyd
> 
> 
> From efcb839969c52065fc7277bebed68287833f8479 Mon Sep 17 00:00:00 2001
> From: floyd <floyd at floyd.ch>
> Date: Thu, 19 Nov 2015 12:50:04 +0100
> Subject: [PATCH] This fixes the issue that the configure script is not
> passing
>  -fsanitize=address in all cases and therefore fails with a "C compiler test
>  failed"
> 
> ---
>  configure | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/configure b/configure
> index 9a736ce..fd6b7f8 100755
> --- a/configure
> +++ b/configure
> @@ -935,7 +935,7 @@ check_ld(){
>      check_$type $($cflags_filter $flags) || return
>      flags=$($ldflags_filter $flags)
>      libs=$($ldflags_filter $libs)
> -    check_cmd $ld $LDFLAGS $flags $(ld_o $TMPE) $TMPO $libs $extralibs
> +    check_cmd $ld $CFLAGS $LDFLAGS $flags $(ld_o $TMPE) $TMPO $libs
> $extralibs
>  }
> 
>  print_include(){
> 

-- 
floyd
@floyd_ch
http://www.floyd.ch