#7790(build system:new): Endianness detection breaks with LTO
#7790: Endianness detection breaks with LTO --------------------------------------+---------------------------------- Reporter: RedDwarf82 | Type: defect Status: new | Priority: normal Component: build system | Version: 4.1 Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | --------------------------------------+---------------------------------- ffmpeg's configure wrongly thinks my toolchain uses little-endian because of LTO. {{{ $ cat test.c unsigned int endian = 'B' << 24 | 'I' << 16 | 'G' << 8 | 'E'; $ gcc -c -o test.o test.c $ od -t x1 test.o | grep '42 *49 *47 *45' $ gcc -c -o test.o test.c -fno-lto $ od -t x1 test.o | grep '42 *49 *47 *45' 0000160 42 49 47 45 00 47 43 43 3a 20 28 42 75 69 6c 64 }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks with LTO -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: defect | Status: new Priority: normal | Component: build | system Version: 4.1 | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by RedDwarf82): To be clear, I show that code because that's what configure does -> https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/c2a221c5ae5017e11654b9688ac97e... -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:1> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks with LTO -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: defect | Status: new Priority: normal | Component: build | system Version: 4.1 | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by heleppkes): If you intend to build ffmpeg with LTO enabled, you should do so by using the --enable-lto switch, which will only enable LTO after such tests are being performed, and not enable it behind FFmpegs back. -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:2> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks with LTO -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: defect | Status: new Priority: normal | Component: build | system Version: unspecified | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by cehoyos): * version: 4.1 => unspecified Comment: I cannot reproduce: `--enable-lto` does not break big-endian detection on different systems with different compilers I tested. Please test current FFmpeg git head (nothing else is supported here) and tell us about your configure line, compiler and operating system (`gcc -v`, `uname -a`). -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:3> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks with LTO -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: defect | Status: new Priority: normal | Component: build | system Version: unspecified | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by RedDwarf82): It's a buildroot gcc build. I can see "-flto" was added to BR2_TARGET_OPTIMIZATION, which is why it's the default. I can workaround my specific case. And I don't expect lots of people (but probably more than 1) to have gcc with LTO enabled by default. So feel free to close the issue if you want to. But at the same time it seems to me that depending on people not enabling it "behind FFmpegs back" is error-prone. I would say at the very least the check in https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/c2a221c5ae5017e11654b9688ac97e... should also check if it can find "45 *47 *49 *42". If it can't find either configure should fail. -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:4> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: defect | Status: new Priority: normal | Component: build | system Version: unspecified | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:5> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: defect | Status: closed Priority: normal | Component: build | system Version: unspecified | Resolution: | needs_more_info Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by cehoyos): * status: new => closed * resolution: => needs_more_info -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:6> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: enhancement | Status: reopened Priority: normal | Component: build | system Version: unspecified | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by RedDwarf82): * status: closed => reopened * type: defect => enhancement * resolution: needs_more_info => Comment: Reopening since I really don't think this is a "needs_more_info" case. heleppkes has clearly described the issue. Also because doesn't even require the compiler to use LTO by default, so this may be a more common issue than I suggested before. Probably it's not more reported only because the more common desktop architecture happens to be little endian. But here you have the extra information requested: {{{ $ git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg Cloning into 'ffmpeg'... remote: Counting objects: 563150, done. remote: Compressing objects: 100% (118600/118600), done. remote: Total 563150 (delta 450571), reused 554797 (delta 443403) Receiving objects: 100% (563150/563150), 133.44 MiB | 6.03 MiB/s, done. Resolving deltas: 100% (450571/450571), done. $ cd ffmpeg/ $ git rev-parse HEAD 0321370601833f4ae47e8e11c44570ea4bd382a4 $ mkdir build_normal build_lto $ cd build_normal/ $ ../configure --enable-cross-compile --cross-prefix=mips-linux- --target- os=linux --arch=mips --cpu=74kc --disable-all install prefix /usr/local source path /opt/ffmpeg C compiler mips-linux-gcc C library uclibc host C compiler gcc host C library glibc ARCH mips (74kc) big-endian yes runtime cpu detection yes MIPS FPU enabled no MIPS DSP R1 enabled yes MIPS DSP R2 enabled yes MIPS MSA enabled no LOONGSON MMI enabled no debug symbols yes strip symbols yes optimize for size no optimizations yes static yes shared no postprocessing support no network support no threading support pthreads safe bitstream reader yes texi2html enabled no perl enabled yes pod2man enabled yes makeinfo enabled no makeinfo supports HTML no External libraries: alsa bzlib zlib External libraries providing hardware acceleration: Libraries: avutil Programs: Enabled decoders: Enabled encoders: Enabled hwaccels: Enabled parsers: Enabled demuxers: Enabled muxers: Enabled protocols: Enabled filters: Enabled bsfs: Enabled indevs: Enabled outdevs: License: LGPL version 2.1 or later WARNING: mips-linux-pkg-config not found, library detection may fail. $ cd ../build_lto $ CFLAGS='-flto' ../configure --enable-cross-compile --cross-prefix=mips- linux- --target-os=linux --arch=mips --cpu=74kc --disable-all --enable-lto install prefix /usr/local source path /opt/ffmpeg C compiler mips-linux-gcc C library uclibc host C compiler gcc host C library glibc ARCH mips (74kc) big-endian no runtime cpu detection yes MIPS FPU enabled no MIPS DSP R1 enabled yes MIPS DSP R2 enabled yes MIPS MSA enabled no LOONGSON MMI enabled yes debug symbols yes strip symbols yes optimize for size no optimizations yes static yes shared no postprocessing support no network support no threading support pthreads safe bitstream reader yes texi2html enabled no perl enabled yes pod2man enabled yes makeinfo enabled no makeinfo supports HTML no External libraries: alsa bzlib zlib External libraries providing hardware acceleration: Libraries: avutil Programs: Enabled decoders: Enabled encoders: Enabled hwaccels: Enabled parsers: Enabled demuxers: Enabled muxers: Enabled protocols: Enabled filters: Enabled bsfs: Enabled indevs: Enabled outdevs: License: LGPL version 2.1 or later WARNING: mips-linux-pkg-config not found, library detection may fail. $ uname -a Linux 7d04a811e585 4.20.13-200.fc29.x86_64 #1 SMP Wed Feb 27 19:42:55 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux $ mips-linux-gcc -v Using built-in specs. COLLECT_GCC=/opt/buildroot_wdr3600/output/host/bin/mips-linux-gcc.br_real COLLECT_LTO_WRAPPER=/opt/buildroot_wdr3600/output/host/libexec/gcc/mips- buildroot-linux-uclibc/7.4.0/lto-wrapper Target: mips-buildroot-linux-uclibc Configured with: ./configure --prefix=/opt/buildroot_wdr3600/output/host --sysconfdir=/opt/buildroot_wdr3600/output/host/etc --enable-static --disable-libstdcxx-dual-abi --disable-symvers --target=mips-buildroot- linux-uclibc --with-sysroot=/opt/buildroot_wdr3600/output/host/mips- buildroot-linux-uclibc/sysroot --enable-__cxa_atexit --with-gnu-ld --disable-libssp --disable-multilib --disable-decimal-float --with- gmp=/opt/buildroot_wdr3600/output/host --with- mpc=/opt/buildroot_wdr3600/output/host --with- mpfr=/opt/buildroot_wdr3600/output/host --with-pkgversion='Buildroot 2019.02-00010-g07502cdd25-dirty' --with-bugurl=http://bugs.buildroot.net/ --disable-libquadmath --disable-libsanitizer --enable-tls --enable-plugins --enable-lto --disable-libmudflap --enable-threads --without-isl --without-cloog --with-float=soft --with-arch=mips32r2 --with-abi=32 --with-nan=legacy --enable-languages=c,c++ --with-build-time- tools=/opt/buildroot_wdr3600/output/host/mips-buildroot-linux-uclibc/bin --enable-shared --disable-libgomp Thread model: posix gcc version 7.4.0 (Buildroot 2019.02-00010-g07502cdd25-dirty) $ }}} I really don't think it's OK for ffmpeg to just say "it's the user's fault for using CFLAGS='-flto'" ("enable LTO behind FFmpegs back"). At the very least the configure scripts needs to notice the situation and report failure. Notice the "LOONGSON MMI" detection is also broken when using -flto. -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:7> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: Type: enhancement | Status: closed Priority: normal | Component: build | system Version: unspecified | Resolution: invalid Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by cehoyos): * status: reopened => closed * resolution: => invalid Comment: Replying to [comment:7 RedDwarf82]:
{{{ $ CFLAGS='-flto' ../configure --enable-cross-compile --cross-prefix =mips-linux- --target-os=linux --arch=mips --cpu=74kc --disable-all --enable-lto }}} Thank you for the explanation! CFLAGS (and `--extra-cflags`) can make the configure line invalid, in general this cannot be detected so it makes little sense to detect it in some exceptional cases.
-- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:8> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#7790: Endianness detection breaks when LTO is enabled through means other than the ffmpeg --enable-lto option -------------------------------------+------------------------------------- Reporter: RedDwarf82 | Owner: (none) Type: enhancement | Status: closed Priority: normal | Component: build | system Version: unspecified | Resolution: invalid Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by RedDwarf): FWIW nowadays LTO can have different options. Just from gcc/clang: -flto -flto=auto -flto=thin -flto=4 And by not letting the user specify LTO in the flags he is losing access to them. -- Ticket URL: <https://trac.ffmpeg.org/ticket/7790#comment:9> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
participants (1)
-
FFmpeg