Visibility implementation
The attached patch adds visibility information for lots of symbols in libavcodec. I used the following macros to mark visibility: HIDDEN Used in a declaration or definition to indicate that the symbol is internal to the currently compiled library. extern HIDDEN int libavcodec_internal_int; HIDDEN void libavcodec_internal_function(void); START_HIDDEN_VISIBILITY_SECTION END_VISIBILITY_SECTION Declarations and definitions between these 2 macros have hidden visibility by default. EXTERNALLY_VISIBLE Used in a declaration/definition between the above 2 macros to indicate that some symbol should NOT have hidden visibility, but should behave as normal. These macros are defined in libavutil/internal.h. Currently they're defined unconditionally; a compiler check should be added for production code. Some other names for the macros could also be used. Visibility information matters in 2 cases: when a symbol is defined (to make the compiler place it in the correct section) and when any other code accesses it (knowing that a symbol has hidden visibility helps avoid the overhead of -fPIC). If a declaration which marks the symbol hidden is visible then it is not necessary to indicate the visibility information again where the symbol is defined. So in most cases marking the declarations in internal headers hidden is enough (if the file defining the symbol and all files using it include the header). Most of the patch is adding START_HIDDEN_VISIBILITY_SECTION at the start and END_VISIBILITY_SECTION at the end of various internal headers. If some .c file has a global definition which does not appear in a public header, and another file has its own internal declaration for that symbol, then it's necessary to mark those hidden separately. See ff_flac_compute_autocorr for an example of this in the patch. When adding START_HIDDEN_VISIBILITY_SECTION/END_VISIBILITY_SECTION in headers one thing to watch out for is to make sure you don't put it around #include statements. Other headers included between those macros would also use hidden visibility by default, which would likely lead to breakage. The following symbols were used outside libavcodec and caused compilation errors when everything in their corresponding headers was declared hidden: ff_cropTbl (used in vhook), ff_frame_rate_tab, ff_mpa_decode_header, ff_mpa_bitrate_tab, ff_mpa_freq_tab, ff_find_start_code (used in libavformat). I added the EXTERNALLY_VISIBLE macro to their definitions to turn them back to publicly available outside the library. In my test compilation this patch lowered the amount of public symbols in libavcodec.so that are exported for use by external programs from 1100 to 378 (counted with "nm -D libavcodec.so | grep -c '^[^ ]'"). Even though I didn't add HIDDEN macros to the various xxx_decoder, xxx_encoder etc definitions referring to them as external hidden symbols in allcodecs.o seems to be enough to prevent the linker from making them publicly available in the final library.
Uoti Urpala wrote:
In my test compilation this patch lowered the amount of public symbols in libavcodec.so that are exported for use by external programs from 1100 to 378 (counted with "nm -D libavcodec.so | grep -c '^[^ ]'"). Even though I didn't add HIDDEN macros to the various xxx_decoder, xxx_encoder etc definitions referring to them as external hidden symbols in allcodecs.o seems to be enough to prevent the linker from making them publicly available in the final library.
But your point was your method allow compiler optimisation. Did you see any code change with your attribute patch ? Matthieu
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote:
Uoti Urpala wrote:
In my test compilation this patch lowered the amount of public symbols in libavcodec.so that are exported for use by external programs from 1100 to 378 (counted with "nm -D libavcodec.so | grep -c '^[^ ]'"). Even though I didn't add HIDDEN macros to the various xxx_decoder, xxx_encoder etc definitions referring to them as external hidden symbols in allcodecs.o seems to be enough to prevent the linker from making them publicly available in the final library.
But your point was your method allow compiler optimisation. Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote:
Uoti Urpala wrote:
In my test compilation this patch lowered the amount of public symbols in libavcodec.so that are exported for use by external programs from 1100 to 378 (counted with "nm -D libavcodec.so | grep -c '^[^ ]'"). Even though I didn't add HIDDEN macros to the various xxx_decoder, xxx_encoder etc definitions referring to them as external hidden symbols in allcodecs.o seems to be enough to prevent the linker from making them publicly available in the final library.
But your point was your method allow compiler optimisation. Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
On which architecture? How did it compare to vanilla code without -fPIC? -- M?ns Rullg?rd mans at mansr.com
On Wed, 2008-07-30 at 15:40 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote:
Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
On which architecture? How did it compare to vanilla code without -fPIC?
I think it should change the resulting code on any architecture, but exactly how differs. On AMD64 hidden visibility should make most code identical to the version without -fPIC; as accesses are PC-relative anyway using -fPIC adds no indirection overhead when you know that the symbol will be in the same library, at a constant offset from the current position. On x86 -fPIC will still add overhead as there are no PC-relative addressing modes and it reserves one of the few registers. Hidden visibility does reduce the amount of indirection required however. Here's an example of gcc-4.3 output with and without hidden visibility. Program: ----- void g(int i); extern int ii; void f(int a) { g(a + ii); } ----- x86, no -fPIC: movl 4(%esp), %eax addl ii, %eax movl %eax, 4(%esp) jmp g x86, -fPIC with g and ii marked hidden: call __i686.get_pc_thunk.cx addl $_GLOBAL_OFFSET_TABLE_, %ecx movl 4(%esp), %eax addl ii at GOTOFF(%ecx), %eax movl %eax, 4(%esp) jmp g x86, -fPIC without visibility: pushl %ebx call __i686.get_pc_thunk.bx addl $_GLOBAL_OFFSET_TABLE_, %ebx subl $8, %esp movl 16(%esp), %eax movl ii at GOT(%ebx), %edx addl (%edx), %eax movl %eax, (%esp) call g at PLT addl $8, %esp popl %ebx ret AMD64, -fPIC with g and ii marked hidden: (same as without -fPIC) addl ii(%rip), %edi jmp g AMD64, -fPIC without visibility: movq ii at GOTPCREL(%rip), %rax addl (%rax), %edi jmp g at PLT
Uoti Urpala wrote:
On Wed, 2008-07-30 at 15:40 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote:
Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
On which architecture? How did it compare to vanilla code without -fPIC?
I think it should change the resulting code on any architecture, but exactly how differs.
Any benchmarks? If it is faster than the fastest alternative on each architecture, I see no point in pursuing this. I know there are people who find some kind of twisted pleasure in obfuscating perfectly good code with non-standard hacks and directives, but I'm not one of them. -- M?ns Rullg?rd mans at mansr.com
M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 15:40 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote:
Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
On which architecture? How did it compare to vanilla code without -fPIC?
I think it should change the resulting code on any architecture, but exactly how differs.
Any benchmarks? If it is faster than the fastest alternative on each
^not
architecture, I see no point in pursuing this. I know there are people who find some kind of twisted pleasure in obfuscating perfectly good code with non-standard hacks and directives, but I'm not one of them.
-- M?ns Rullg?rd mans at mansr.com
On Wed, 2008-07-30 at 16:54 +0100, M?ns Rullg?rd wrote:
M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 15:40 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote:
Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
On which architecture? How did it compare to vanilla code without -fPIC?
I think it should change the resulting code on any architecture, but exactly how differs.
Any benchmarks?
No.
If it is faster than the fastest alternative on each
^not
architecture, I see no point in pursuing this.
It makes the code faster when compiling with -fPIC, the question is only whether or in which cases the speedup is significant. I'm not sure what "alternatives" you're talking about. Code improvements are not the only benefit. Currently libavcodec.so exports 1100 symbols for use by external programs, most of which it shouldn't. The patch I posted cuts this to about one third, and completing the visibility markup would make it fall further. That difference exists whether the library is compiled with -fPIC or not.
I know there are people who find some kind of twisted pleasure in obfuscating perfectly good code with non-standard hacks and directives, but I'm not one of them.
Visibility is fairly standard library functionality nowadays.
Uoti Urpala wrote:
On Wed, 2008-07-30 at 16:54 +0100, M?ns Rullg?rd wrote:
M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 15:40 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote: > Did you see any code change with your attribute patch ?
Yes, if compiling with -fPIC.
On which architecture? How did it compare to vanilla code without -fPIC?
I think it should change the resulting code on any architecture, but exactly how differs.
Any benchmarks?
No.
Come back again when you have some.
If it is faster than the fastest alternative on each
^not
architecture, I see no point in pursuing this.
It makes the code faster when compiling with -fPIC, the question is only whether or in which cases the speedup is significant. I'm not sure what "alternatives" you're talking about.
Using static libraries, building without -fPIC (where possible), building with -fPIC.
Code improvements are not the only benefit. Currently libavcodec.so exports 1100 symbols for use by external programs, most of which it shouldn't. The patch I posted cuts this to about one third, and completing the visibility markup would make it fall further. That difference exists whether the library is compiled with -fPIC or not.
1. Does it matter? 2. Static libs still export all the stuff, so any namespace arguments are irrelevant. 3. Selectively exporting symbols can be done more easily, and on more platforms, with linker tricks.
I know there are people who find some kind of twisted pleasure in obfuscating perfectly good code with non-standard hacks and directives, but I'm not one of them.
Visibility is fairly standard library functionality nowadays.
If by standard, you mean gcc, yes. -- M?ns Rullg?rd mans at mansr.com
On Wed, 2008-07-30 at 17:45 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
It makes the code faster when compiling with -fPIC, the question is only whether or in which cases the speedup is significant. I'm not sure what "alternatives" you're talking about.
Using static libraries, building without -fPIC (where possible), building with -fPIC.
Static libraries are often not a good alternative for other reasons. Using shared libraries compiled without -fPIC may or may not be a reasonable alternative depending on the platform.
Code improvements are not the only benefit. Currently libavcodec.so exports 1100 symbols for use by external programs, most of which it shouldn't. The patch I posted cuts this to about one third, and completing the visibility markup would make it fall further. That difference exists whether the library is compiled with -fPIC or not.
1. Does it matter?
Yes somewhat. Perhaps not so much as to be worth doing a huge amount of work, but adding two marker lines per header isn't that much work or complication either.
2. Static libs still export all the stuff, so any namespace arguments are irrelevant.
Not really. Static libs are a special enough use case that whatever happens with them doesn't necessarily have much relevance to most use.
3. Selectively exporting symbols can be done more easily, and on more platforms, with linker tricks.
I don't think it would be significantly easier. Adding two lines per header isn't that complicated, and the special cases would need manual handling in any other implementation too.
I know there are people who find some kind of twisted pleasure in obfuscating perfectly good code with non-standard hacks and directives, but I'm not one of them.
Visibility is fairly standard library functionality nowadays.
If by standard, you mean gcc, yes.
Fairly standard as in something normal to library developers; concentrating on the compiler used to achieve it is misleading. Of course most of library development and compilation on UNIX by far is done with GCC. ICC supports the same syntax. On Windows Microsoft compilers have their own syntax to express DLL visibility, and projects supporting VC can map the same macros to that. I think there's little reason to oppose visibility as a "nonstandard compiler-specific feature" or anything like that. It's not a very old feature (at least in active use) compared to the most basic stuff and so there are still platforms without support for it - mainly those that see little development but aren't quite dead yet - and the exact syntax for it is not standardized. However it is already widely used and the usage will become more standard with time.
Uoti Urpala wrote:
On Wed, 2008-07-30 at 16:54 +0100, M?ns Rullg?rd wrote:
M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 15:40 +0100, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Wed, 2008-07-30 at 08:16 +0200, matthieu castet wrote: > Did you see any code change with your attribute patch ? Yes, if compiling with -fPIC. On which architecture? How did it compare to vanilla code without -fPIC? I think it should change the resulting code on any architecture, but exactly how differs. Any benchmarks?
No.
If it is faster than the fastest alternative on each
^not
architecture, I see no point in pursuing this.
It makes the code faster when compiling with -fPIC, the question is only whether or in which cases the speedup is significant. I'm not sure what "alternatives" you're talking about.
Code improvements are not the only benefit. Currently libavcodec.so exports 1100 symbols for use by external programs, most of which it shouldn't.
Is the resulting libs smaller when exporting less symbols and if so, how much smaller ? MvH Benjamin Larsson
On Wed, 2008-07-30 at 19:08 +0200, Benjamin Larsson wrote:
Uoti Urpala wrote:
Code improvements are not the only benefit. Currently libavcodec.so exports 1100 symbols for use by external programs, most of which it shouldn't.
Is the resulting libs smaller when exporting less symbols and if so, how much smaller ?
Most of the code can be called through other functions and so cannot be omitted completely from the library. So size in bytes in not much smaller, something like 1%.
On Wed, Jul 30, 2008 at 05:45:36AM +0300, Uoti Urpala wrote:
--- a/libavcodec/i386/dsputil_mmx.c +++ b/libavcodec/i386/dsputil_mmx.c @@ -788,7 +788,7 @@ static void draw_edges_mmx(uint8_t *buf, int wrap, int width, int height, int w)
#define PAETH(cpu, abs3)\ -void add_png_paeth_prediction_##cpu(uint8_t *dst, uint8_t *src, uint8_t *top, int w, int bpp)\ +static void add_png_paeth_prediction_##cpu(uint8_t *dst, uint8_t *src, uint8_t *top, int w, int bpp)\
Unrelated? Diego
On Wed, 2008-07-30 at 10:21 +0200, Diego Biurrun wrote:
On Wed, Jul 30, 2008 at 05:45:36AM +0300, Uoti Urpala wrote:
--- a/libavcodec/i386/dsputil_mmx.c +++ b/libavcodec/i386/dsputil_mmx.c @@ -788,7 +788,7 @@ static void draw_edges_mmx(uint8_t *buf, int wrap, int width, int height, int w)
#define PAETH(cpu, abs3)\ -void add_png_paeth_prediction_##cpu(uint8_t *dst, uint8_t *src, uint8_t *top, int w, int bpp)\ +static void add_png_paeth_prediction_##cpu(uint8_t *dst, uint8_t *src, uint8_t *top, int w, int bpp)\
Unrelated?
Not really - I changed some globally visible symbols to more limited visibility, and because that was only used in the same file I changed it to static instead of hidden.
Uoti Urpala wrote:
On Wed, 2008-07-30 at 10:21 +0200, Diego Biurrun wrote:
On Wed, Jul 30, 2008 at 05:45:36AM +0300, Uoti Urpala wrote:
--- a/libavcodec/i386/dsputil_mmx.c +++ b/libavcodec/i386/dsputil_mmx.c @@ -788,7 +788,7 @@ static void draw_edges_mmx(uint8_t *buf, int wrap, int width, int height, int w)
#define PAETH(cpu, abs3)\ -void add_png_paeth_prediction_##cpu(uint8_t *dst, uint8_t *src, uint8_t *top, int w, int bpp)\ +static void add_png_paeth_prediction_##cpu(uint8_t *dst, uint8_t *src, uint8_t *top, int w, int bpp)\ Unrelated?
Not really - I changed some globally visible symbols to more limited visibility, and because that was only used in the same file I changed it to static instead of hidden.
Maybe you could split this hunk so it will be committed first. lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used. One special case is the ff_inverse symbol. It's defined in dsputil.c but declared in libavutil/internal.h and used in the FASTDIV macro. I made the symbol hidden, which means FASTDIV will not work outside libavcodec as the symbol is not visible outside the library (it's not used elsewhere now). If it's supposed to be used for cases where speed matters then slower access to a global from another library probably isn't good. Does anyone have comments about the substance of the patch (something more than M?ns's general hostility towards visibility)? As for the effects on code, you can see from the generated asm that it does improve. I haven't tried any large benchmarks and don't know where access to globals would matter most (and so would be most worthwhile to benchmark for obvious differences). I did a small test with h264 video and h264.c compiled with -fPIC: times without visibility: 15.212 15.247 15.192 15.238 15.206 times with visibility: 15.115 15.082 15.114 15.067 15.062 The patch does make decoding consistently faster, but I don't think such benchmark results show much (except that it does have some effect on generated code). Random variation in how well the compiler happens to optimize a particular version of the code or how things happen to align in cache can be several percent.
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
One special case is the ff_inverse symbol. It's defined in dsputil.c but declared in libavutil/internal.h and used in the FASTDIV macro. I made the symbol hidden, which means FASTDIV will not work outside libavcodec as the symbol is not visible outside the library (it's not used elsewhere now). If it's supposed to be used for cases where speed matters then slower access to a global from another library probably isn't good.
Does anyone have comments about the substance of the patch (something more than M?ns's general hostility towards visibility)?
I have no technical objections to the patch, it does make -fPIC code faster, smaller and i suspect improves linking times. A technical objection would be if i knew a better solution but i do not, at least not with current gcc. If its -visibility=hidden was a little smarter, that is mark all symbols hidden unless they are in a header under /usr/... then all this could be done much nicer. But truth is gcc does not treat externs as hidden even with -visibility=hidden. thus one has no choice, if hidden is wanted pragma or attribute must be in the files ... Besides this i agree with mans that it is very ugly. Also as is the patch is missing configure changes. If someone does finish it and applies it, i wont complain, though i hope that one day gcc will be able to handle a default of hidden that is useable. Then we can implement it cleanly and drop this in case someone finished & commited it ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws. -- Plato
Michael Niedermayer wrote:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
Does anyone have comments about the substance of the patch (something more than M?ns's general hostility towards visibility)? It can help to do smaller shared library (I am not sure if the visibility stuff hide all internal symbol). But that's a start.
$ ./configure --enable-shared --extra-cflags="-ffunction-sections -fdata-sections" --extra-ldflags="-Wl,--gc-sections -Wl,--print-gc-sections" --disable-decoders --disable-encoders --disable-muxers $size */*.so text data bss dec hex filename 1164633 3576 32556 1200765 12527d libavcodec/libavcodec.so 20334 652 12 20998 5206 libavdevice/libavdevice.so 481499 9144 32 490675 77cb3 libavformat/libavformat.so 41078 300 13876 55254 d7d6 libavutil/libavutil.so 8155 388 8 8551 2167 vhook/drawtext.so 7135 384 16 7535 1d6f vhook/fish.so 2464 316 8 2788 ae4 vhook/null.so 5001 376 8 5385 1509 vhook/ppm.so 8026 384 8 8418 20e2 vhook/watermark.so $ ./configure --enable-shared --disable-decoders --disable-encoders --disable-muxers $size */*.so text data bss dec hex filename 1270473 3768 545612 1819853 1bc4cd libavcodec/libavcodec.so 20334 652 12 20998 5206 libavdevice/libavdevice.so 481643 9144 32 490819 77d43 libavformat/libavformat.so 41078 300 13876 55254 d7d6 libavutil/libavutil.so 8155 388 8 8551 2167 vhook/drawtext.so 7135 384 16 7535 1d6f vhook/fish.so 2464 316 8 2788 ae4 vhook/null.so 5001 376 8 5385 1509 vhook/ppm.so 8026 384 8 8418 20e2 vhook/watermark.so
I have no technical objections to the patch, it does make -fPIC code faster, smaller and i suspect improves linking times. A technical objection would be if i knew a better solution but i do not, at least not with current gcc. If its -visibility=hidden was a little smarter, that is mark all symbols hidden unless they are in a header under /usr/... then all this could be done much nicer.
Or use the difference between include <...> and include "..." Matthieu
matthieu castet wrote:
Michael Niedermayer wrote:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in Does anyone have comments about the substance of the patch (something more than M?ns's general hostility towards visibility)? It can help to do smaller shared library (I am not sure if the visibility stuff hide all internal symbol). But that's a start.
$ ./configure --enable-shared --extra-cflags="-ffunction-sections -fdata-sections" --extra-ldflags="-Wl,--gc-sections -Wl,--print-gc-sections" --disable-decoders --disable-encoders --disable-muxers $size */*.so text data bss dec hex filename 1164633 3576 32556 1200765 12527d libavcodec/libavcodec.so 20334 652 12 20998 5206 libavdevice/libavdevice.so 481499 9144 32 490675 77cb3 libavformat/libavformat.so 41078 300 13876 55254 d7d6 libavutil/libavutil.so 8155 388 8 8551 2167 vhook/drawtext.so 7135 384 16 7535 1d6f vhook/fish.so 2464 316 8 2788 ae4 vhook/null.so 5001 376 8 5385 1509 vhook/ppm.so 8026 384 8 8418 20e2 vhook/watermark.so
$ ./configure --enable-shared --disable-decoders --disable-encoders --disable-muxers $size */*.so text data bss dec hex filename 1270473 3768 545612 1819853 1bc4cd libavcodec/libavcodec.so 20334 652 12 20998 5206 libavdevice/libavdevice.so 481643 9144 32 490819 77d43 libavformat/libavformat.so 41078 300 13876 55254 d7d6 libavutil/libavutil.so 8155 388 8 8551 2167 vhook/drawtext.so 7135 384 16 7535 1d6f vhook/fish.so 2464 316 8 2788 ae4 vhook/null.so 5001 376 8 5385 1509 vhook/ppm.so 8026 384 8 8418 20e2 vhook/watermark.so
And for information with the same configuration, --gc-sections, -fvisibility=hidden + default pragma visibility in public header, I manage to a single library (containing libavcodec, libavdevice, ...) : text data bss dec hex filename 5153895 36364 2249092 7439351 7183f7 libavcodec/libavxxx.so So the current visibility stuff is far from covering everything. Matthieu
On Thu, 2008-07-31 at 21:53 +0200, matthieu castet wrote:
And for information with the same configuration, --gc-sections, -fvisibility=hidden + default pragma visibility in public header, I manage to a single library (containing libavcodec, libavdevice, ...) : text data bss dec hex filename 5153895 36364 2249092 7439351 7183f7 libavcodec/libavxxx.so
So the current visibility stuff is far from covering everything.
I'm not sure what you're trying to show with the numbers above. Anyway my patch only covers libavcodec and no other part of FFmpeg. There's still some stuff in libavcodec outside it, but less than one tenth of the symbols that should be hidden. More could be marked later but I think it's better to do that after the basic visibility support is in FFmpeg.
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing... Diego
Diego Biurrun <diego at biurrun.de> writes:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing...
diff --git a/libavutil/internal.h b/libavutil/internal.h index cc62d3b..6ed5c31 100644 --- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -50,6 +50,12 @@ #endif #endif
+#define START_HIDDEN_VISIBILITY_SECTION _Pragma("GCC visibility push(hidden)") +#define END_VISIBILITY_SECTION _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define EXTERNALLY_VISIBLE __attribute__((visibility("default"))) +
These should be under appropriate ifdefs. The names are awfully long too. -- M?ns Rullg?rd mans at mansr.com
On Sat, Oct 18, 2008 at 11:57:17AM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing...
--- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -50,6 +50,12 @@
+#define START_HIDDEN_VISIBILITY_SECTION _Pragma("GCC visibility push(hidden)") +#define END_VISIBILITY_SECTION _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define EXTERNALLY_VISIBLE __attribute__((visibility("default")))
These should be under appropriate ifdefs. The names are awfully long too.
START_HIDDEN_VISIBILITY END_HIDDEN_VISIBILITY ? Diego
Diego Biurrun <diego at biurrun.de> writes:
On Sat, Oct 18, 2008 at 11:57:17AM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing...
--- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -50,6 +50,12 @@
+#define START_HIDDEN_VISIBILITY_SECTION _Pragma("GCC visibility push(hidden)") +#define END_VISIBILITY_SECTION _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define EXTERNALLY_VISIBLE __attribute__((visibility("default")))
These should be under appropriate ifdefs. The names are awfully long too.
START_HIDDEN_VISIBILITY END_HIDDEN_VISIBILITY
What about this: +#define START_HIDDEN _Pragma("GCC visibility push(hidden)") +#define END_HIDDEN _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define VISIBLE __attribute__((visibility("default"))) Just because a feature was designed to work around C++ bloat, we don't have to adopt their bloated naming style when we use it. -- M?ns Rullg?rd mans at mansr.com
On Sun, Oct 19, 2008 at 01:48:10PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Sat, Oct 18, 2008 at 11:57:17AM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote:
The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing...
--- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -50,6 +50,12 @@
+#define START_HIDDEN_VISIBILITY_SECTION _Pragma("GCC visibility push(hidden)") +#define END_VISIBILITY_SECTION _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define EXTERNALLY_VISIBLE __attribute__((visibility("default")))
These should be under appropriate ifdefs. The names are awfully long too.
START_HIDDEN_VISIBILITY END_HIDDEN_VISIBILITY
What about this:
+#define START_HIDDEN _Pragma("GCC visibility push(hidden)") +#define END_HIDDEN _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define VISIBLE __attribute__((visibility("default")))
Just because a feature was designed to work around C++ bloat, we don't have to adopt their bloated naming style when we use it.
Updated version attached. Diego
Diego Biurrun <diego at biurrun.de> writes:
On Sun, Oct 19, 2008 at 01:48:10PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Sat, Oct 18, 2008 at 11:57:17AM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Thu, Jul 31, 2008 at 08:03:50PM +0300, Uoti Urpala wrote:
On Wed, 2008-07-30 at 05:45 +0300, Uoti Urpala wrote: > The attached patch adds visibility information for lots of symbols in
I created a new version that adds visibility information for most libavcodec symbols. This time I split the patch into two parts. The first one has the repeated changes per internal header marking them as such. The second adds visibility information for some of the symbols that are not static but not declared in any header either, adds exceptions for some symbols which are declared in libavcodec headers but used outside it (mainly in libavformat), and defines the macros used.
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing...
--- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -50,6 +50,12 @@
+#define START_HIDDEN_VISIBILITY_SECTION _Pragma("GCC visibility push(hidden)") +#define END_VISIBILITY_SECTION _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define EXTERNALLY_VISIBLE __attribute__((visibility("default")))
These should be under appropriate ifdefs. The names are awfully long too.
START_HIDDEN_VISIBILITY END_HIDDEN_VISIBILITY
What about this:
+#define START_HIDDEN _Pragma("GCC visibility push(hidden)") +#define END_HIDDEN _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define VISIBLE __attribute__((visibility("default")))
Just because a feature was designed to work around C++ bloat, we don't have to adopt their bloated naming style when we use it.
Updated version attached.
Diego
Index: libavutil/internal.h =================================================================== --- libavutil/internal.h (revision 15672) +++ libavutil/internal.h (working copy) @@ -50,6 +50,18 @@ #endif #endif
+#ifdef CONFIG_VISIBILITY
Where do you expect this to get set? Is there any reason to disable it when the compiler supports it? I would expect to see some compiler version ifdeffery or a configure check setting HAVE_ATTR_VISIBILITY or similar. Just VISIBILITY is a bit too vague a name for the overall control symbol, IMHO.
+#define START_HIDDEN _Pragma("GCC visibility push(hidden)") +#define END_HIDDEN _Pragma("GCC visibility pop") +#define HIDDEN __attribute__((visibility("hidden"))) +#define VISIBLE __attribute__((visibility("default"))) +#else +#define START_HIDDEN +#define END_HIDDEN +#define HIDDEN +#define VISIBLE +#endif + #ifndef INT16_MIN #define INT16_MIN (-0x7fff-1) #endif @@ -124,7 +136,7 @@
/* math */
-extern const uint32_t ff_inverse[256]; +extern HIDDEN const uint32_t ff_inverse[256];
#if defined(ARCH_X86) # define FASTDIV(a,b) \ Index: libavcodec/mjpeg.h =================================================================== --- libavcodec/mjpeg.h (revision 15672) +++ libavcodec/mjpeg.h (working copy) @@ -37,6 +37,9 @@ #include "bitstream.h"
+START_HIDDEN + + /* JPEG marker codes */ typedef enum { /* start of frame */ @@ -152,4 +155,7 @@ const uint8_t *bits_table, const uint8_t *val_table);
+ +END_HIDDEN + #endif /* AVCODEC_MJPEG_H */
Why so many blank lines? -- M?ns Rullg?rd mans at mansr.com
On Thu, Oct 23, 2008 at 09:25:35PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Sun, Oct 19, 2008 at 01:48:10PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
On Sat, Oct 18, 2008 at 11:57:17AM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
Here is a new version of those patches, updated for HEAD. Uoti, maybe you can doublecheck nothing is missing...
--- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -50,6 +50,12 @@
+#define START_HIDDEN_VISIBILITY_SECTION _Pragma("GCC visibility push(hidden)") +#define END_VISIBILITY_SECTION _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define EXTERNALLY_VISIBLE __attribute__((visibility("default")))
These should be under appropriate ifdefs. The names are awfully long too.
START_HIDDEN_VISIBILITY END_HIDDEN_VISIBILITY
What about this:
+#define START_HIDDEN _Pragma("GCC visibility push(hidden)") +#define END_HIDDEN _Pragma("GCC visibility pop") + +#define HIDDEN __attribute__((visibility("hidden"))) +#define VISIBLE __attribute__((visibility("default")))
Just because a feature was designed to work around C++ bloat, we don't have to adopt their bloated naming style when we use it.
Updated version attached.
--- libavutil/internal.h (revision 15672) +++ libavutil/internal.h (working copy) @@ -50,6 +50,18 @@
+#ifdef CONFIG_VISIBILITY
Where do you expect this to get set? Is there any reason to disable it when the compiler supports it? I would expect to see some compiler version ifdeffery or a configure check setting HAVE_ATTR_VISIBILITY or similar.
ATM it would have to be set manually in config.h, i.e. configure support is still missing. I'll see if I implement it in the next round.
Just VISIBILITY is a bit too vague a name for the overall control symbol, IMHO.
OK, I will change that.
--- libavcodec/mjpeg.h (revision 15672) +++ libavcodec/mjpeg.h (working copy) @@ -37,6 +37,9 @@ #include "bitstream.h"
+START_HIDDEN + + /* JPEG marker codes */ typedef enum { /* start of frame */ @@ -152,4 +155,7 @@ const uint8_t *bits_table, const uint8_t *val_table);
+ +END_HIDDEN + #endif /* AVCODEC_MJPEG_H */
Why so many blank lines?
Why not? Diego
On Fri, Oct 24, 2008 at 10:30:55AM +0200, Diego Biurrun wrote:
On Thu, Oct 23, 2008 at 09:25:35PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
Updated version attached.
--- libavutil/internal.h (revision 15672) +++ libavutil/internal.h (working copy) @@ -50,6 +50,18 @@
+#ifdef CONFIG_VISIBILITY
Where do you expect this to get set? Is there any reason to disable it when the compiler supports it? I would expect to see some compiler version ifdeffery or a configure check setting HAVE_ATTR_VISIBILITY or similar.
ATM it would have to be set manually in config.h, i.e. configure support is still missing. I'll see if I implement it in the next round.
OK, here is the next version, along with configure support. This should be fit for committing. Diego
On Mon, 2008-10-27 at 21:52 +0100, Diego Biurrun wrote:
OK, here is the next version, along with configure support. This should be fit for committing.
Is there a reason why the START_HIDDEN stuff is under #ifdef CONFIG_VISIBILITY but attribute_hidden is not?
On Tue, Oct 28, 2008 at 11:52:15AM +0200, Uoti Urpala wrote:
On Mon, 2008-10-27 at 21:52 +0100, Diego Biurrun wrote:
OK, here is the next version, along with configure support. This should be fit for committing.
Is there a reason why the START_HIDDEN stuff is under #ifdef CONFIG_VISIBILITY but attribute_hidden is not?
I figured that I could treat all attribute_ things equally. I have no strong opinion on this. Diego
On Fri, 2008-10-31 at 17:33 +0100, Diego Biurrun wrote:
On Tue, Oct 28, 2008 at 11:52:15AM +0200, Uoti Urpala wrote:
On Mon, 2008-10-27 at 21:52 +0100, Diego Biurrun wrote:
OK, here is the next version, along with configure support. This should be fit for committing.
Is there a reason why the START_HIDDEN stuff is under #ifdef CONFIG_VISIBILITY but attribute_hidden is not?
I figured that I could treat all attribute_ things equally. I have no strong opinion on this.
The START/END_HIDDEN macros are a shorthand for adding the visibility("hidden") attribute to every declaration and definition between them. Having the configure switch affect symbols using one syntax but not those using another seems wrong - it could only make sense if there was some convention what kind of symbols to mark with which syntax, but so far there isn't.
Diego Biurrun wrote:
On Fri, Oct 24, 2008 at 10:30:55AM +0200, Diego Biurrun wrote:
On Thu, Oct 23, 2008 at 09:25:35PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
Updated version attached.
--- libavutil/internal.h (revision 15672) +++ libavutil/internal.h (working copy) @@ -50,6 +50,18 @@
+#ifdef CONFIG_VISIBILITY
Where do you expect this to get set? Is there any reason to disable it when the compiler supports it? I would expect to see some compiler version ifdeffery or a configure check setting HAVE_ATTR_VISIBILITY or similar.
ATM it would have to be set manually in config.h, i.e. configure support is still missing. I'll see if I implement it in the next round.
OK, here is the next version, along with configure support. This should be fit for committing.
I don't like --enable-visiblity. Especially for something video-related its meaning is not at all clear. Call it --enable-symbol-hiding or similar instead. You're still adding insane amounts of whitespace around the section start/stop lines. Why? -- M?ns Rullg?rd mans at mansr.com
On Tue, Oct 28, 2008 at 04:59:24PM +0000, M?ns Rullg?rd wrote:
Diego Biurrun wrote:
On Fri, Oct 24, 2008 at 10:30:55AM +0200, Diego Biurrun wrote:
On Thu, Oct 23, 2008 at 09:25:35PM +0100, M?ns Rullg?rd wrote:
Diego Biurrun <diego at biurrun.de> writes:
Updated version attached.
--- libavutil/internal.h (revision 15672) +++ libavutil/internal.h (working copy) @@ -50,6 +50,18 @@
+#ifdef CONFIG_VISIBILITY
Where do you expect this to get set? Is there any reason to disable it when the compiler supports it? I would expect to see some compiler version ifdeffery or a configure check setting HAVE_ATTR_VISIBILITY or similar.
ATM it would have to be set manually in config.h, i.e. configure support is still missing. I'll see if I implement it in the next round.
OK, here is the next version, along with configure support. This should be fit for committing.
I don't like --enable-visiblity. Especially for something video-related its meaning is not at all clear. Call it --enable-symbol-hiding or similar instead.
Done.
You're still adding insane amounts of whitespace around the section start/stop lines. Why?
Very well, here is a version without so much whitespace. Diego
Diego Biurrun <diego at biurrun.de> writes:
Diego Index: configure =================================================================== --- configure (revision 15760) +++ configure (working copy) @@ -148,6 +148,7 @@ echo " --enable-small optimize for size instead of speed" echo " --enable-hardcoded-tables use hardcoded tables instead of runtime generation" echo " --enable-memalign-hack emulate memalign, interferes with memory debuggers" + echo " --enable-symbol hiding enable symbol hiding support"
s/enable-symbol hiding/enable-symbol_hiding/ -- Gruesse/greetings, Reinhard Tartler, KeyID 945348A4
Reinhard Tartler <siretart at tauware.de> writes:
Diego Biurrun <diego at biurrun.de> writes:
Diego Index: configure =================================================================== --- configure (revision 15760) +++ configure (working copy) @@ -148,6 +148,7 @@ echo " --enable-small optimize for size instead of speed" echo " --enable-hardcoded-tables use hardcoded tables instead of runtime generation" echo " --enable-memalign-hack emulate memalign, interferes with memory debuggers" + echo " --enable-symbol hiding enable symbol hiding support"
s/enable-symbol hiding/enable-symbol_hiding/
--enable-symbol-hiding Hyphens and underscores are equivalent here, and I think hyphens look nicer. -- M?ns Rullg?rd mans at mansr.com
participants (8)
-
banan@ludd.ltu.se -
castet.matthieu@free.fr -
diego@biurrun.de -
lu_zero@gentoo.org -
mans@mansr.com -
michaelni@gmx.at -
siretart@tauware.de -
uoti.urpala@pp1.inet.fi