[Flyweight] Hidden symbol visibility
Hi there, I recently made some interesting observation regarding Boost.Flyweight and (hidden) symbol visibility and have now some questions regarding this. Background story: ----------------- In our company's code base we have been using Boost.Flyweight for several years now, compiling it with GCC (and Clang) and it worked just fine. (Thanks for it!) However, lately we are trying to apply hidden symbol visibility to our code base and some of our unit-tests started failing mysteriously. After some intensive investigation I found out that it seems to be due to how we used Boost.Flyweight. We are instantiating it with some custom types, using the default `boost::flyweights::static_holder` class, and are compiling it into a shared library A. However, these Boost.Flyweight types are not only used within that shared library A but also in some other shared library B and in some unit-test executables which link against shared library A. (And these unit-tests started failing.) Observations: ------------- Reading the documentation of Boost.Flyweight and looking at the header `boost/flyweight/static_holder.hpp` I realized that we should have used `boost::flyweights::intermodule_holder` in the first place instead of `boost::flyweights::static_holder`, because we were using Boost.Flyweight from different shared libraries. Still, with non-hidden visibility applied everything always worked correctly even when using `boost::flyweights::static_holder`. Looking at the generated symbols in the different shared libraries I realized that these `static_holder` class types were marked as "unique global symbol" (an GNU extension to the standard set of ELF symbol bindings). So, all shared libraries and executables still used the same single instance of that `static_holder` class (which is the reason why everything worked). With hidden symbol visibility applied by default, these symbols are now marked as local symbols (from the BSS section). So, each shared library as well as the executables have their own version of the `static_holder` class types (which would explain why the equality-comparisions between different Boost.Flyweight variables in our unit-tests now started failing). If I understand it correctly, using `boost::flyweights::intermodule_holder` is the right thing to do, in particular with hidden symbol visibility. Questions: ---------- 1. However, apart from longer program startup times, what other disadvantages might it have to use the `intermodule_holder` instead of the `static_holder` (assuming it still would be marked as "unique global symbol")? 2. And if, as it seems, `static_holder` has some advantages in general, would it be possible to (maybe conditionally) mark the `static_holder` (and/or `static_holder_class`) class with `BOOST_SYMBOL_VISIBLE` or something similar to allow using it even with hidden symbol visibility from different shared libraries? - Only applying default visibility to the template instantiations is ignored by GCC (with warning: "type attributes ignored after type is already defined [-Wattributes]"). - Besides, the C++ standard does not allow applying C++ attributes to template instantiations. (There was a proposal for allowing it, wg21.link/p0537, but I do not know what happened to it.) 3. Would you have any other recommendations or suggestions I did not think about? Thanks for any help or opinion, Deniz -- BENOCS GmbH Dipl.-Inform. Deniz Bahadir Reuchlinstr. 10 D 10553 Berlin Germany Phone: +49 - 30 / 577 0004-22 Email: deniz.bahadir@benocs.com www.benocs.com Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese Commercial Register: Amtsgericht Bonn HRB 19378
El 07/02/2021 a las 16:36, Deniz Bahadir via Boost escribió:
[...]
In our company's code base we have been using Boost.Flyweight for several years now, compiling it with GCC (and Clang) and it worked just fine. (Thanks for it!)
However, lately we are trying to apply hidden symbol visibility to our code base and some of our unit-tests started failing mysteriously. After some intensive investigation I found out that it seems to be due to how we used Boost.Flyweight.
We are instantiating it with some custom types, using the default `boost::flyweights::static_holder` class, and are compiling it into a shared library A. However, these Boost.Flyweight types are not only used within that shared library A but also in some other shared library B and in some unit-test executables which link against shared library A. (And these unit-tests started failing.)
[...]
1. However, apart from longer program startup times, what other disadvantages might it have to use the `intermodule_holder` instead of the `static_holder` (assuming it still would be marked as "unique global symbol")?
2. And if, as it seems, `static_holder` has some advantages in general, would it be possible to (maybe conditionally) mark the `static_holder` (and/or `static_holder_class`) class with `BOOST_SYMBOL_VISIBLE` or something similar to allow using it even with hidden symbol visibility from different shared libraries? - Only applying default visibility to the template instantiations is ignored by GCC (with warning: "type attributes ignored after type is already defined [-Wattributes]"). - Besides, the C++ standard does not allow applying C++ attributes to template instantiations. (There was a proposal for allowing it, wg21.link/p0537, but I do not know what happened to it.) You can't make static_holder visible without changing Boost.Flyweight
None in principle, other than, as you say, longer startup times. intermodule_holder relies on boost::interprocess::ipcdetail::intermodule_singleton, which is notably complex. As for the advantages, intermodule_holder is compatible with Windows, where you don't have symbol visibility attributes. I guess this is not a concern to you. source code, but see below.
3. Would you have any other recommendations or suggestions I did not think about?
It's very easy to provide your own visible static holder: struct visible_static_holder:boost::flyweights::holder_marker { template<typename C> struct apply { struct type { BOOST_SYMBOL_VISIBLE static C& get() { static C c; return c; } }; }; }; ... boost::flyweightstd::string,visible_static_holder fw("hello"); Joaquín M López Muñoz
Am 08.02.21 um 10:05 schrieb Joaquin M López Muñoz via Boost:
El 07/02/2021 a las 16:36, Deniz Bahadir via Boost escribió:
[...]
In our company's code base we have been using Boost.Flyweight for several years now, compiling it with GCC (and Clang) and it worked just fine. (Thanks for it!)
However, lately we are trying to apply hidden symbol visibility to our code base and some of our unit-tests started failing mysteriously. After some intensive investigation I found out that it seems to be due to how we used Boost.Flyweight.
We are instantiating it with some custom types, using the default `boost::flyweights::static_holder` class, and are compiling it into a shared library A. However, these Boost.Flyweight types are not only used within that shared library A but also in some other shared library B and in some unit-test executables which link against shared library A. (And these unit-tests started failing.)
[...]
1. However, apart from longer program startup times, what other disadvantages might it have to use the `intermodule_holder` instead of the `static_holder` (assuming it still would be marked as "unique global symbol")?
None in principle, other than, as you say, longer startup times. intermodule_holder relies on boost::interprocess::ipcdetail::intermodule_singleton, which is notably complex.
As for the advantages, intermodule_holder is compatible with Windows, where you don't have symbol visibility attributes. I guess this is not a concern to you.
You guessed right. We are developing on and for Linux and do not really care for Windows compatibility.
2. And if, as it seems, `static_holder` has some advantages in general, would it be possible to (maybe conditionally) mark the `static_holder` (and/or `static_holder_class`) class with `BOOST_SYMBOL_VISIBLE` or something similar to allow using it even with hidden symbol visibility from different shared libraries? - Only applying default visibility to the template instantiations is ignored by GCC (with warning: "type attributes ignored after type is already defined [-Wattributes]"). - Besides, the C++ standard does not allow applying C++ attributes to template instantiations. (There was a proposal for allowing it, wg21.link/p0537, but I do not know what happened to it.) You can't make static_holder visible without changing Boost.Flyweight source code, but see below. 3. Would you have any other recommendations or suggestions I did not think about?
It's very easy to provide your own visible static holder:
struct visible_static_holder:boost::flyweights::holder_marker { template<typename C> struct apply { struct type { BOOST_SYMBOL_VISIBLE static C& get() { static C c; return c; } }; }; };
...
boost::flyweightstd::string,visible_static_holder fw("hello");
Thanks, I thought about doing it like so. I might give it a try. Although, if `intermodule_holder` really has no real runtime-disadvantage we might just switch to using it. (Some of my colleagues would prefer to not rely on "unique global symbols".)
Joaquín M López Muñoz
Thanks a lot, Joaquín. Deniz -- BENOCS GmbH Dipl.-Inform. Deniz Bahadir Reuchlinstr. 10 D 10553 Berlin Germany Phone: +49 - 30 / 577 0004-22 Email: deniz.bahadir@benocs.com www.benocs.com Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese Commercial Register: Amtsgericht Bonn HRB 19378
participants (2)
-
Deniz Bahadir -
Joaquin M López Muñoz