On 11 October 2013 09:30, Gavin Lambert
On 10/10/2013 3:07 PM, Quoth Roger Sanders:
The only compiler requirement here is that each assembly uses a compatible
C++ ABI, or at least, as much as is required to allow calling a virtual member function on an object that was constructed within another assembly.
That could be a steep requirement, since AFAIK vtable location and layout isn't standardised, especially once you start getting into multiple and virtual inheritance. And name mangling can get in the way of finding the things to be called in the first place.
I'd love to have something that didn't rely on ABI compatibility, but I don't think that's really going to be possible, or at least, not while adhering to the portability requirements of a boost library. With the new standard layout type definition in C++11, and with the help of the new alignment querying/control in C++11, it's now finally possible to produce a data structure that you can create in memory and guarantee you can correctly read in code compiled under a different compiler, without relying on undefined or implementation-specific behaviour. Unfortunately, while sharing data is nice, C++11 has done nothing to help with the bigger issue of sharing code. Using the extern keyword with a linkage specification (IE, as in the extern "C" usage) is too limited and tries to combine a few different concepts together, and as a result, ends up doing a poor job at all of them. What we really need is a new keyword, separate to the extern keyword, that is specifically intended to specify a platform-specific function "format", IE, the calling convention, that the compiler should use for that function. In particular, as opposed to the extern keyword, it should: 1. Be able to be applied on member functions as well as non-member functions 2. Be able to be used on overloaded functions 3. Not affect function linkage when used (IE, doesn't mark a function as having external linkage) With a new keyword that meets this requirement, you'd be able to actually mark a given function as explicitly having a particular calling convention. As long as two compilers support a compatible calling convention, you could then safely invoke that function from code compiled using another compiler, or even another language. The calling convention is still "implementation specific", as it has to be, since calling conventions are by their very nature platform specific, but it's now supported by the language, with compile-time checking for compatibility. With support for calling an individual function with compile-time checking for calling convention compatibility, you could then just provide a base class which meets the requirements of a standard layout type, and contains function pointers to each exposed function, with some inline wrapper functions to invoke them. This would give identical usage syntax to calling the methods natively, fully checked for compatibility at compile time, with behaviour guaranteed by the C++ standard, without requiring ABI compatibility. Without this kind of an enhancement to the language itself, I don't think it's really going to be a satisfying result attempting to provide any kind of boost interop library that attempts to provide interop without assuming some level of ABI compatibility. I am optimistic on another front though. The simple fact is, for any given platform, there's a limited number of C++ compilers people actually use, and most of them attempt a degree of compatibility anyway. The biggest problem is really on x86/x64 systems, with GCC and MSVC being incompatible. Clang is rapidly developing though, and I've seen there's a push for Clang to have full ABI support for both GCC and MSVC, IE, as a compiler switch to select between the two. Once we reach that point, it really just becomes a matter of any given software system selecting a base ABI to use, and Clang can then be used regardless of that choice to compile compatible code for that system. With this kind of solution in place, you could then just write code relying on ABI compatibility, and implement a language-supported solution when one becomes available (IE, just by marking all functions on an exposed interface with the kind of keyword I've proposed above). Whatever you attempt with ABI compatibility though, you still need to solve the STL issue as a higher-level problem though, which is what I'm proposing to add to boost at this stage.
And memory management is always entertaining given that all sorts of weird and different allocators can be in use even before you introduce a different compiler into the mix. (Though this is where shared_ptr and unique_ptr's pointer+deleter concept can get you out of a jam, although it's more common to use opaque handles and explicit destroy functions.)
I may be able to package up something for boost on the memory management side that can help too actually. In my work, we had the issue of wanting to be able to create an instance of a type, where only a base interface of that type was known to external code, with the actual implementation being completely internal to a particular module. We wanted to be able to easily create these types without having to specifically call allocators and deallocators (especially the deallocators, to avoid memory leaks), and we needed to be able to pass these types back from function calls, again, without the caller having to manually deallocate them. Something like shared_ptr was unsuitable though, not only because it felt too heavy-handed (we didn't need or want the reference counting), but also because it couldn't be passed safely across DLL boundaries either. My solution was a very thin "pointer" type, which basically just referenced the allocator and deallocator within its constructor and destructor, and used C++11 move semantics to allow returning them easily from function calls. The final version was macro-based to automatically generate these pointer types for any interface type, with only a couple of macro calls to create all the necessary code, including the allocators and deallocators. This pointer system, along with the STL interop code, are the basis for our communication model we use between components, and the payoff is very significant. We have easy bi-directional communication across pure virtual interfaces, with STL objects being shared natively on those interfaces, with minimal overhead and without ever having to manually call an allocator or deallocator, or manually unpack or repack an STL container. You can write code which creates a type and calls functions on that type, which is actually calling an allocator from a different module and invoking functions over a pure virtual interface for that type, with STL objects being shared, that looks identical to creating that type on the stack and calling functions on that type as if it was defined within the same module, with a compatible STL implementation. I could have another look at this code and see if it could be improved. I think it should be possible to build a more general fully template-based solution that doesn't rely on macros, in which case it might be a worthwhile addition along with the STL marshalling classes.