Boost Shmem-Shared memory library first archive available

Hello to all, As promised two weeks ago, I have a first rough version of shared memory library in Yahoo Files. I've renamed and reenginered the classes from my STL in shared memory article and boostified almost all parts. The file is in: http://groups.yahoo.com/group/boost/files/boost.shmem.zip Shared memory mutex interface is taken from Boost Threads and scoped locks are needed to lock the mutexes. Since thread scoped locks can't be used from single threaded processes due to header checks, I've cloned them in Boost Shmem, but I would want to reuse thread locks if this library goes on. The offset smart pointer takes much of its interface from boost smart pointers. Boost config is used to build the different dll versions with auto-link and some boost utilities are used (noncopyable for example). This version uses templatized shared memory front-end so that a new shared memory management allocation can be used without changing existing code. In general, I've tried to all that I think it was necessary to adapt to boost, although surely there are many flaws here. The boost shmem jamfile builds for VC-7_1 in WinXP and gcc 3.4.1 in Mandrake 10.1. I've put several test and example files, although my poor jam knowledge makes that tests can only be build for windows using bjam (i don't know how to pass -l link specific options when building in linux to link with librt.a, sorry). Under proj directory there are some VC .NET IDE files to build the library, examples and tests and also a Makefile to build all under linux). I've also documented almost all files using doxygen style and the reference can be build using bjam --v2. If this library goes on, I plan to build a complete boostbook, since I've been reading all the documentation from boost utils. For now, there is a single page help as a tutorial/internal explanation. I would to know if those people that saw the library idea as interesting need any more features for this first version. Francis Andre needs some more sofisticated options and I would like to work with him for a more elaborated version if this library is accepted for boost. Francis suggested compatibility with boost.serialization and I would need some help with that. For this first and surely far from acceptable version I would like receive any comment you like regarding dessign, interface, etc... Help with bjam for building all correctly will be appreciated. I've put some attention in exception handling but surely there will be some problems due to my lack of experience in this area. If anyone wants to colaborate in any aspect of this library, please let me know, I'm totally open for this. Regards, Ion Gaztañaga

"Ion Gaztañaga" wrote:
first rough version of shared memory library http://groups.yahoo.com/group/boost/files/boost.shmem.zip
________________________________________________ 1. The HTML documentation does't allow to change font in browser. ________________________________________________ 2. End of document can be marked so it doesn't look like cut in half during transmission. ________________________________________________ 3. docs 3.2: "Boost Shmem provides some STL containers" probably means "Boost Shmem provides some STL compliant containers" ________________________________________________ 4. Example in 3.2: the "alignement" parameter in segment.create() isn't found in code. ________________________________________________ 5. Example in 4.2: segment.named_new<MyType> ("MyType instance", /*name of the object*/ 10 /*number of elements*/, false /*if previously present, return error*/, 0 /*ctor first argument*/, 0 /*ctor second argument*/); a. What does t mean "return error" here? Exception? b. Wouldn't it be better to split the function into two, one which allow to overwrite, second which doesn't. This would get rid of boolean parameter which is always hard to understand w/o context. c. Could the syntax be like: segment.named_new<MyType>("name")(ctor_param1, ....); so it looks more like operator new? Current signature feels very prone to bugs. In: std::pair< MyType *, std::size_t> res = segment.find_named_new< MyType > ("MyType instance"); Why do I need the "size"? Doesn't a type have always the same size regardless? Isn't sizeof(MyType) == res.second always? ________________________________________________ 6. Could you use namespace shmem_detail or so instead of "detal" to avoid possible clashes? ________________________________________________ 7. exceptions.hpp: a) file name should be shmem_exceptions.hpp or so b) does it make sense to have common base for both exceptions there? ________________________________________________ 8. offset_ptr.hpp: full_offset_ptr class a) using char as de-facto bool class has usually no practical advantage and may be actually slowe. b) The flag could be eliminated completely. If m_offset == 0 it is NULL pointer and no data allocated in shmem will starts on the beginning. This would also eliminate need for min_offset_ptr. c) swap() could be added and other operators. ________________________________________________ 9. Maybe the protection of mutext from shared ptr lib could be worked around so the code duplication would be eliminated. Even if it is dirty it would be worth. ________________________________________________ 10. The example in 4.3 uses very dirty C-like approach with casts. Cannot it be rewritten in C++ way with overloaded new? ________________________________________________ 11. Some source files use Unix line ends, some DOS line ends. Just bit strange. ________________________________________________ 12. The simple algorithm to find fitting memory block may not be adequate for high-performance apps (who are most likely to use shared memory). ________________________________________________ 13. Can be be possible to identify named objects with something else than C string? Say wchar_t* or numeral or other templated type? ________________________________________________ 14. What I would like to see: a) avoiding shmem specific containers/mutexes/etc as much as possible. b) ability to "equalize" shared memory features between operating systems. E.g. Windows deletes shared memory when no one is using it, Unix keeps it until destroyed explicitly. I would like to have both approaches on both platforms (handled possibly by exec()ing helper process. In addition, Shmem should provide platform specific access to its internals, e.g. to Win32 handles to allow security settings. c) support for inheritance in shmem using object factories, e.g. like one in Classloader http://s11n.net/papers/classloading_cpp.html d) support for "transactions": I would like to be able to add/modify/remove objects in shared memory and made all changes visible to other process in one moment. This would save me one more IPC channel to signal readiness of data. If possible the library should provide several levels of isolation. Long lasting transactions could be also considered. /Pavel

Hi I'm also quite interested in this library as I have this enterprise application, which uses XML quite heavily. The XML DOM is as it currently stands is being serialized to a XML String for moving between processes. As Xerces-C and Pathan (XPath) allows one to provide your own memory manager, it will reduce XML DOM generation to happen only once for it's lifetime instead of once in each process it visits. I don't know if this is at all possible, especially considering the problems with the vtables, but hey it will be fun to find out :) An initial comment is that a suggestion to make a shared-memory policy that defined how to create the region. Some of the possibilities could be * Bound/fixed memory address - for OS's that support that - like Windows and Linux * Unbound memory address The fixed memory address would not need offset_ptr and if start memory address is fixed, the vtable problem might actually not be an issue for that creation policy. /Anders Ion Gaztañaga wrote:
Hello to all,
As promised two weeks ago, I have a first rough version of shared memory library in Yahoo Files. I've renamed and reenginered the classes from my STL in shared memory article and boostified almost all parts. The file is in:
http://groups.yahoo.com/group/boost/files/boost.shmem.zip [snip]
participants (3)
-
Anders Hybertz
-
Ion Gaztañaga
-
Pavel Vozenilek