July 2011 - Boost - lists.preview.boost.org

[lockfree] Review
by Grund, Holger 09 Aug '11

09 Aug '11

Hi everyone, I'm sorry for the late review -- things have been busy as usual. I've briefly gone over the code and think Tim's Lockfree library would make an excellent addition to Boost and should be accepted, provided some points are addressed. First and foremost, I don't think the library can be called ready when it doesn't support Visual C++/x86 in a meaningful way -- which I believe it doesn't right now. What is your evaluation of the design? -------------------------------------- The library seems well-designed. As I believe has been discussed in a separate thread, naming is somewhat inconsistent. The failure modes are not clear to me (e.g. does enqueue/push return false or throw an exception when there is no more memory), but overall I think Tim did an excellent job. What is your evaluation of the implementation? ---------------------------------------------- As mentioned, it would appear that bundled atomic implementation has a lot of shortcomings, but one is particularly bad: atomic<tagged_ptr> is not actually lock-free for Visual C++. The DCAS implementation should be straightforward, but seems to be absent . in case you wonder I did an initial implementation of <(cstd)atomic> and added all required intrinsics for a simple implementation to the compiler. These made it all into VC++2010 RTM (IIRC, there was only one 64-bit, most where 8 & 16-bit variants) There are a few points that make me a bit nervous. Specifically, the alignment story is a bit weak. It seems there is only one use of BOOST_LOCKFREE_CACHELINE_ALIGNMENT and that's for fifo<>::node. Older compilers (GCCs and VC at least) don't dynamically align the stack with __declspec(align)/__attribute__((aligned). Other classes do not use alignment at all. It seems somewhat inconsequent to add internal padding to move members to different cachelines, but at the same time not ensuring alignment (which means potential cacheline overlaps at back and front of the object). Also the code seems to assume that compilers will use an empty base class optimization for noncopyable -- I guess, that's fine for most major compilers. Also it is not clear how it is ensured that nodes allocated are actually aligned. Allocators can and usually do ignore alignment requirements greater than intmax_t types. Some may just fail to instantiate. likely/unlikely are often defined as macros as in the Linux kernel. People probably shouldn't do that for C++, but I have seen it quite a bit. It probably doesn't help that many compilers' inlining oracles don't do very well for modern code. The name freelist_t is somewhat confusing for a template parameter and while I'm not sure what harm it could do it should probably follow the general convention of CamelCase. What is your evaluation of the documentation? --------------------------------------------- This is a bit of a weak point right now. There isn't much documentation to begin with. The definition of wait-free and lock-free is a bit weird. Specifically, the term concurrent operations is a bit vague. I think it's easier to grasp if you say something along the lines of at least one thread makes forward progress at any given point in time. With the current definition in the docs (and most others I know of), it may not be immediately clear why a mutex-based implementation wouldn't be lock-free. There's a paragraph: > The synchronization is done completely in user-space with no interaction without any direct interaction with the operating system [1] . > This implies that they are not prone to issues like priority inversion (a low-priority thread needs to wait for a high-priority thread). I think there's some duplication in the first sentence. I'm not sure what lock-free has to do with priority inversion and I would think that priority inversion means priorities are -- well, inverted. I.e. a high-priority thread is not scheduled, because a lower priority thread owns a resource needed by the high-priority thread. This also makes it sound a bit like lock-free is superior to OS mechanisms, but that really depends. A standard lock-free list is not fair for writers, but an implementation making use of OS facilities can be. On the other hand ticket lockets can provide some fairness. fifo<> and ringbuffer::empty says it is not thread-safe. I would assume that means the value returned is potentially stale, but it can't break class invariants, or can it? What is your evaluation of the potential usefulness of the library? ------------------------------------------------------------------- I think the library addresses a very important use-case and could therefore be a very useful addition to Boost. But to reiterate, I would expect Lockfree to actually be lock-free on all mainstream platforms. Did you try to use the library? With what compiler? Did you have any problems? ------------------------------------------------------------------------------ I don't have Boost libs built here, so only tried some simple tests with VC++ 10 SP1. No problems. I'm planning to run some perf tests with GCC 4.1 on AS5 x86_64. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? -------------------------------------------------------------------------------------------- I spent about four hours mostly going over the code and the docs. Are you knowledgeable about the problem domain? ----------------------------------------------- I have done quite a lot of lock-free work over the last the last 8 years. I did an initial implementation of <atomic> for Microsoft and my current job involves a lot of tweaking of very low latency code, where we very much care about upper limits for execution time (but also about throughput). While I'm still constantly surprised by modern CPUs, I do consider myself to be quite knowledgeable about this area. -hg Holger Grund, Vice President Morgan Stanley | MS Strats and Modeling 25 Cabot Square | Canary Wharf | Floor 03 London, E14 4QA Phone: +44 20 7677-4095 Holger.Grund(a)morganstanley.com -------------------------------------------------------------------------- NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing.

5 16

[interprocess::file_mapping] Please add resize method
by Dmytro Ovdiienko 07 Aug '11

07 Aug '11

Hello, Could you please add resize method to the file_mapping class? Also please don't fill file with zeros by default? This functionality increases latency spikes. Make it as quick as possible. Thanks. Best regards. Dima -- Dmytro Ovdiienko e-mail: dmitriy.ovdienko(a)gmail.com skype: dmitriy.ovdienko(a)gmail.com mobile: +38050-1909731

2 5

[type trait extension] I hate volatile...
by Frédéric Bron 05 Aug '11

05 Aug '11

I am still working on the implementation of the traits to detect operators. I have an issue with operators returning a volatile object (being a reference or not). This breaks the code I wrote to detect if the return type is void or not. In particular, the following code compiles with RET=int but not with RET=ret. Can somebody tell me why? It seems that any volatile object (reference or not) cannot be bound to a const reference apart for fundamental types. So is there any useful case to return a volatile object? If not, I can maybe just ignore those cases or it becomes to be a nightmare. By the way, it is 1:30am. I am going to bed! Frédéric struct ret { }; #define RET int RET result; RET r() { return result; } RET const rc() { return result; } RET volatile rv() { return result; } // fails to compile RET const volatile rcv() { return result; } // fails to compile RET & rr() { return result; } RET const & rcr() { return result; } RET volatile & rvr() { return result; } // fails to compile RET const volatile & rcvr() { return result; } // fails to compile int main() { RET x; x=r(); x=rc(); x=rv(); // fails with RET=ret x=rcv(); // fails with RET=ret x=rr(); x=rcr(); x=rvr(); x=rcvr(); return 0; }

4 13

Re: [boost] [rpc] Introducing Boost.Reflect, Boost.CMT, and Boost.RPC
by Brian Wood 05 Aug '11

05 Aug '11

Daniel Larimer: On Jul 30, 2011, at 7:16 PM, Brian Wood wrote: > Daniel Larimer <dlarimer(a)gmail.com> > >> 1) No code generation required (best for C++ to C++, but with a proper > JSON interface can communicate with any language) > > Code generation is being done by something here. In this case it is > probably being done by a C++ compiler that starts from scratch for > every file it has to compile. > > I think there is a big difference between using the pre-processor / templates and > using another program, another language, another dependency, another build step. Yes, there's another dependency and build step, but at least if you're using the approach I advocate you are rewarded with a cleaner build. I've been comparing the differences (between Boost.Serialization and the C++ Middleware Writer) for a number of years now. The most recent comparison using Boost 1.47.0 shows the Boost executables are between 1.7 and 2.8 times larger than the functionally equivalent Ebenezer executables. I haven't compared the build times though so can't say which is faster, but obviously it takes more time to produce a chubby file than a svelte file. > Besides, even if you use a 3rd party tool like google protocol buffers, ICE, etc that does > code generation, the C++ compiler still must start from scratch every time one of the > generated headers is included. Only a fraction of the code is in headers. As much as possible is in source files that don't have to be processed again and again. > Often times you end up deriving from the generated interface and your code breaks if you > change the interface file, etc. Furthermore, now you must maintain the code for the > generator, find bugs, bootstrap by building the generator, etc. I don't think there is any more maintaining the code or finding bugs involved with this approach than with the approach you are describing. The code is just in an executable form as opposed to in a library. With most serialization libraries you bootstrap by building a large library. What I advocate takes 1/10 as long to get bootstrapped as the Boost Serialization library. -- Brian Wood Ebenezer Enterprises http://webEbenezer.net <http://webebenezer.net/>

3 5

numeric_cast
by Brandon Kohn 03 Aug '11

03 Aug '11

Hello, Recently I was creating a new numeric type which I wanted to use along with boost::numeric_cast. I read over the docs and examples and found that it seemed to support (at least partially) the notion of policy driven casting operations. I was unable to figure out how one might use a custom conversion or overflow policy along with the actual boost::numeric_cast<N>(x) function. In the end I created a workaround which seemed to work nicely: template <typename ConversionTraits, typename EnableIf = void> struct numeric_cast_traits { typedef def_overflow_handler overflow_handler; typedef UseInternalRangeChecker range_checker; typedef Trunc < typename ConversionTraits::source_type > rounding_policy; }; template <typename Target, typename Source> inline Target numeric_cast( Source arg ) { typedef boost::numeric::conversion_traits<Target, Source> conv_traits; typedef numeric_cast_traits< conv_traits > cast_traits; typedef boost::numeric::converter < Target , Source , conv_traits , typename cast_traits::overflow_handler , typename cast_traits::rounding_policy , boost::numeric::raw_converter< conv_traits > , typename cast_traits::range_checker > converter; return converter()(arg); } This change allowed me to easily customize the numeric_cast function for individual conversion types. Did I miss something in the official library? If not, would this change be something good to go in? Regards, Brandon Kohn

9 29

Move on Sandia intel
by Ion Gaztañaga 03 Aug '11

03 Aug '11

Hi, A test is failing for Move in Sandia-darwin-intel-11.1 (and 12.0): http://www.boost.org/development/tests/release/developer/output/Sandia-darw… and I don't know why, since Intel 11.0 in windows seems to work fine. Anyone with access to that compiler can help me? Thanks! Ion

4 6

what happened to context mini-review?
by Nathan Ridge 01 Aug '11

01 Aug '11

Hello, Up until a few days ago, the Review Schedule contained an entry for the mini-review of the Context library (which has previously been accepted provisionally) starting from July 28. Now the entry is gone and there is no sign of a mini-review on the list. What is the status of the library? Has the mini-review been postponed? Or will the library be accepted without a mini-review? Thanks, Nate

2 1

Re: [boost] [rpc] Introducing Boost.Reflect, Boost.CMT, and Boost.RPC
by Brian Wood 31 Jul '11

31 Jul '11

Daniel Larimer <dlarimer(a)gmail.com> > 1) No code generation required (best for C++ to C++, but with a proper JSON interface can communicate with any language) Code generation is being done by something here. In this case it is probably being done by a C++ compiler that starts from scratch for every file it has to compile. -- Brian Wood Ebenezer Enterprises http://webEbenezer.net "The heavens declare the glory of G-d; and the firmament shows His handiwork." Psalm 19:1

2 1

[proto] Error with Proto Domains in 1.45
by Fernando Pelliccioni 30 Jul '11

30 Jul '11

Hi Eric, hi all, I'm getting the following errors when I try to compile the example in the atteched file. test_proto_145.cpp: In function ‘int main()’: test_proto_145.cpp:91: error: no match for ‘operator<<’ in ‘emergency << 1’ I am using Boost 1.45 and GCC 4.4.5 This works fine in 1.46.0 and above. Is there a patch ? Thanks and regards. Fernando Pelliccioni.

2 2

[fusion] mpl::clear doesn't work on fusion views
by Eric Niebler 30 Jul '11

30 Jul '11

Today I ran into a limitation in Fusion/MPL interoperability. I couldn't use an mpl algorithm on a Fusion sequence because mpl::clear wasn't defined on it. The sequence was a joint_view. I see that fusion::detail::clear is only defined for Fusion cons, map, set, vector and deque. Can we get it defined for all Fusion sequences? I think for everything else, an empty Fusion vector would do. That is all fusion::clear does, btw. (Aside: it seems strange to me that mpl::clear of a fusion::set gives an empty set, but that fusion::clear of an empty set gives an empty vector. This should probably be made consistent.) -- Eric Niebler BoostPro Computing http://www.boostpro.com

3 4