Hi Boost devs, I need your help to get rid of auto_ptr forever. I want to implement C++17's N4190 "Removing auto_ptr, random_shuffle(), And Old <functional> Stuff", although it's going to take a while. In VC 2015, the removal will be opt-in (requested via a macro; I've already checked this in). My plan is for the removal to be opt-out in the version after that, and unconditional in the version after that. To achieve the opt-in removal, I've already eliminated the STL's internal dependencies on this stuff (nothing derives from unary_function/binary_function anymore, and shuffle() isn't directly implemented with random_shuffle() anymore). To prepare for the future of opt-out removal, I've also been removing usage from the compiler and other stuff. However, Boost is used in our tests in a number of places. What I'd like to ask is for Boost to begin removing its usage of this stuff (unconditionally or for VC 2015+, your pick). According to my understanding, only unary_function/binary_function and auto_ptr are used extensively by Boost, with the other stuff being used rarely or not at all. Thanks, Stephan T. Lavavej Senior Developer - Visual C++ Libraries
On 3/18/2015 5:10 PM, Stephan T. Lavavej wrote:
Hi Boost devs,
I need your help to get rid of auto_ptr forever.
I want to implement C++17's N4190 "Removing auto_ptr, random_shuffle(), And Old <functional> Stuff", although it's going to take a while. In VC 2015, the removal will be opt-in (requested via a macro; I've already checked this in). My plan is for the removal to be opt-out in the version after that, and unconditional in the version after that.
To achieve the opt-in removal, I've already eliminated the STL's internal dependencies on this stuff (nothing derives from unary_function/binary_function anymore, and shuffle() isn't directly implemented with random_shuffle() anymore). To prepare for the future of opt-out removal, I've also been removing usage from the compiler and other stuff. However, Boost is used in our tests in a number of places.
What I'd like to ask is for Boost to begin removing its usage of this stuff (unconditionally or for VC 2015+, your pick). According to my understanding, only unary_function/binary_function and auto_ptr are used extensively by Boost, with the other stuff being used rarely or not at all.
A table of which Boost libraries use auto_ptr, unary_function, and binary_function might help your cause.
Stephan T. Lavavej wrote:
Hi Boost devs,
I need your help to get rid of auto_ptr forever.
I want to implement C++17's N4190 "Removing auto_ptr, random_shuffle(), And Old <functional> Stuff", although it's going to take a while. In VC 2015, the removal will be opt-in (requested via a macro; I've already checked this in). My plan is for the removal to be opt-out in the version after that, and unconditional in the version after that.
To achieve the opt-in removal, I've already eliminated the STL's internal dependencies on this stuff (nothing derives from unary_function/binary_function anymore, and shuffle() isn't directly implemented with random_shuffle() anymore). To prepare for the future of opt-out removal, I've also been removing usage from the compiler and other stuff. However, Boost is used in our tests in a number of places.
What I'd like to ask is for Boost to begin removing its usage of this stuff (unconditionally or for VC 2015+, your pick). According to my understanding, only unary_function/binary_function and auto_ptr are used extensively by Boost, with the other stuff being used rarely or not at all.
shared_ptr has a constructor taking auto_ptr, but it's guarded by a Config macro, BOOST_NO_AUTO_PTR. This was originally for outdated compilers that didn't have auto_ptr yet, and now it's going to be for futuristic compilers that don't have it anymore. Everything old is new again, yet with a twist.
[Edward Diener]
A table of which Boost libraries use auto_ptr, unary_function, and binary_function might help your cause.
Grepping headers and ignoring comments: [unary_function, binary_function] accumulators algorithm bimap function functional gil graph heap icl interprocess iostreams mpi msm multi_index numeric polygon signals2 smart_ptr test typeof utility xpressive [auto_ptr] asio assign date_time detail geometry graph iostreams locale mpi ptr_container python serialization smart_ptr spirit statechart test thread typeof [old <functional> stuff, random_shuffle] accumulators container gil graph lambda msm multi_index phoenix random range spirit test This isn't counting Boost's tests. [Peter Dimov]
shared_ptr has a constructor taking auto_ptr, but it's guarded by a Config macro, BOOST_NO_AUTO_PTR.
Oh, that's good to know. Thanks, STL
________________________________________ From: Boost [boost-bounces@lists.boost.org] on behalf of Stephan T. Lavavej [stl@exchange.microsoft.com] Sent: 18 March 2015 23:23 To: boost@lists.boost.org Subject: Re: [boost] Removing auto_ptr/etc. from Boost [Edward Diener]
A table of which Boost libraries use auto_ptr, unary_function, and binary_function might help your cause.
Grepping headers and ignoring comments: [old <functional> stuff, random_shuffle]
phoenix
This isn't counting Boost's tests.
Oh, that's good to know.
Thanks, STL
I am the current active maintainer of Phoenix and currently working on other deprecated stuff e.g. hash. I have been having a look at C++17's N4190 "Removing auto_ptr, random_shuffle(), And Old <functional> Stuff" and as I read it, the question is not the whole of <functional> but some part of it. What were you testing for which is in Phoenix? I have a file boost/phoenix/config.hpp which adds to boost/config.hpp as needed, so I can configure there for different standards. My aim is to pass all tests, for C++03, C++11 etc. Thanks John
[Fletcher, John P]
as I read it, the question is not the whole of <functional> but some part of it.
Correct - the subset that was deprecated in C++11. Specifically: unary_function/binary_function, ptr_fun(), mem_fun()/mem_fun_ref(), bind1st()/bind2nd(), and their associated classes. Thanks, STL
________________________________________ From: Boost [boost-bounces@lists.boost.org] on behalf of Stephan T. Lavavej [stl@exchange.microsoft.com] Sent: 19 March 2015 08:09 To: boost@lists.boost.org Subject: Re: [boost] Removing auto_ptr/etc. from Boost
[Fletcher, John P]
as I read it, the question is not the whole of <functional> but some part of it.
Correct - the subset that was deprecated in C++11. Specifically: unary_function/binary_function, ptr_fun(), mem_fun()/mem_fun_ref(), bind1st()/bind2nd(), and their associated classes.
Thanks, STL
I have had a look around in the Phoenix code, which I did not write most of. I can find <functional> used in some tests but not in the header code. Do you have any information from your analysis of where there could be problems? Thanks John
[Fletcher, John P]
I have had a look around in the Phoenix code, which I did not write most of. I can find <functional> used in some tests but not in the header code. Do you have any information from your analysis of where there could be problems?
Ah, it was random_shuffle(), sorry for the confusion caused by grouping them together. Here's what I'm seeing: C:\Temp\boost-1.57.0\include\boost>grep -rP "std::(unary_function|binary_function|ptr_fun|mem_fun|mem_fun_ref|bind1st|bind2nd|auto_ptr|random_shuffle)\b" phoenix phoenix/stl/algorithm/transformation.hpp: return std::random_shuffle(detail::begin_(r), detail::end_(r)); phoenix/stl/algorithm/transformation.hpp: return std::random_shuffle(detail::begin_(r), detail::end_(r), g); [STL]
[auto_ptr] geometry
[Adam Wulkiewicz]
And it's gone. Thanks for the heads up.
Yay, thank you! STL
On Wednesday 18 March 2015 23:23:25 Stephan T. Lavavej wrote:
[Edward Diener]
A table of which Boost libraries use auto_ptr, unary_function, and binary_function might help your cause.
Grepping headers and ignoring comments:
[old <functional> stuff, random_shuffle] accumulators container gil graph lambda msm multi_index phoenix random range spirit test
Boost.Log is missing here. I'm worried about random_shuffle. In Boost.Log case I don't care about randomness quality, what's important is performance. I suspect replacing random_shuffle with shuffle+RNG will be inferior, let alone the need to still support random_shuffle for older compilers.
AMDG On 03/19/2015 03:03 AM, Andrey Semashev wrote:
Boost.Log is missing here.
I'm worried about random_shuffle. In Boost.Log case I don't care about randomness quality, what's important is performance. I suspect replacing random_shuffle with shuffle+RNG will be inferior,
I highly doubt it. The algorithm is essentially the same, the only difference is the source of randomness. In fact, if you don't care about the randomness, with shuffle, it's easier to substitute a faster, lower quality PRNG.
let alone the need to still support random_shuffle for older compilers.
In Christ, Steven Watanabe
On Thursday 19 March 2015 09:28:11 Steven Watanabe wrote:
AMDG
On 03/19/2015 03:03 AM, Andrey Semashev wrote:
Boost.Log is missing here.
I'm worried about random_shuffle. In Boost.Log case I don't care about randomness quality, what's important is performance. I suspect replacing random_shuffle with shuffle+RNG will be inferior,
I highly doubt it. The algorithm is essentially the same, the only difference is the source of randomness.
That's the source of my suspicion. Looking at Boost.Random, creating RNG involves seed generation, which is probably one or several syscalls, let alone any possible resource allocations. This has to be done on each call to shuffle, if I understand correctly. Compare that with rand(), which is presumably purely computational and never fails.
In fact, if you don't care about the randomness, with shuffle, it's easier to substitute a faster, lower quality PRNG.
That's what rand() is for, isn't it? But as far as I can see, there is no standard adapter of this function to the RNG interface, so I have to write one. I can do that, no problem, but this is clearly a case for random_shuffle to exist for.
Andrey Semashev wrote:
Looking at Boost.Random, creating RNG involves seed generation, which is probably one or several syscalls, let alone any possible resource allocations.
Creating what type of RNG?
This has to be done on each call to shuffle, if I understand correctly.
Well, it depends, you can keep the URNG around if you like. You don't have to recreate it before each call.
Compare that with rand(), which is presumably purely computational and never fails.
I would expect rand() to take a lock (or use an atomic compare/exchange) under implementations that don't make it thread-specific, so it should be slower than a simple local RNG. Even a thread-local rand() should be slightly slower, in principle.
On Thursday 19 March 2015 18:03:13 Peter Dimov wrote:
Andrey Semashev wrote:
Looking at Boost.Random, creating RNG involves seed generation, which is probably one or several syscalls, let alone any possible resource allocations.
Creating what type of RNG?
I was looking at random_device. Now that I look at other generators, they don't use it, so my worries are probably unfounded.
This has to be done on each call to shuffle, if I understand correctly.
Well, it depends, you can keep the URNG around if you like. You don't have to recreate it before each call.
Again, I was mistaken - seeing rvalue reference in the shuffle interface I assumed it had to be rvalue. I would have to deal with thread safety myself though.
Compare that with rand(), which is presumably purely computational and never fails.
I would expect rand() to take a lock (or use an atomic compare/exchange) under implementations that don't make it thread-specific, so it should be slower than a simple local RNG.
On Linux rand() is not thread safe in the sense that its result is not predictable, which is fine. On Windows rand() uses TLS. Not sure about other systems.
Even a thread-local rand() should be slightly slower, in principle.
Why? On Windows and Linux TLS is implemented as access through fs: qualified pointer, on other systems I would expect something similar or, at the worst, double indirection (TLS key is an index in an array of pointers, for example). It should be faster than creating RNG (even a simpler one than random_device) and comparable to referencing an existing one, given that such a reference can be trivially obtained.
On 19 March 2015 at 13:19, Andrey Semashev
On Linux rand() is not thread safe in the sense that its result is not predictable, which is fine.
This sounds suspicious. Are you sure there are no race conditions inside the implementation of rand()? If there are race conditions, how do you know they are always benign? -- Nevin ":-)" Liber mailto:nevin@eviloverlord.com (847) 691-1404
On Thursday 19 March 2015 13:24:46 Nevin Liber wrote:
On 19 March 2015 at 13:19, Andrey Semashev
wrote:
On Linux rand() is not thread safe in the sense that its result is not predictable, which is fine.
This sounds suspicious.
Are you sure there are no race conditions inside the implementation of rand()? If there are race conditions, how do you know they are always benign?
I was basing that on the man page. Now I have downloaded glibc sources and can see it uses a lock internally. So yet again, I was wrong. Live and learn...
[Andrey Semashev]
Boost.Log is missing here.
I grepped for std::MEOW, avoiding false positives at the cost of false negatives.
I would have to deal with thread safety myself though.
If you don't care about quality, you could use a thread_local linear_congruential_engine, with either a fixed seed or a nondeterministic seed obtained from random_device. (mt19937 is awesome, but at 5 KB I'd be slightly nervous about making it thread_local.) STL
[STL]
(mt19937 is awesome, but at 5 KB I'd be slightly nervous about making it thread_local.)
[Steven Watanabe]
boost::mt19937 is only 2.5 KB.
Thanks - I see where the factor of 2 is coming from, although it'll need a bit more thought to untangle. I've filed a bug in our internal database. STL
AMDG On 03/19/2015 01:33 PM, Stephan T. Lavavej wrote:
[STL]
(mt19937 is awesome, but at 5 KB I'd be slightly nervous about making it thread_local.)
[Steven Watanabe]
boost::mt19937 is only 2.5 KB.
Thanks - I see where the factor of 2 is coming from, although it'll need a bit more thought to untangle. I've filed a bug in our internal database.
It's quite painful. If you update the entire buffer in a single pass, you need to invert the state transition function to implement streaming correctly. libc++ only updates a single element at a time, which is noticeably slower, and last time I checked libstdc++, its output format was wrong. In Christ, Steven Watanabe
On 19 Mar 2015 at 19:33, Stephan T. Lavavej wrote:
[STL]
(mt19937 is awesome, but at 5 KB I'd be slightly nervous about making it thread_local.)
[Steven Watanabe]
boost::mt19937 is only 2.5 KB.
Thanks - I see where the factor of 2 is coming from, although it'll need a bit more thought to untangle. I've filed a bug in our internal database.
Anyone considering using mt19937 should be aware it was painfully slow on ARM where it used the generic C implementation. I use the past tense because I don't know the current state of things having excised all uses of Mersenne from my code as a result - nevertheless, Mersenne is only awesome on Intel, and a similar dog slow speed to a crypto random source on some other CPUs. Personally speaking if I need portable, fast, not awful randomness, I like http://burtleburtle.net/bob/rand/smallprng.html. It's much faster to initialise than mersenne too, runs very well on all CPUs without special opcodes, but the output isn't quite as random. As a rand() replacement it ought to be plenty fine, a 2^126 average period is plenty. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
Andrey Semashev wrote:
I was looking at random_device. Now that I look at other generators, they don't use it, so my worries are probably unfounded.
random_device is overkill here. You can get a reasonably random seed by, f.ex. rand() + ( rand() << 15 ), and then create a local RNG from it. The Boost.Random page listing the generators says that taus88 is 6 times faster than minstd_rand, which is the usual implementation of rand(). And it has only three words of state, so constructing it can't be expensive.
Stephan T. Lavavej wrote:
[Edward Diener]
A table of which Boost libraries use auto_ptr, unary_function, and binary_function might help your cause. Grepping headers and ignoring comments: <snip> [auto_ptr] geometry
And it's gone. Thanks for the heads up.
[Peter Dimov]
shared_ptr has a constructor taking auto_ptr, but it's guarded by a Config macro, BOOST_NO_AUTO_PTR. Oh, that's good to know.
The same is true for boost::scoped_ptr. Regards, Adam
Stephan T. Lavavej
Hi Boost devs,
I need your help to get rid of auto_ptr forever.
I want to implement C++17's N4190 [...]
What I'd like to ask is for Boost to begin removing its usage of this stuff [...]
Done for multi_index: https://github.com/boostorg/multi_index/ commit/7683cec9199073ca7f9b6efb7bb7f74a2cc8e01d Joaquín M López Muñoz Telefónica
[Joaquin M Lopez Munoz]
Done for multi_index: https://github.com/boostorg/multi_index/ commit/7683cec9199073ca7f9b6efb7bb7f74a2cc8e01d
Wonderful, thank you! One step closer to a more modern world. STL
participants (10)
-
Adam Wulkiewicz -
Andrey Semashev -
Edward Diener -
Fletcher, John P -
Joaquin M Lopez Munoz -
Nevin Liber -
Niall Douglas -
Peter Dimov -
Stephan T. Lavavej -
Steven Watanabe