- Boost - lists.preview.boost.org

[Sort] Parallel Sorting Mini review ends November 20th (Need reviewers)
by Francisco José Tapia 16 Nov '16

16 Nov '16

Dear Boost community, The mini-review of Francisco Tapia's Parallel sorting library began in November 11, and ends November 20th. The purpose of this review is to assess whether the sub-library is useful and up to Boost software standards. If the Boost community agrees on both, Francisco and I will integrate it into the existing Boost.sort library as a separate sub-library from Spreadsort. In doing so we'll integrate the documentation with the existing documentation for Boost.Sort, making it more consistent. If you review Francisco's library, please make sure you're using a compiler that supports C++11, and answer each of these questions: 1. Were you able to run it and see accurate and fast results? Please specify your compiler, OS, and processor type, and any problems (or slowdowns) encountered. 2. Would you use this library if accepted? Why or why not? 3. What is your evaluation of the design? 4. What is your evaluation of the implementation? 5. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? The library is here: https://github.com/fjtapia/sort_parallel This library provides both stable and unstable sorting algorithms, in single threaded and parallel versions. - All the algorithms have average and worst case runtime of* O(NLogN).* - These algorithms *do not use any other library or utility*. To compile and run these you will need a *C++11 compliant compiler.* - This library is *include only*, only requiring the files in the boost/sort/parallel folder, with no external dependencies. - The algorithms *use a comparison object*, in the same fashion as std::sort, defaulting to the std::less object, which uses the < operator internally for comparisons. - The algorithms are *exception safe*, meaning that the exceptions generated by the algorithms guarantee the integrity of the objects to sort, but not their relative order. If the exception is generated inside the objects (in the move or in the copy constructor) the results can be unpredictable. The library has a a* new parallel_sort algorithm ( internally named Block Indirect)*, for processors connected with shared memory, *invented for the library*. This new algorithm combines the speed of algorithms like the GCC parallel sort (based on OpenMP), or the Microsoft parallel buffered sort, with the small memory consumption of algorithms like the TBB sort and the Microsoft parallel Sort. Results from running on an I7 5820 with 12 threads: - GCC parallel_sort 1,25 secs 1560 MB - TBB parallel_sort 1,64 secs 783 MB - Boost parallel_sort 1,08 secs 786 MB In a *separate repository* (because it uses licence-restricted software like TBB and Microsoft parallel sort, and can’t be included in the Boost library), you can find *all the code needed to run the benchmarks*, the library code, the benchmark programs, and shell scripts to compile and run it on your machine in Linux and Windows. For the Linux benchmarks you will need to install TBB. (https://github.com/fjtapia/sort_parallel_benchmark) We look forward to your feedback!

1 0

[MPL] A Proposal
by Bruno Dutra 16 Nov '16

16 Nov '16

Dear Community, About a year ago today, there was a proliferation of threads [1][2][3] in this mailing list concerning the future of Boost.MPL, or even whether it had a future at all, and opinions diverged widely on the matter. Many have expressed concerns about the fact Boost.MPL was long outdated and proposed major changes to open path for modernization, while some others argued against it, fearing unexpected and disastrous side-effects could propagate throughout the rest of Boost. A few questioned if it was worth it at all to spend any further resources on Boost.MPL given the fact new alternatives were coming up, such as Boost.Hana, which departed entirely from template metaprogramming, yet some argued TMP remained a competitive alternative for type computations in C++14 as it has been demonstrated by Peter Dimov in his impressive articles [4] on modern TMP. Amidst all this, a proposal by Robert Ramey [5] caught my attention and inspired me to develop what I call Metal [6], a template metaprogramming library designed on the same core principles as Boost.MPL, but written from the very beginning to take full advantage of C++11. Metal was not designed to be a strict drop in replacement for Boost.MPL though, it was rather meant to be regarded as a modern version of Boost.MPL's original API, diverging only where it could really benefit from modern C++ and that essentially means gains of several orders of magnitude in performance** and lower memory consumption. At this point Metal is very close to where I originally intended to bring it and I think it's time I gauge interest from the community on a modest proposal I've been maturing along the past year. As a modern approach to the same task that Boost.MPL was meant to address, I don't believe Boost would benefit at all from yet another TMP library that would essentially compete with Boost.MPL and even Boost.Hana in some cases, specially as maintainers of older code that depends on Boost.MPL would have to go through major refactoring efforts in order to benefit from such an addition to Boost. I would like to propose something different instead. My proposal is to make Metal officially into a new revision of Boost.MPL's API, essentially MPL2 as the original proposal by Robert Ramey put it, merging both into one single TMP library. The idea would be to provide a thin proxy for the current Boost.MPL API which would have two backends configurable by preprocessor switches: the original implementation and a another one based on Metal. This way older code using Boost.MPL could see immediate gains on performance by selecting the newer backend on supported compilers, while at the same time legacy code maintained on older compilers could still rely on Boost.MPL's impressive ability to run virtually anywhere. On the other hand, newer projects interested on pure type computations would have the newer API entirely available to them. This proposal also overcomes the problem of dealing with the maintainability of Boost.MPL, since it provides a way for smooth transition from legacy hacks required by ancient compilers into a cleaner and easier to maintain C++11 codebase, besides avoiding any breaking changes within existing code in the process. Furthermore, should this proposal make it into Boost, I would be naturally willing to take responsibility for the maintenance of Boost.MPL, which AFAIK has been maintained by the CMT for quite some time now, often with limited resources. Comparing Metal and Boost.MPL there are a few noteworthy differences. Metafunction Classes have proven themselves entirely unnecessary and have thus been removed altogether in favour of pure Lambdas (in the Boost.MPL sense). Moreover, iterators make little sense in a functional world, so these too have been removed despite the fact they may easily be emulated in the new design. Moreover Metal develops the concept of optionals, that is, its metafunctions only have a nested ::type defined when they may be successfully computed, what may be safely tested using metal::is_just and may also be used to SFINAE out of miscomputations. Finally I'd like to point out that Metal was designed to be portable across as many compilers as possible and three of the most popular ones are continuously tested using Travis and Appveyor: Clang 3.5+, GCC 4.8+ and MSVC 14 (2015). I hope this is a reasonable proposal and I look forward into your opinions about it. ** Even though I haven't published any benchmarks yet, an example follows just to back my claims regarding performance (just tested on my personal laptop running linux): //Generate a list containing 10 thousand instances of std::integral_constant, from 0 to 9999 using l = metal::enumerate_t<metal::integer<10000>>; // GCC 5.3 ~0.4 s // Clang 3.7 ~0.5 s //Transform a list containing 5 thousand numbers by squaring each element using l1 = metal::enumerate_t<metal::number<long long, 5000>>; using l2 = metal::transform_t<metal::pow<metal::_1, metal::integer<2>>, l1>; // GCC 5.3 ~5.6 s // Clang 3.7 ~3.0 s [1]: http://thread.gmane.org/gmane.comp.lib.boost.devel/257680 [2]: http://thread.gmane.org/gmane.comp.lib.boost.devel/257983 [3]: http://thread.gmane.org/gmane.comp.lib.boost.devel/258843 [4]: http://pdimov.com/cpp2/simple_cxx11_metaprogramming.html [5]: http://rrsd.com/blincubator.com/bi_suggestion/mpl-lite-or-mpl2/ [6]: http://brunocodutra.github.io/metal/ Regards, Bruno Dutra

12 48

Re: [boost] [MPL] A Proposal
by Bruno Dutra 16 Nov '16

16 Nov '16

On Nov 15, 2016 23:57, "Peter Dimov" <lists(a)pdimov.com> wrote: > > Bruno Dutra wrote: > >> Meanwhile, following Rene's suggestion to move away from Travis' container based infrastructure, I was also able to run tests on Clang 3.9 and even on what seems to be nightly snapshots of Clang 4.0: >> >> https://github.com/brunocodutra/metal/blob/develop/.travis.yml. > > > I don't care that much for 3.9 and 4.0, but the -stdlib=libc++ option is intriguing. Does it require sudo? I just checked and libc++-dev is whitelisted on Precise images, so it shouldn't require sudo. That said, the version of libc++ available on such an old version of Ubuntu might not fit your needs (it didn't mine) so you might need to upgrade to Trusty, in which case I believe it requires sudo, because it doesn't seem to be whitelisted yet.

1 0

[fiber] destruction order fiasco
by Tatsuyuki Ishi 15 Nov '16

15 Nov '16

The fiber still makes use of the scheduling algorithm after main context exits, which is a problem for my case that puts a stack-allocated shared data to the algorithm. Some points: https://github.com/boostorg/fiber/blob/master/src/context.cpp#L52 I doubt if the schwarz counter is needed; isn't it only one? https://github.com/boostorg/fiber/blob/master/src/scheduler.cpp#L137 I don't think there's a need to wake it up; if it's sleeping, the thread is blocked and it never enters here. This is the main problem: use-after-free. Anyway, what I think is ideal is that just making the algorithm a reference and assign it from the main context. An example of this approach is Qt's Q*Application.

1 0

Re: [boost] [Boost-users] [process] Formal review results
by Bjorn Reese 13 Nov '16

13 Nov '16

On 11/09/2016 07:45 PM, Antony Polukhin wrote: > The Boost.Process library is accepted. Unconditionally? Please notice that some of the votes were conditional, and if those conditions are not met, they should be counted as no votes. > The most controversial part of the library is it's interface. Many > comments were addressing: > * passing arguments > * starting process right in `system(...)` function The comments were about starting the process in the child constructor.

4 3

Boost.Config for clang on Linux with libc++
by Belcourt, Kenneth 13 Nov '16

13 Nov '16

Hi, Attached are user.hpp and the console output from running configure with clang-3.9. Note that I tried to configure using libc++ but the configure script immediately failed, so these are the results for clang-3.9 with the default stdlib (libstdc++). Can someone please take a look and see what adjustments might need to be made to Boost.Config to support this configuration? Once it works for the default stdlib, I’d like to rerun it to use libc++. Thanks, Noel Belcourt

3 4

[process] Formal Review
by Nat Goodspeed 13 Nov '16

13 Nov '16

I would like to surface a number of issues, large and small. You are clearly proud to support a number of different syntactic ways to express the same semantic operation. To me this is a minor negative. While in theory it sounds nice to be able to write Process consumer code any way I want, in a production environment I spend more time reading and maintaining other people's code than I do writing brand-new code. Since each person contributing code to our (large) code base will select his/her own preferred Process style, I will be repeatedly confused as I encounter each new usage. I admire the support for synchronous pipe I/O, while remaining skeptical of its practical utility. Synchronous I/O with a child process presents many legitimate opportunities for deadlock: traps for the unwary. I would be content with a combination of: * async I/O on pipes (yes, using Asio) * system() for truly simple calls * something analogous to Python's subprocess.Popen.communicate(): pass an optional string for the child's stdin, retrieve a string (or, say, a stringstream) for each of its stdout and stderr. The example under I/O pipes the stdout from 'nm' to the stdin of 'c++filt'. But the example code seems completely unaware that c++filt could be delayed for any of a number of reasons. It assumes that as soon as nm terminates, c++filt must have produced all the output it ever will. I worry about the Process implementation being confused about such issues. I'm dubious about the native_environment / environment dichotomy. As others have questioned, why isn't 'environment' a typedef for a map<string, string> (or unordered_map)? I understand that you desire to avoid copying the native process environment into such a map until necessary, but to me that suggests something like an environment_view (analogous to string_view) that can perform read-only operations on either back-end implementation. Operations involving splitting and joining on ':' or ';' should be defined purely in terms of strings and ranges of strings. They should not be conflated with environment-map support. The documentation so consistently uses literal ';' as the PATH separator that I worry the code won't correctly process standard Posix PATH strings. At this moment in history, an example showing integration of Process with Boost.Fiber seems as important as examples involving coroutines. Why is the native_handle_t typedef in the boost::this_process namespace? While in full context it makes sense to speak of "assigning" an individual process to a process group, the method name assign() has conventional connotations. Use add() instead. There's a Note that says: "If a default-constructed group is used before being used in a process launch, the behaviour is undefined." I assume you mean "destroyed before being used," but this is a concern. If a program has already instantiated a process group, but for any reason decides not to (or fails to) launch any more child processes, does that put the entire parent process at risk? What remedial action can it take? Move the group object to the heap somewhere and let it leak? Spawn a bogus child process for the purpose of defusing the ticking process group instance? If you're going to reify process group at all, you should wrap more logic around it to give it well-defined cross-platform behavior. And it should definitely be legal to create and destroy one without associating any child process with it. Given support elsewhere for splitting/joining PATH strings, the string_type path parameter to search_path() feels oddly low-level. Maybe accept a range of strings? > At a minimum, kindly state: > - Whether you believe the library should be accepted into Boost > * Conditions for acceptance YES, IF the present Boost.Process preserves the ability of its predecessor to extend it without modifying it. I'm sorry, thorough reading of the documentation plus some browsing through the code leaves me doubtful. Examples of such extensions: * With Boost.Process 0.5 I can use Boost.TypeErasure to pass an "initializer" which is a runtime collection of other initializers. Such a feature in Process 0.6 would support people's requests to be able to assemble an arbitrary configuration with conditional elements and use it repeatedly. * I can create an initializer (actually one for each of Posix and Windows, presenting the same API, so portable code can use either transparently) to implicitly forcibly terminate the child process being launched when the parent process eventually terminates in whatever way. Please understand that I am not asking for the above features to be absorbed into the Process library: I am asking for a Process library extensible enough that I can implement such things with consumer code, without having to modify the library. Perhaps Process 0.6 already is! It's just hard for me tell. Another feature should, in my opinion, be incorporated into the library: * On Windows, if you desire to pass any file handles at all to the new child process, it is completely whimsical what *other* open file handles you may inadvertently pass -- unless you play games with STARTUPINFOEXA and PROC_THREAD_ATTRIBUTE_LIST to enumerate exactly the set of handles you intend to pass. If the library doesn't natively support that, I *must* be able to pass a custom initializer with deep access to the relevant control blocks to set it up. This is a showstopper, as in "you can't remove that file because, unbeknownst to you, some other process has it open." > - Your knowledge of the problem domain I have hand-written process-management code in C++ several times before -- each with multiple implementations to support multiple platforms. I have tested the previous candidate Boost.Process 0.5 with a number of programs exercising a number of features. > You are strongly encouraged to also provide additional information: > - What is your evaluation of the library's: > * Design Design notes are at the top of this mail. > * Implementation I have only glanced over parts of the implementation. It seems somewhat more obscure than the corresponding pieces of Process 0.5, which is why I couldn't quickly satisfy myself as to the library's extensibility. > * Documentation Others have noted that the documentation is very sketchy in places. I too wish for more explanation. Much of the time I spent on this review was reading through the documentation. Apologies if I have misunderstood the library's capabilities. > * Usefulness I have felt for years now that it is essential to get a child-process management library into Boost. It grieves me to have to write platform-specific API calls into different applications. > - Did you attempt to use the library? I did not, sorry, ran out of time -- as you can infer from this review arriving at the end of the final day! > - How much effort did you put into your evaluation of the review? Most of my time was spent reading the documentation. I looked through a couple of the implementation files for further information.

4 8

Visual Studio 2015 Update 3 has removed std::unary_function and std::binary_function
by Marshall Clow 12 Nov '16

12 Nov '16

... if you build with /std:c++latest. See https://svn.boost.org/trac/boost/ticket/12586 for an example. Other vendors will be removing these types in C++17 mode in the future (or at least requiring a -D RETAIN_REMOVED_STUFF=1 or something like that) A quick grep for these terms found 153 instances of "std::unary_function" and 118 of "std::binary_function" across several libraries, including accumulators, algorithm, config, container, function, gil, graph, icl, mpi, msm, polygon, ptr_container, serialization, smart_ptr, tr1, unordered, utility - and probably others. I think that we ought to have a boost-wide solution for this (i.e, everyone should solve this the same way). I can think of three possibilities: * Deny, deny, deny: Boost doesn't work with C++17. Not my recommendation. * Since std::unary|binary_function is an empty struct with a set of two (or three) typedefs, just define the typedefs everywhere. Note that the typedefs (result_type, first_argument_type, second_argument_type) were deemed to not be useful in C++17 due to the existence of advanced type traits, etc. * Define a boost:unary_function and boost::binary_function struct that mimics the one that was in namespace std, and change over to use that. * Something else. Comments? -- Marshall

15 74

Adding any_dynamic_cast?
by Peter Dimov 12 Nov '16

12 Nov '16

I stumbled across https://github.com/bytemaster/Boost.DynamicAny which implements, in a fully backward compatible manner, dynamic cast functionality for boost::any. That is, it can give you X* from an 'any' holding a value of type Y if Y derives from X. It achieves this by simply changing template<typename ValueType> class holder : public placeholder { ValueType held; to template<typename ValueType> class holder : public placeholder, public ValueType which then enables dynamic_cast from holder<>* to OtherValueType*. This is pretty cool, and I don't remember seeing this idea anywhere else. Perhaps we should adopt it? One might even argue that this is how any_cast should always have worked, but I suppose we can't change it now for compatibility reasons. Although it's an interesting question whether C++17 'any' shouldn't be enhanced in this manner.

2 2

Request for a new submodule, tools/depinst
by Peter Dimov 12 Nov '16

12 Nov '16

I'd like to request the creation of a new submodule, tools/depinst, that will contain a Python script depinst.py which would, given a module in libs/, install the dependencies needed for running that module's tests. Currently, the .travis.yml files I use enumerate the dependencies manually, which is a bit tiresome: https://github.com/boostorg/function/blob/develop/.travis.yml - git submodule init libs/algorithm - git submodule init libs/align ... - git submodule init libs/type_traits - git submodule init libs/typeof - git submodule init libs/utility - git submodule init libs/winapi - git submodule init tools/build - git submodule update --depth 1 This will be replaced by just - git submodule update --init --depth 1 tools/depinst - git submodule update --init --depth 1 tools/build - python tools/depinst/depinst.py function

5 19