October 2015 - Boost - lists.preview.boost.org

Re: [boost] [c++std-parallel-2020] Re: We need a coherent higher level parallelization story for C++ (was [thread] Is there a non-blocking future-destructor?)
by Peter Dimov 14 Oct '15

14 Oct '15

Lawrence Crowl wrote: > First, std::async and std::future have no flaws for what they were > designed to do. They were designed to provide relatively easy parallelism > for small core systems under the fork-join model. I thought that it was well established by now that std::async isn't very useful for fork-join (if at all.) First, it's required to execute each (asynchronous) task as-if in a new thread with respect to thread locals, which in practice means "in a new thread". Second, it has to allocate the future shared state. These two non-flaws kill its performance for fork-join parallelism. The first one is not related to its use of std::future, but the second one is. Given an interface of the form X x; async_result<R> r; std::async( r, [&]{ return x.f(); } ); with async_result being non-copyable and non-movable and ~async_result blocking on the async task, the task can store its result directly into r, without needing to allocate.

1 0

Re: [boost] [c++std-parallel-2020] Re: We need a coherent higher level parallelization story for C++ (was [thread] Is there a non-blocking future-destructor?)
by Peter Dimov 14 Oct '15

14 Oct '15

Lawrence Crowl wrote: > Third, extending the use of std::future beyond its design goals caused > problems. You would expect this of any system used beyond its goals. In > particular, using std::futures for anything other than std::async was a > step too far. This is absolutely backwards. Future was always intended to be used as a generic future value. Hence the name, and the existence of 'promise'. Extending it beyond these design goals in order to imbue it with a lifetime management function caused the problems.

1 0

Test Runner Crashing
by Tom Kent 12 Oct '15

12 Oct '15

Over the weekend I've seen my windows test runner completely crashed (and other OS applications crashed as well), it is preventing the runner from getting through a complete run. I don't really have any idea where it is happening, but I have seen some errors in wserialization throwing up application crashed window...could this be related? Here's the info from the window: Problem signature: Problem Event Name: APPCRASH Application Name: test_non_intrusive_text_warchive.exe Application Version: 0.0.0.0 Application Timestamp: 561b13c0 Fault Module Name: MSVCP120D.dll Fault Module Version: 12.0.21005.1 Fault Module Timestamp: 524f8411 Exception Code: c0000005 Exception Offset: 000000000007c170 OS Version: 6.3.9600.2.0.0.272.7 Locale ID: 1033 Additional Information 1: 4de6 Additional Information 2: 4de60485dd55f55c5bf1ba0ca18762f0 Additional Information 3: 57a7 Additional Information 4: 57a7ea5113dd7a4cdeb1d88e51b65649 Read our privacy statement online: http://go.microsoft.com/fwlink/?linkid=280262 If the online privacy statement is not available, please read our privacy statement offline: C:\Windows\system32\en-US\erofflps.txt Here's the Visual Studio call stack: msvcp120d.dll!00007fff81a4c170() [Frames below may be incorrect and/or missing, no symbols loaded for msvcp120d.dll] msvcp120d.dll!00007fff819d88d2() msvcp120d.dll!00007fff819d9918() msvcp120d.dll!00007fff819e1511() > test_non_intrusive_text_warchive.exe!std::basic_filebuf<wchar_t,std::char_traits<wchar_t> >::_Endwrite() Line 643 + 0xcf bytes C++ test_non_intrusive_text_warchive.exe!std::basic_filebuf<wchar_t,std::char_traits<wchar_t> >::close() Line 341 + 0xa bytes C++ test_non_intrusive_text_warchive.exe!std::basic_filebuf<wchar_t,std::char_traits<wchar_t> >::~basic_filebuf<wchar_t,std::char_traits<wchar_t> >() Line 156 C++ test_non_intrusive_text_warchive.exe!std::basic_ofstream<wchar_t,std::char_traits<wchar_t> >::~basic_ofstream<wchar_t,std::char_traits<wchar_t> >() Line 1061 + 0x13 bytes C++ test_non_intrusive_text_warchive.exe!std::basic_ofstream<wchar_t,std::char_traits<wchar_t> >::`vbase destructor'() + 0x1c bytes C++ test_non_intrusive_text_warchive.exe!save(const char * testfile) Line 143 + 0x1b bytes C++ test_non_intrusive_text_warchive.exe!test_main(int __formal, int __formal) Line 169 C++ test_non_intrusive_text_warchive.exe!main(int argc, char * * argv) Line 203 + 0xe bytes C++ test_non_intrusive_text_warchive.exe!__tmainCRTStartup() Line 626 + 0x19 bytes C test_non_intrusive_text_warchive.exe!mainCRTStartup() Line 466 C kernel32.dll!00007fff8f7013d2() ntdll.dll!00007fff91fc5454() Any ideas? Could it be possible that too many application crashes could be taking out the test runner (and other OS applications) completely? Thanks, Tom

1 0

[Hana] Integration with the regression Matrix
by Louis Dionne 12 Oct '15

12 Oct '15

Hi, With the goal of shipping Hana with the next release of Boost, I am currently trying to satisfy the requirement that Hana's testing results show up in the Boost regression matrix. One way to achieve this would be to have Travis CI upload the results on each push to master. To do so, I would need to somehow create an XML file compatible with what the regression matrix expects. That brings up a few questions: (1) Is there a way to upload the result of CMake tests to the Boost regression matrix? I suspect something like that might have already been done as part of the (unfortunately defunct) CMake effort? (2) If not, would it be difficult to implement? Any input on how to best integrate Hana's testing results with the rest of Boost's regression testing will be greatly appreciated. Also, before someone suggests using Boost.Build: I have been looking at the possibility of maintaining a separate Jamfile. However, the need to maintain two separate build systems is annoying, so I'd like to avoid it if possible. I also think that Travis-based testing could provide a better coverage for Hana than the voluntary testers given the need for cutting edge compilers, but I could be wrong. Regards, Louis -- View this message in context: http://boost.2283326.n4.nabble.com/Hana-Integration-with-the-regression-Mat… Sent from the Boost - Dev mailing list archive at Nabble.com.

7 9

[thread] Is there a non-blocking future-destructor?
by Mikael Olenfalk 11 Oct '15

11 Oct '15

Hi, I have stumbled on a minor inconvenience when it comes to using boost::future. Below is a small example which first wraps an asio io_service as an Executor (see even further below) and then uses boost::async() to launch a simple task on the executor and tries to queue up a continuation using future::then(Ex, F): 1. the destructor of async_result does not block which is nice 2. the destructor of continuation_result does block which is a bit of an annoyance as this future I do not care about at all Is there a way to make the destructor not block? /M int main(int argc, char* argv[]) { boost::asio::io_service io_service; asio_executor executor{ io_service }; auto async_result = boost::async(executor, []() { return 42; }); // hangs: // async_result.then(executor, &g); // does not hang: auto continuation_result = async_result.then(executor, [](boost::future<int> r) { cout << r.get() << endl; }); cout << "start io_service" << endl; io_service.run(); cout << "stop io_service" << endl; return 0; } class asio_executor : public boost::executors::executor { public: asio_executor(boost::asio::io_service &ios) : _ios(ios) {} void close() override { _ios.stop(); } bool closed() override { return _ios.stopped(); } void submit(work&& w) override { // for some reason _ios.post(w); causes endless // recursion and eventually a crash _ios.post(std::move(w)); } bool try_executing_one() override { return _ios.run_one() > 0; } private: boost::asio::io_service &_ios; };

2 2

Boost.Fiber mini-review report
by Nat Goodspeed 10 Oct '15

10 Oct '15

The mini-review of Boost.Fiber by Oliver Kowalke ran from Friday September 4th through Sunday September 13th. There was lively discussion, but only two actual reviews: Agustín K-ballo Bergé: not yet (list of issues below) Niall Douglas: yes, plus suggestions (below)[0] In addition I received two "review withheld" notes: Vicente J. Botet Escriba wanted me to address the previous review point by point. That seems like a reasonable request, and I apologize, but I too have time pressures. Giovanni Piero Deretta made suggestions (synopsized below) with the statement that he will make a recommendation when he sees the library author's progress along those lines. While in theory these results might seem like a hung jury, the very paucity of formal reviews supports a cautious approach. The verdict is: not yet, with specific points as noted below. No reviewer explicitly designated specific points as being essential. (Niall Douglas explicitly designated his suggestions as post-adoption enhancements.) I have made an attempt to rank the recommendations received in descending order of importance to the community. My criteria are: Robustness: does lack of this request pose significant risk to a consuming program? API stability: could this request be considered an implementation detail, or would the change break early adopters? Demand: how many people requested the same thing? If you feel that I have misrepresented the importance of one of your points (or omitted something entirely), please respond. Ultimately, of course, the importance of any given point to the community at large will be reflected in the next mini-review. ----------------------------------------------------------------- Requests: Cross-thread fiber migration: Evidently the community strongly desires the ability to transfer a suspended fiber from one thread to another.[1] Of course there must be controls. Migration might be explicit; but if implicit, there must be a way to designate which fibers may and may not be migrated. An RAII class that temporarily disables migration would also be desirable -- for instance, to guarantee TLS consistency for some specific period. Cross-thread migration support implies a requirement to document the thread safety requirements of the customization API. Standard required timing specifications: meet (30.2.4 Timing specifications [thread.req.timing]). Adapt one or more existing test suites for thread facilities to validate the correctness of the Fiber library's synchronization constructs: e.g. from libc++, libstdc++, Boost.Thread, HPX. std::chrono support: ensure that wait_until() can accept any supported time_point specialization. (The library author asserts that this has already been fixed on the 'develop' branch.) C++11 compatibility: Agustín K-ballo Bergé volunteered to help with this point post-adoption. Paul Fultz offered fit::capture to emulate C++14 move capture. Document that this_fiber::yield() (et al.) are valid from main(), or in general from a thread function, as well as from explicitly-launched fibers. Make shared_state lock requirements explicit: for example, make mark_ready_and_notify_() accept a std::unique_lock<>& (or &&) to force its caller to provide one. Avoid holding mutex during condition_variable::notify_*() calls. (The library author asserts that this has already been fixed on the 'develop' branch.) packaged_task must not decay-copy its task arguments. Align fibers::async() signature, result description and invoke_helper implementation with std::async(). Avoid returning std::move(...). Wait list implementation: try an intrusive linked list of fiber_context objects. Avoid sleeping thread when no fibers are ready -- block on external event. Better handling of waiting tasks (decentralized wait queues?). Optimization to resume awakened fiber on the thread that awakens it. Template redundancy: eliminate duplication between specializations for T, T& and void. The future, shared_future, shared_state, promise and task_object templates were specifically cited. Leverage predicate-based condition::wait() overload internally. Nested schedulers: make schedulers be schedulable entities themselves. Improve robustness of `promise' handling in Asynchronous Callbacks documentation. Avoid overconfident remarks about it. Fix Note overstating the requirements on condition_variable::wait(). The discussion also surfaced the desirability of Grand Unified Future Theory, but it was generally agreed that this is out of scope. Put differently, it would be unreasonable to put the onus on Fiber to solve that very broad cross-library problem. Many thanks to all participants, and thanks especially to Oliver Kowalke for all his work on this library! Nat Goodspeed Boost.Fiber Review Manager ________________________________ [0] Niall Douglas snuck his review recommendation into the following mail message: http://lists.boost.org/Archives/boost/2015/09/225228.php (second to final paragraph). His suggestions were in private mail. [1] Giovanni Deretta suggests that it might be possible to allocate -- without actually running -- an actual kernel thread per fiber, simply to give each fiber its own TLS area.

2 1

Boost Library Testing - a modest proposal - was boost.test regression or behavior change (was Re: Boost.lockfree)
by Robert Ramey 10 Oct '15

10 Oct '15

I believe this whole thread started from the changes in Boost.Test such that it can no longer support testing of C++03 compatible libraries. This is totally unrelated to the testing of Boost libraries. Here is what I would like to see: a) local testing by library developers. Of course library developers need this in order to develop and maintain libraries. Currently we have this and has worked quite well for many years. Making Boost.Test require C++11+ throws a monkey wrench into things for the libraries which use it. But that's only temporary. Libraries whose developers feel they need to maintain compatibility with C++98 can move to lightweight test with relatively little effort. Developers who are concerned that the develop branch is a "soup" can easily isolate themselves from this by testing against the master branch of all the other libraries. The Boost modularization system with git has made this very simple and practicle (thank you Beman!). So - not a problem. b) Testing on other platforms. We have a system which has worked pretty well for many years. Still it has some features that I'm not crazy about. i) it doesn't scale well - as boost gets bigger the testing load gets bigger. ii) it tests the develop branch of each library against the develop branch of all the other libraries - hence we have a testing "soup" where a test might show failure but this failure might not be related to the library under test but some other library. It diminishes the utility of the test results in tracking down problems. iii) it relies on volunteer testers to select compilers/platforms to test under. So it's not exhaustive and the selection might not reflect that which people are actually using. I would like to see us encourage our users to test the libaries that they use. This system would work in the following way. a) A user downloads/builds boost. b) he decides he's going to use library X, and Y c) he runs a tool which tells him which libraries he has to test. This would be the result of a dependency analysis. We have tools which do similar dependency analysis but they would have to be slightly enhanced to distinguish between testing, deployment, etc. I don't think this would be a huge undertaking given the work that has already been done. d) he runs the local testing setup on those libraries and their dependents. e) he uploads the test results to a dashboard similar if not identical to the current one. f) we would discourage uses from just using the boost libraries without runnig they're own tests. We would do this by exhortation and by refusing to support users who have been unwilling to run and post local tests. This would give us the following: a) a scalable testing setup which could handle a Boost containing any number of libraries. b) All combinations of libraries/platforms/compilers actually being used would be those being tested and vice versa. We would have complete and efficient test coverage. c) We would have statistics on libraries being used. Something we are sorely lacking now. d) We would be encouraging better software development practices. Sometime ago someone posted that he had a problem but couldn't run the tests because "management" wouldn't allocate the time - and this was a critical human life safety app. He escaped before I could weedle out of him which company he worked. And best of all - We're almost there !!!! we'd only need to: a) enhance slightly the dependency tools we've crafted but aren't actually using. b) develop a tool to post the local results to a common dashboard c) enhance the current dashboard to accept these results. Robert Ramey

3 3

[nowide] Library Updates and Boost's broken UTF-8 codecvt facet
by Artyom Beilis 09 Oct '15

09 Oct '15

Some updated regarding boost.nowide 1. Library moved to github and its format is converted to modular boost layout: https://github.com/artyom-beilis/nowide2. Fixed unsupported std::ios::ate flag by boost::nowide::fstream3. Added some C++11 interfaces to boost::nowide::fstream4. Added integration functionality with boost::filesystem: https://github.com/artyom-beilis/nowide/blob/master/include/boost/nowide/in… And another important update is that I implemented proper utf8 to utf-16/utf-32 codecvt facet https://github.com/artyom-beilis/nowide/blob/master/include/boost/nowide/ut… It implemented as template version working with wchar_t, char16_t and char32_t. Now I explain. There is widely used utf8 codecvt facet in various parts of code: https://github.com/boostorg/detail/blob/master/include/boost/detail/utf8_co… https://github.com/boostorg/detail/blob/master/include/boost/detail/utf8_co… However it is buggy and actually broken for 3 reasons: 1. It supports UCS-2 instead of UTF-16 - i.e it does not code properly Unicode characters outside BMP i.e. code points with values abouve 0xFFFF2. It allows invalid code points in UTF-32/UCS-4 i.e. above 10FFFF or that are reserved for surrogate pairs of UTF-163. And actually allows UTF-8 sequences longer than 4 (which is wrong) As a result, for example you can't use boost::filesystem::path with characters like "𝒞" U+1D49E or may actually create wrong encodings trying to read/write filesystem objects. Independently of Boost.Nowide I would like to propose replacement of boost/detail/utf8_codecvt_facet by one thatactually takes in account proper Unicode handling. Artyom Beilis--------------CppCMS - C++ Web Framework: http://cppcms.com/CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

7 47

[Build] Test file name conflicts
by Louis Dionne 09 Oct '15

09 Oct '15

Hi, While trying to create Jamfiles for Boost.Hana's unit tests, the following situation came up. I have two unit tests in files foo/main.cpp bar/main.cpp Then, my Jamfile.v2 contains the following: import testing ; test-suite xxx : [ run foo/main.cpp ] [ run bar/main.cpp ] ; When I run `b2`, I get the following: error: No best alternative for ./main next alternative: required properties: (empty) matched next alternative: required properties: (empty) matched error: No best alternative for ./main next alternative: required properties: (empty) matched next alternative: required properties: (empty) matched I think the problem is that Boost.Build tries to create two executables named `main` in the same directory, which obviously causes a name conflict. Is there any way to avoid this, besides renaming the test files? Regards, Louis -- View this message in context: http://boost.2283326.n4.nabble.com/Build-Test-file-name-conflicts-tp4680806… Sent from the Boost - Dev mailing list archive at Nabble.com.

2 2

Boost.lockfree
by Aparna Kumta 09 Oct '15

09 Oct '15

All tests for lockfree in both master and develop branch seem to fail. Error message is "../boost/system/config.hpp", line 34: Error: #error Must not define both BOOST_SYSTEM_DYN_LINK and BOOST_SYSTEM_STATIC_LINK. See develop branch: http://www.boost.org/development/tests/develop/developer/lockfree.html See master branch: http://www.boost.org/development/tests/master/developer/lockfree.html I looked at lockfree/test/Jamfile.v2 but am not sure what change needs to be made. Any suggestions? Thanks, Aparna

17 51