May 2016 - Boost-users - lists.preview.boost.org

File-mapped shared memory access through Boost interprocess is slow on some HPC file-systems due to mmap
by Scott Ritchie 24 May '16

24 May '16

Hi all, I'm wondering if there is some way to use the Boost interprocess library without the use of mmap for file I/O? I have developed an R-language package that uses another third party package (bigmemory) to access objects in shared memory when parallelising R across multiple cores. This third party package uses the Boost interprocess library to make these file-backed shared memory objects available to R. This worked perfectly, until I tested my package on a multi-node cluster. After some digging I found that the code calling these shared memory objects was extremely slow, and had hugely variable runtimes. Accessing these shared memory objects became exponentially slower as I parallelised over more and more cores. After talking to the authors of the bigmemory library and the administrators of the cluster, we have determined the route cause to be the use of mmap() on that particular filesystem (GPFS). On this filesystem, mmap() performs concurrency checking on every read and write since the filesystem is shared across multiple physical nodes. Marking the shared memory segments as read_only did not solve the issue. The advice of the cluster administrators is to not use mmap() for file I/O. I'm hoping there is a simple fix, such as a compilation flag I can provide to boost to use some alternative file I/O function/library? If not I will need to rewrite my code so that it is multi-threaded instead of multi-process without the use of Boost. Kind regards, --- Scott Ritchie, Ph.D. Student | Integrative Systems Biology | Pathology | http://www.inouyelab.org The University of Melbourne ---

2 1

[Parallel Boost Graph Library] General Questions and Redistribution Questions
by Jun Kudo 24 May '16

24 May '16

Hi everyone, I've been playing around with the Parallel Boost Graph Library in hopes of using it for a distributed mesh with redistribution capabilities. Unfortunately, I've been having a lot of issues and I was hoping to get some clarification from the community on some points. I am trying to construct a distributed graph that represents a mesh. The idealized plan is to use this library to create a mesh class that allows for the easy introduction of new vertices and edges as well as repartitioning/data migration capabilities. For this effort, I created a simple struct called Cell that is attached as an interior vertex property and identified the cell id value as the key for indexing. I've attached a short program to really quickly demonstrate what I've been trying to prototype. At this point, I have some questions regarding some things about the library that I can't seem to figure out. 1. How do you control the initial distribution of the names? That is, how do you control which processes own what vertex names when you dynamically add named vertices? 2. How do you create a VertexProcessorMap to use with the redistribution function? The test adjlist_redist_test.cpp looks like it creates a property_map of the vertex_rank but I'm not sure how to do something similar when I have named graph. 3. Is there any way and any point to accomplish what I'm trying to do without using named graphs? I started down this route because the referenced test did not use named graphs. I quickly ran into a situation where I did not know how to grab a vertex descriptor for an off-process vertex without using the vertex(vertices_size_type n, const adjacency_list& g) function or without knowing a priori where that vertex lived. Similar to the named graph situation, I want to be able to dynamically add numbered vertices (which according to the documentation invalidates the use of vertex()) on different processes and then define edges between them based on these numbers. Does the fact that I'm trying to use a global numbering system as a name convention make this question silly? Sorry about the wall of text and I hope my questions are not too nonsensical! Thanks in advance for all the help. Sincerely, Jun

1 0

Rebooting Boost: Call for Lightning Talks @ CppCon
by Niall Douglas 23 May '16

23 May '16

At C++ Now a number of informal meetings showed there to be frustrations with the current ecosystem around the Boost C++ Libraries (to paraphrase a founder of Boost, "Boost is supposed to serve *the entire C++ community; it isn't Boost's goal to serve Boost's community*". Recent and recurrent boost-dev discussions have raised the lack of cmake based tooling; lack of ABI management and the consequent ODR violation when mixing Boost versions in the same process; the anti-social behaviour of Boost towards library end users, new ideas, new blood and the wider C++ community; and the chronic lack of maintenance of up to half the Boost libraries. One question which needs to be answered is whether a clean reboot of Boost from the first principles of proving high quality C++ libraries well suited for use with the latest C++ standard is viable. This new collection of C++ libraries would be started completely from scratch (even if some existing libraries were ported into the new organisation), so ANYTHING is possible. All we need is a critical mass of library developers willing to collaborate on developing a shared infrastructure mutually beneficial to all participants, whether library developer or library end user. To that end of discovering if this critical mass is there, five minute lightning talk topics are requested for CppCon this September on this topic: "What design pattern, practice or idiom should a standards aspiring collection of C++ 14/17 libraries share?" Please keep each talk topic focused on a single design pattern, practice or idiom (see suggested talk format below). Each speaker may submit up to THREE talk topics (not all may be accepted if there are a lack of slots, so please rank them in order of preference). Please send your talk topics to me privately by email, only if sufficient numbers present to fill a session will your name and your talk topic be shared with the CppCon Programme Committee. We'll try to arrange a short panel session at the end so attendees can ask questions of the speakers. After the lightning talk we'll arrange a restaurant so all the speakers and any interested attendees can dine together, followed by a night in a Bellevue bar. If after the dust has settled it is felt there is a critical mass sufficient to reboot a brand shiny new edition of Boost, we'll start development! I should *stress* that it is up to the CppCon Programme Committee whether these lightning talks are approved or not. I'll just copy and paste your proposed talk to the CppCon PC as-is. If insufficient lightning talk topics are received by me to fill a 60 minute session, I'll withdraw this talk proposal from CppCon. Suggested talk format ================ Five minutes is not much time, so here is a suggested lightning talk outline: 1. A current widely practiced library design pattern (show example). 2. The problems it generates for library users or maintainers. 3. Your proposed alternative design pattern (show example). 4. The problems the alternative design pattern fixes. 5. Any mockup, prototype or even live demo (!) of the alternative design pattern in action. Suggested talk themes might be documentation, packaging, distribution, testing, tooling, quality assessment, barriers to entry, governance, strategy, important gaps in the standard libraries, automation and deprecation/removal policies. Here is an example of such a lightning talk: 1. Current design pattern: Library v1.00: namespace boost { namespace lib { void foo(size_t, char); } } User code: boost::lib::foo(100, 'n'); 2. Problem: Library v1.01: namespace boost { namespace lib { void foo(char, size_t); } } User code: boost::lib::foo(100, 'n'); // oh oh! 3. Alternative: Library v1.00: namespace boost { namespace lib { inline namespace v100 { void foo(size_t, char); } } } #define BOOST_LIB_NAMESPACE_V100 boost::lib::v100 Library v1.01: namespace boost { namespace lib { inline namespace v101 { void foo(char, size_t); } } } #define BOOST_LIB_NAMESPACE_V101 boost::lib::v101 User code: // Mount v100 of Boost.Lib into "lib" namespace lib=BOOST_LIB_NAMESPACE_V100; lib::foo(100, 'n'); // ABI stable! No ODR violation possible! 4. Problems fixed: a) Mixing incompatible versions of individual Boost libraries in the same process (or even translation unit) is now safe. b) Libraries can now depend on other libraries without worrying about breaking code which depends on them which uses a different version of the same dependency. This greatly helps reduce change ripple effects. c) Forward compatibility or alternative implementation shims are now very easy to implement e.g. library users can externally configure either Boost.Filesystem or std::experimental::filesystem for your library to use, your code simply refers to _some_ Filesystem TS implementation e.g. namespace filesystem = FILESYSTEM_TS_IMPLEMENTATION; filesystem::path mypath; -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

7 7

How to write one property page .props to select the right x86 or x64 Boost libraries folder automatically using Visual Studio
by Paul A. Bristow 22 May '16

22 May '16

I have downloaded the shiny new release boost_1_61_0_b1 and am trying to set up so that I can test a lot of Boost-ish console applications using both 32 and 64-bit. This is easier with bjam, but I'd like to use the Visual studio IDE too. I'd like a single property page that can be placed in the same folder as my .sln file and added to each and every project. (or better, made the default for all projects). I have a I:\boost_1_61_0_b1; as the include directory, and that is common to all configurations and platforms. But the libraries files are in two separate folders because the name does not yet include the platform :-( Linking to lib file: libboost_chrono-vc140-mt-gd-1_61.lib lib32-msvc-14.0 and lib64-msvc-14.0 Googling gives some suggestions to select the right folder for libraries, but I wonder if anyone has found a slick solution? (I'll edit the property xml if necessary - it would seem that the IDE GUI tool isn't up to the job?) Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830

5 5

Windows 10 and VC14, fixing C2117 error when using Asio.
by Andrew Maclean 21 May '16

21 May '16

If you find that when using Asio in Windows 10 with the Windows 10 SDK and Visual Studio 2015 you get lots of C2177 errors in winnt.h and sdkdkver.h then this should fix it: Essentially, just add: #define WINVER 0x0A00 #define _WIN32_WINNT 0x0A00 before any header that includes SDKDDKVer.h either directly or indirectly. If using CMake, then in the CMakeLists.txt file just add these lines: if(WIN32) add_definitions(-DWINVER=0x0A00) add_definitions(-D_WIN32_WINNT=0x0A00) endif() The fix is documented at https://msdn.microsoft.com/en-us/library/6sehtctf.aspx I hope this helps others. Regards Andrew -- ___________________________________________ Andrew J. P. Maclean ___________________________________________

1 0

[test] Standalone use with multiple translation units?
by Adam Nielsen 20 May '16

20 May '16

Hi all, I am currently using Boost Test for my test code, however it is always an extreme headache trying to get Boost available on the target platforms I'm using. At the moment due to some environment restrictions it is just not possible to use a recent version of Boost, and since the Test library broke backwards compatibility with 1.59, my code will no longer compile on versions earlier than 1.59. So I am looking to see whether I can avoid the need to compile the full Boost, and just pull in the bits that I need. The Boost Test docs say that the library can be used in single-header mode, however this only works when there is one translation unit. Since my tests are spread across dozens of files, I don't think this will work for me. Is there any way to use this single-header mode, and flag one file as the "primary" one, and others as extras, so that you can use single-header mode across multiple translation units? This would remove the need to do battle with getting the Boost libraries installed on various platforms, which would make it much easier and allow me to spend more time on my code, and less time on writing build scripts. Many thanks, Adam.

2 2

[boost::compute] [OS X 10.11.5] [GeForce GT 750M] Segmentation fault
by Prashant 20 May '16

20 May '16

Hi, Example code to sort a vector on GPU (GeForce GT 750M) generates SEGV. However, it runs fine on Iris Pro (Intel(R) Core(TM) i7-4850HQ CPU). Having said that, it works as long as size of vector being sorted is small(<2 million). The code throws an exception if the size of the vector is less 9 million. Exception: boost/1_61_0/include/boost/compute/command_queue.hpp(453): Throw in function boost::compute::event boost::compute::command_queue::enqueue_write_buffer(const boost::compute::buffer &, size_t, size_t, const void *, const boost::compute::wait_list &) Dynamic exception type: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector <boost::compute::opencl_error> > std::exception::what: Invalid Value Another observation: if the size of vector is 50 million the sorting works fine, though the timings are worse than Iris Pro. Regards, Prashant ----------------------------Cut here------------------------------------- #include <iostream> #include <vector> #include <algorithm> #include <boost/foreach.hpp> #include <boost/compute/core.hpp> #include <boost/compute/platform.hpp> #include <boost/compute/algorithm.hpp> #include <boost/compute/container/vector.hpp> #include <boost/compute/functional/math.hpp> #include <boost/compute/types/builtin.hpp> #include <boost/compute/function.hpp> #include <boost/chrono/include.hpp> #include <boost/exception/all.hpp> namespace compute = boost::compute; int main(int argc, char* argv[]) { if (argc != 2) { std::cout << "Usage: " << argv[0] << " <size> " << std::endl; return 0; } // generate random data on the host std::vector<float> host_vector(atoi(argv[1])); std::generate(host_vector.begin(), host_vector.end(), rand); std::cout << "===============CPU==================\n"; for (size_t k=0; k<5; k++) { std::vector<float> host_copy_vector(host_vector); auto start = std::chrono::high_resolution_clock::now(); std::sort(host_copy_vector.begin(), host_copy_vector.end()); auto duration = std::chrono::duration_cast<std::chrono::milliseconds> (std::chrono::high_resolution_clock::now() - start); std::cout << "time: iteration ("<< k << ") : " << duration.count() << " ms" << std::endl; } std::vector<compute::platform> platforms = compute::system::platforms(); for(size_t i = 0; i < platforms.size(); i++){ const compute::platform &platform = platforms[i]; std::cout << "Platform '" << platform.name() << "'" << std::endl; std::vector<compute::device> devices = platform.devices(); for(size_t j = 0; j < devices.size(); j++){ const compute::device &device = devices[j]; std::string type; if(device.type() & compute::device::gpu) type = "GPU Device"; else if(device.type() & compute::device::cpu) type = "CPU Device"; else if(device.type() & compute::device::accelerator) type = "Accelerator Device"; else type = "Unknown Device"; if (type != "GPU Device") { std::cout << "Ignoring non GPU devices.\n"; continue; } std::cout << "====\n"; std::cout << " " << type << ": " << device.name() << std::endl; std::cout << "====\n"; compute::context context(device); compute::command_queue queue(context, device); for (size_t k=0; k<5; k++) { compute::vector<float> device_vector(host_vector.size(), context); // copy data from the host to the device compute::copy( host_vector.begin(), host_vector.end(), device_vector.begin(), queue ); auto start = std::chrono::high_resolution_clock::now(); try { compute::sort(device_vector.begin(), device_vector.end(), queue); } catch (boost::exception & e) { std::cerr << diagnostic_information(e); break; } auto duration = std::chrono::duration_cast<std::chrono::milliseconds> (std::chrono::high_resolution_clock::now() - start); std::cout << "time: iteration ("<< k << ") : " << duration.count() << " ms" << std::endl; } std::cout << "====\n"; } } return 0; }

2 2

Boost 1.61.0 has been released
by Rene Rivera 19 May '16

19 May '16

Release 1.61.0 of the Boost C++ Libraries is now available. These open-source libraries work well with the C++ Standard Library, and are usable across a broad spectrum of applications. The Boost license encourages both commercial and non-commercial use. This release contains four new libraries and numerous enhancements and bug fixes for existing libraries. New Libraries * Compute: Parallel/GPU-computing library, from Kyle Lutz. * DLL: Library for comfortable work with DLL and DSO, from Renato Tegon Forti and Antony Polukhin * Hana: A modern C++ metaprogramming library, from Louis Dionne. * Metaparse: A library for generating compile time parsers parsing embedded DSL code as part of the C++ compilation process, from Abel Sinkovics. For details, including download links, see http://www.boost.org/users/news/version_1.61.0 You can also download directly from SourceForge: http://sourceforge.net/projects/boost/files/boost/1.61.0/ Tom Kent has made pre-built windows binaries, and they are available at: https://sourceforge.net/projects/boost/files/boost-binaries/1.61.0/ To install this release on your system, see http://www.boost.org/doc/libs/release/more/getting_started/index.html Thanks, --The Boost release team Vladimir Prus Rene Rivera Marshall Clow Eric Niebler Daniel James Beman Dawes

5 8

DMARC and mailman
by James Sharpe 19 May '16

19 May '16

In June 2016 gmail is going to be changing its DMARC policy to reject any unaligned mail. This means that we will be seeing any mailing list responses from a gmail.com address to be considered spam by any mail server that enforces DMARC. For a while now yahoo.com has had this policy turned on which has meant that yahoo.com addresses have been considered spam. Mailman has released a workaround in version 2.1.18: https://www.mail-archive.com/mailman-users@python.org/msg64098.html The boost mailing list is currently on mailman 2.1.15. I'd strongly suggest that this should be updated as a matter of urgency else we'll find a lot of the conversations on this list will end up in the spam folder... Regards, James

1 0

[boost::python] cannot concatenate 'str' and 'property' objects
by sergei nazarenko 19 May '16

19 May '16

Hello! I have just simple question. python code: a = A() bla = "lala" + a.pp *Traceback (most recent call last):TypeError: cannot concatenate 'str' and 'property' objects* Why it is not work? How make it is work? boost c++ code: #include <boost/python/module.hpp> #include <boost/python/def.hpp> #include <boost/python/class.hpp> #include <boost/python/init.hpp> class A { public: std::string pp; A(): pp("babla") {}; }; using namespace boost::python; BOOST_PYTHON_MODULE (test1) { class_<A> ("A") .def(py::init<>()) .add_property ("pp", &A::pp) ; } Thanks, Sergiy

1 0