- Boost-users - lists.preview.boost.org

Boost cannot find numpy37 on Dockerized Ubuntu 20.04
by Roy de Bokx 17 Feb '21

17 Feb '21

Hi there, I've been trying to get GP-GOMEA running in a dockerized environment <https://github.com/marcovirgolin/GP-GOMEA/issues/2>, which is a project that is built using Boost. As part of the build process, the source code is compiled by the g++ compiler, using the flag -lboost_numpy37, which fails because (it seems) numpy cannot be found. I've reported this also as an issue also in the boost repository <https://github.com/boostorg/boost/issues/462>, but mclow advised me to send this to the boost-users mailinglist too. I've create the following Dockerfile as a minimal test case, which runs into the same problem. I'm a bit of a noob when it comes to Boost and numpy, but from what I found so far on the web, this should work (but it doesn't) FROM ubuntu:20.04 ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && apt-get upgrade -y && apt-get clean RUN apt-get install -y curl python3.7 python3-dev python3-distutils python3-numpy # Install latest boost version RUN apt-get -y install wget g++ RUN wget https://dl.bintray.com/boostorg/release/1.75.0/source/boost_1_75_0.tar.bz2 && \ tar --bzip2 -xf boost_1_75_0.tar.bz2 && \ cd boost_1_75_0 && \ # this will generate ./b2 ./bootstrap.sh --prefix=/usr/local && \ ./b2 --with=all install && \ sh -c 'echo "/usr/local/lib" >> /etc/ld.so.conf.d/local.conf' && \ ldconfig # Check numpy installation RUN find / -name "numpy.hpp" RUN find / -name "libboost_numpy37.so" RUN find / -name "libboost_numpy3.so" RUN find / -name "libboost_numpy.so" RUN ld -lboost_numpy37 --verbose This eventually results in the following error: The command '/bin/sh -c ld -lboost_numpy37 --verbose' returned a non-zero code: 1, because it could not find any libboost_numpy37.so or libboost_numpy37.o file. The full logs are attached in the GH issue linked above. Did anyone encounter something similar, or does anyone have an idea what could cause this? Any help is much appreciated! -Roy

2 2

How to change read/write method of source/sink device in C++ boost::iostreams?
by mostaan fereidooni 16 Feb '21

16 Feb '21

I'm trying to get familiar with boost::iostream, so in a example program I'm writing, I want to read a text from a file and write it to file. I would use my inherited class from file_source/file_sink as device to read/write. In the read method, my class needs to add each character with one and write method needs to subtract one from each character. First of all, I want to make sure that the reading part of the program works properly so you can see the code as follow: #include <boost/iostreams/stream.hpp>#include <fstream>#include "MyFileSource.h"#include <iostream>using namespace std;using namespace boost::iostreams;int main(){ MyFileSource<char> fileSource("source.txt"); stream<MyFileSource<char>> myStream(fileSource); std::cout << myStream.rdbuf(); fileSource.close(); } and my inherited source device code is as follow: template <typename charType>class MyFileSource : public boost::iostreams::file_source {public: MyFileSource(const std::string& path) : boost::iostreams::file_source(path) { file_path = path; if (!is_open()) open(path); seek(0, ios::end); sizeOfFile = seek(0, ios::cur); seek(0, ios::beg); buffer = new charType[sizeOfFile]; } std::streamsize read(charType*, std::streamsize) { std::streamsize readCount = boost::iostreams::file_source::read(buffer, sizeOfFile); if (readCount > 0) { std::string result(buffer, readCount); for (char& c : result) c++; finalResult = result; } return readCount; } private: string file_path; int sizeOfFile; charType* buffer; std::string finalResult; }; unfortunately, the result of *std::cout << myStream.rdbuf()* with above read method is nothing while if I delete the read method of MyFileSource class to use the read method of parent, the result will be correct. Any help will be appreciated.

1 0

How to link boost library DLLs?
by Anand Arumugam 14 Feb '21

14 Feb '21

Hello all, How to properly link boost library DLLs? I have #define-d the following: #define BOOST_ALL_DYN_LINK #define BOOST_DYN_LINK #define BOOST_THREAD_USE_DLL #define BOOST_THREAD_PROVIDES_FUTURE_CONTINUATION #define BOOST_THREAD_VERSION 4 And I still get the following problem: --------------------------- testprogs.exe - System Error --------------------------- The code execution cannot proceed because boost_thread-vc142-mt-gd-x64-1_75.dll was not found. Reinstalling the program may fix this problem. --------------------------- OK --------------------------- I get the same error when I try to run the program using DLLs that I built and while running using the prebuilt binaries. Not sure what step I'm missing. The program runs fine if I build using static libs. The problem is only with DLLs. Any help is much appreciated. Thanks, Anand.

3 3

Boost.Asio as "thread-pool": How to apply back-pressure?
by Dominique Devienne 10 Feb '21

10 Feb '21

Hi, I'm using Boost.ASIO, not for networking, but simply to parallelize work items, with a simple graph of "processing nodes" as detailed below. It's working fine, but uses too much memory. I'd like insights on limiting memory use via "throttling", or what I've also seen called "back-pressure". At a high level, I process two files (A and B) composed of several "entries" each, extracting a subset or a transformation of those entries, that I then write into an output file (C). (those 3 files reach into the many GBs, thus the need for parallelism, and limiting memory use). Extracting entries from A and B are independent operations, implemented single-threaded, producing independent work items (for subsetting or transforming each item), A#1...A#n, and B#1..B#m. That's the "fan-out" part, with each work-item (task) scheduled on any thread of the ASIO pool, since independent. Writing to C is also single-threaded, and needs to "fan-in" the work posted by the A#n and B#m functors, and I serialize that via posting to a C-specific strand (still on any thread, as long as serialized, doesn't matter). Lets call all those tasks writing to C the C#n+m tasks, which are posted to the strand via the A#s and B#s. My issue is that Boost.Asio seems to schedule an awful lots of A# and B# tasks, before getting to the C# tasks, which results in accumulating in memory too many C# tasks, and thus using too much memory. I don't see a way to force more C "downstream" tasks to be scheduled, before processing so much A and B tasks, and accumulating pending C tasks in the work queue, thus using too much memory again. Could someone please recommend a way to have a more balanced flow of tasks in the graph? Or alternative designs even, if what I do above is not ideal. Is Boost.Asio even suitable in this case? Thanks, --DD

7 17

[boost-process] why does my child process not get EOF on stdin
by harald 01 Feb '21

01 Feb '21

Dear community, dear experts! I am trying to use boost::process to call weasyprint to convert HTML to a picture. To do that I wrote a test program (further down in the mail). This worked fine and did what i expected. But it stopped working as soon as I tried to convert two files at the same time by changing new renderer_t(io_context, "<head></head><body></body>"); in main to new renderer_t(io_context, "<head></head><body></body>"); new renderer_t(io_context, "<head></head><body></body>"); Now weasyprint hangs. I changed the call to weasyprint to weasyprint2, which is a short bash script for testing purposes: --------------------------------------------------------------------------------- #! /bin/bash cat > /dev/null echo "This should be a PNG picture" --------------------------------------------------------------------------------- It reads stdin until it gets EOF and then writes something to stdout before terminating. Same result. It seems that my program does not close stdin of the child, so it never gets EOF on stdin. But only when more than one child process is running at the same time. So obviously I am doing something wrong, but I have no idea what! Does anybody have a hint for me? I am on xubuntu 20.10 compiler is g++ (Ubuntu 10.2.0-13ubuntu1) 10.2.0, boost version 1.75.0 compiler command line (from my IDE): g++ -c main.cpp -g -O0 -Wall -Wextra -std=c++17 -o build-Debug/main.cpp.o -I. -I. --------------------------------------------------------------------------------- #include <boost/smart_ptr/intrusive_ptr.hpp> #include <boost/smart_ptr/intrusive_ref_counter.hpp> #include <boost/process.hpp> #include <boost/asio/write.hpp> #include <iostream> namespace bp = boost::process; bool is_eof( std::error_code error) { boost::system::error_code eof = boost::asio::error::eof; return error == eof; } struct renderer_t : public boost::intrusive_ref_counter<renderer_t> { using pointer = boost::intrusive_ptr<renderer_t>; boost::asio::io_context &io_context; const std::string html; bp::child child; bp::async_pipe in; bp::async_pipe out; bp::async_pipe err; std::array<unsigned char, 256> outbuf; std::array<char, 256> errbuf; std::vector<unsigned char> result; std::error_code result_error; std::string errmsg; ~renderer_t() { std::cout << "renderer_t::~renderer_t\n"; if (!errmsg.empty()) std::cerr << "renderer_t::~renderer_t: stderr: " << errmsg << "\n"; } renderer_t( boost::asio::io_context &io_context, const std::string &html) : io_context(io_context), html(html), in(io_context), out(io_context), err(io_context) { post( io_context, [self = pointer(this)]() { self->start(); }); } void start() { std::error_code ec; std::cout << "renderer_t::start\n"; bp::child c( "weasyprint2 -m screen -p -f png - /dev/stdout", bp::std_in < in, bp::std_out > out, bp::std_err > err, bp::on_exit=[self = pointer(this)]( int exit, std::error_code error) { self->on_exit(error, exit); }, bp::start_dir="/data/build/", io_context, ec); if (ec) { std::cerr << "renderer_t::start: creating child failed: " << ec.message() << "\n"; if (!result_error) result_error = ec; return; } child = std::move(c); boost::asio::async_write( in, boost::asio::buffer(html), [self = pointer(this)]( std::error_code error, std::size_t bytes) { self->on_write(error, bytes); }); out.async_read_some( boost::asio::buffer(outbuf), [self = pointer(this)]( std::error_code error, std::size_t bytes) { self->on_read_out(error, bytes); }); err.async_read_some( boost::asio::buffer(errbuf), [self = pointer(this)]( std::error_code error, std::size_t bytes) { self->on_read_err(error, bytes); }); } void on_write( std::error_code error, std::size_t bytes) { std::cout << "renderer_t::on_write\n"; if (error) { std::cerr << "renderer_t::on_write: failed: " << error.message() << "\n"; if (!result_error) result_error = error; return; } if (bytes != html.size()) { std::cerr << "renderer_t::on_write: short write\n"; } boost::system::error_code ec; in.close(ec); if (ec) { std::cerr << "renderer_t::on_write: close failed: " << error.message() << "\n"; } } void on_read_out( std::error_code error, std::size_t bytes) { std::cout << "renderer_t::on_read_out\n"; if (is_eof(error)) { std::cout << "renderer_t::on_read_out: EOF\n"; return; } else if (error) { std::cerr << "renderer_t::on_read_out: failed: " << error.message() << "\n"; if (!result_error) result_error = error; return; } else { result.insert( end(result), begin(outbuf), begin(outbuf) + bytes); out.async_read_some( boost::asio::buffer(outbuf), [self = pointer(this)]( std::error_code error, std::size_t bytes) { self->on_read_out(error, bytes); }); } } void on_read_err( std::error_code error, std::size_t bytes) { std::cout << "renderer_t::on_read_err\n"; if (is_eof(error)) { std::cout << "renderer_t::on_read_err: EOF\n"; return; } else if (error) { std::cerr << "renderer_t::on_read_err: failed: " << error.message() << "\n"; if (!result_error) result_error = error; return; } else { std::string e(begin(errbuf), begin(errbuf) + bytes); std::cerr << "renderer_t::on_read_err: from stderr: " << e << "\n"; errmsg.insert( end(errmsg), begin(errbuf), begin(errbuf) + bytes); err.async_read_some( boost::asio::buffer(errbuf), [self = pointer(this)]( std::error_code error, std::size_t bytes) { self->on_read_err(error, bytes); }); } } void on_exit( std::error_code error, int exit) { std::cout << "renderer_t::on_exit\n"; if (error) { std::cerr << "renderer_t::on_exit: error: " << error.message() << "\n"; if (!result_error) result_error = error; return; } if (exit) { std::cerr << "renderer_t::on_exit: weasyprint returned " << exit << "\n"; if (!result_error) result_error = make_error_code(std::errc::no_such_process); } } }; int main() { boost::asio::io_context io_context; bp::async_pipe in(io_context); new renderer_t(io_context, "<head></head><body></body>"); io_context.run(); return 0; } --------------------------------------------------------------------------------- Regards Harald

1 0

TOMS748 question
by Shriramana Sharma 29 Jan '21

29 Jan '21

Namaste. This is wrt the TOMS748 math root finding implementation: Given that `if(fa == 0)` is handled at: https://github.com/boostorg/math/blob/develop/include/boost/math/tools/toms… … and the function returns in case of the above condition, it is not clear to me what the need is for the `if (fa != 0)` at: https://github.com/boostorg/math/blob/develop/include/boost/math/tools/toms… Note that I do not find the condition being applied at the SciPy implementation of this algorithm at: https://github.com/scipy/scipy/blob/v1.6.0/scipy/optimize/zeros.py#L1220 Now I also note similar checks at: https://github.com/boostorg/math/blob/develop/include/boost/math/tools/toms… https://github.com/boostorg/math/blob/develop/include/boost/math/tools/toms… … but regarding those lines, while I can understand the need for checking there after more bracketing work is done, it is not clear to me why `(fa == 0)` is alone checked for, and not `(fb == 0)` as well. I would have thought the algorithm is not biased towards a or b. Indeed, in the last steps from: https://github.com/boostorg/math/blob/develop/include/boost/math/tools/toms…, … both the conditions are checked for, which is what I would expect even in the intervening steps. Now the lines seem to have originated in the very first commit implementing this algorithm: https://github.com/boostorg/math/commit/79a8199f9c26ba353fb52ec4543f75ffc2a… … but I do not see a way to contact the author jzmaddock other than posting to this list, so I hope someone/they see here and reply. Thanks! -- Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा 𑀰𑁆𑀭𑀻𑀭𑀫𑀡𑀰𑀭𑁆𑀫𑀸

2 7

Correct way to handle error in boost::filesystem::recursive_directory_iterator::increment()?
by Dominique Pellé 23 Jan '21

23 Jan '21

Hi I see code like this in an application to iterate over files in a directory with boost::filesystem (v1.70) without using exceptions: boost::filesystem::path dir("somedir"); boost::system::error_code error; boost::filesystem::directory_iterator node(dir, error), end; if (error) { return ...; } for (; node != end; node.increment(error)) { const boost::filesystem::path& path = node->path(); // ... } It appears to work... or mostly work since a crash was reported and the stack points to this area of the code. It's a rare crash that I could not reproduce myself. I suspect that there is a bug in the above code and it should instead be: boost::filesystem::path dir("somedir"); boost::system::error_code error; boost::filesystem::directory_iterator node(dir, error), end; for (; !error && node != end; node.increment(error)) { const boost::filesystem::path& path = node->path(); // ... } In other words, I suspect that we need to check that node.increment(error) in the for loop did not give an error before checking the condition "node != end", or else we may enter the loop in case of error and then it may access node->path() with an invalid node iterator. I could not find in the doc at ... https://www.boost.org/doc/libs/1_75_0/libs/filesystem/doc/reference.html ... what exactly happens when node.increment(error) gives an error. Perhaps the doc deserves clarification or an example illustrating how to reliably iterate over files without using exceptions would be welcome. Thanks Dominique

1 1

Serialization and static Libraries
by Wolfgang Meyerle 20 Jan '21

20 Jan '21

Hi, probably has been asked already thousand of times but I'm stuck currently with some code: Assuming I have the following files (The Base class Action is serialized in the same way): File: Ping.h #include"Action.h" classPing:publicAction{ public: voidoperator()(); Ping(); Ping*clone()const; ~Ping(); template<typenameArchive> voidserialize(Archive&ar,constunsignedint); }; BOOST_CLASS_EXPORT_KEY2(Ping,"Ping"); File: Ping.cpp #include"Ping.h" #include<iostream> #include<boost/serialization/base_object.hpp> #include<boost/serialization/export.hpp> #include<boost/serialization/tracking.hpp> #include"Action.h" #include"system.h" #include"pong.h" #include"serializerdeserializer.h" #include"system.h" #include"connection.h" #include"communication.h" voidPing::operator()(){ } Ping::Ping() { } Ping*Ping::clone()const{ Ping*ping=newPing(*this); returnping; } template<typenameArchive> voidPing::serialize(Archive&ar,constunsignedint){ ar&BOOST_SERIALIZATION_BASE_OBJECT_NVP(Action); } Ping::~Ping(){} BOOST_CLASS_EXPORT_IMPLEMENT(Ping); BOOST_CLASS_TRACKING(Ping,boost::serialization::track_never); If I now compile the project into a library. And link this library to a different project. If I try to serialize the class now it doesn't work. I get the following error: unregistered class - derived class not registered or exported According to the boost documentation I need to use BOOST_CLASS_EXPORT_KEY in the header file and BOOST_CLASS_EXPORT_IMPLEMENT in the corresponding cpp file. What did I miss? Regards, Wolfgang

1 0

[Boost-MPI][Boost-ODEINT] Running odeint within MPI-context without using mpi_state
by Roland Richter 19 Jan '21

19 Jan '21

Hei, I am currently using the odeint-solver for calculating the right hand side of a non-linear equation. For speeding up the calculation of the right hand side, I decided to test the MPI-implementation. Now, after I would prefer to keep the code backwards compatible, I did not want to replace state_type with mpi_state, and instead split up the initial vector by hand before calling odeint without telling odeint that I am operating within an MPI context. I take care of all communications which involve the vectors, and therefore I assumed that this approach should work (especially after it worked correctly in small test programs). Nevertheless, I encountered the issue that after a certain amount of steps the step size between the different threads starts to differ by a small amount, even though I am in the same iteration. As example: step number rank step size n 1 0.00025 n 0 0.00025 //Ok n + 1 1 0.00051152 n + 1 0 0.000511523 //Not ok I checked my program with valgrind for possible memory leaks/corruption which could overwrite something, but nothing relevant came up. The problem is repeatable. Therefore, is the general idea correct at all, or am I doing something wrong here by neglecting a possible communication between the different threads within odeint? Thanks! Regards, Roland

2 2

Conversion numeric::array in numpy::ndarray
by philippe DALET 17 Jan '21

17 Jan '21

Hello, It's my first mail on this maillist. I started to upgrade a package for python 3.7, 3.8, 3.9 (linux and windows): vpython-wx. One file gives me some problems num_util.hpp My context: ubuntu 20.04: boost 1.71 python3.7 windows 10: boost 1.75 python3.7, 3.8, 3.9 I want to create a double_array class: Original version --------------------- namespace cvisual { namespace python { using boost::python::numeric::array; class double_array : public array { public: explicit double_array( const boost::python::handle<>& h ) : array(h) {} }; array makeNum(npy_intp n, NPY_TYPES t =NPY_DOUBLE); New version ----------------- namespace cvisual { namespace python { using boost::python::numpy::ndarray; class double_array : public ndarray { public: explicit double_array( const boost::python::handle<>& h ) : ndarray(h) {} }; ndarray makeNum(npy_intp n, NPY_TYPES t =NPY_DOUBLE); error message with VC2019 community error conversion 'const boost::python::handle<PyObject>' in 'boost::python::detail::borrowed_reference' error message with gcc 9.3 no matching function for call to ‘boost::python::numpy::ndarray::ndarray(const boost::python::handle<>&)’ Thanks a lot for your help. Philippe DALET Laboratoire de Physique Lyp Champollion, Figeac FRANCE

1 0

Re: [Boost-users] boost::regex: regex meta data or AST for regex?
by Frank Bergemann 16 Jan '21

16 Jan '21

On 16.12.20 18:05, Frank Bergemann via Boost-users wrote: > in concrete: > Is it possible to not only get the #no of groups (sub-expressions) via > basic_regex::mark_count(), but also their position (begin() / end() > iterators) in the given regex string (c'tor argument)? > > more general: > Can an AST be created for a the given regex string (c'tor argument)? > E.g with boost::spirit? I am busy with #2 parsers: 1) RegexParser: Intended to be a full regular expression parser https://github.com/F-Bergemann/RegexParser 2) RegexSplitter: to just break up regular expression into substrings for top level regex-group & non-regex-group substrings: https://github.com/F-Bergemann/RegexSplitter It's just a start and so much wrong within, i know. But i have one concrete question for the RegexSplitter: How can i avoid in RegexSplitter.cpp to use #157 ( tok_element >> *tok_element ) [ qi::_val = phx::new_<ASTNode> (qi::_1, qi::_2, ASTNode::COLLECTION_c) ]; I want to pass a SINGLE std::vector<ATSNode *> only to the receiving #51 // COLLECTION_c #52 ASTNode::ASTNode( #53 ASTNode const * first, #54 std::vector<ASTNode *> & others, #55 Type const type) without the SEPARATE parameter 'first'. I.e. i need to "merge" qi::_1 and qi::_2.

1 0

Re: [Boost-users] boost ptr_map memory access violation
by roland kapl 08 Jan '21

08 Jan '21

Dear All! After finding some inspiration on https://stackoverflow.com/questions/4088387/how-to-store-pointers-in-map , I've succeeded storing the property tree pointers in a std::map... std::string TreeizeRelD::writeTree(const std::vector<std::vector<std::string>> &control, const std::vector<std::vector<std::vector<std::string>>> &data, pt::ptree & ptTree) { // now the map is local... typedef std::map<std::string, pt::ptree*> treeMap; std::map<std::string, treeMap> parentTableTrees; .... std::string subRootNode = parentNode + (subnodeOfParent != "" ? "." + subnodeOfParent : ""); std::string tableLookup = subRootNode + "." + rootElemRec; // simply add records if no foreign key name exists (usually the top (root) table) if (foreignKeyName == "") { for (std::pair<std::string, pt::ptree> && keyPair : tables[tableLookup]) { parentTableTrees[subRootNode].insert(std::make_pair(keyPair.first, &ptTree.add_child(subRootNode, keyPair.second))); ... // iterate through all parent key record collections referred to by subtables parentNode (rootnode + optional subnode) for (std::pair<std::string, pt::ptree*> && parentKeysPair : parentTableTrees.find(lookupNode)->second) { // iterate through all records (ptrees) in key collection for (pt::ptree::iterator parentRecPair = parentKeysPair.second->begin(); parentRecPair != parentKeysPair.second->end(); ++parentRecPair) { // important here: assign a reference to parentRecPtree (pointer!), otherwise a copy is created! pt::ptree &parentRecPtree = parentRecPair->second; ... parentTableTrees[subRootNode].insert(std::make_pair(rowsFK, &parentRecPair->second.get_child(subnodeFRec))); ... parentTableTrees[subRootNode].insert(std::make_pair(rowsFK, &parentRecPtree.put_child(subnodeOfParent, foreignRecordset))); The complete source of this: https://github.com/rkapl123/OreControl/blob/master/OreMgr/TreeizeRelD/treei… -regards, Roland ---------------------------------------------------------------------- Message: 1 Date: Wed, 6 Jan 2021 11:49:07 +0000 From: roland kapl <roland.kapl(a)hotmail.com<mailto:roland.kapl@hotmail.com>> To: "boost-users(a)lists.boost.org<mailto:boost-users@lists.boost.org>" <boost-users(a)lists.boost.org<mailto:boost-users@lists.boost.org>> Subject: [Boost-users] boost ptr_map memory access violation (stack/heap problem) Message-ID: <AM0PR03MB424285F7832EDC3C73C96860EDD00(a)AM0PR03MB4242.eurprd03.prod.outlook.com<mailto:AM0PR03MB424285F7832EDC3C73C96860EDD00@AM0PR03MB4242.eurprd03.prod.outlook.com>> Content-Type: text/plain; charset="us-ascii" Dear Boost-Experts! I'm trying to use a ptr_map along with boost property trees. I need to store pointers to specific parts of a built up property tree to access these parts later on for easy insertion there. However, when the ptr_map is cleared/destroyed, I get memory access violations/stack overflow messages for my test cases. The ptr_map (typedef treeMap) is globally defined within a std::map to a ptr_map to property trees as follows (I know this isn't good practice, but passing it as a reference didn't make a difference, so I left it that way): namespace pt = boost::property_tree; typedef boost::ptr_map<std::string, pt::ptree> treeMap; std::map<std::string, treeMap> parentTableTrees; The property tree is defined as follows in this function (this builds a property tree from tabular data in the passed table variable "data" using definitions form table variable "control"): std::string TreeizeRelD::writeTreeAndCreateXML(const std::vector<std::vector<std::string>>& control, const std::vector<std::vector<std::vector<std::string>>>& data, int *result) { std::string resultString; pt::ptree &propTree = pt::ptree(); std::string returnStr = TreeizeRelD::writeTree(control, data, propTree); *result = 0; if (returnStr != "") { *result = 1; return returnStr; } returnStr = TreeizeRelD::createXML(propTree, resultString); if (returnStr != "") { *result = 1; return returnStr; } // here the memory access violation occurs: parentTableTrees.clear(); return resultString; } In function writeTree the propTree (being passed by ref as ptTree) is populated in following three lines: std::string TreeizeRelD::writeTree(const std::vector<std::vector<std::string>> &control, const std::vector<std::vector<std::vector<std::string>>> &data, pt::ptree & ptTree) ... // FIRST LINE parentTableTrees[subRootNode].insert(keyPair.first, &ptTree.add_child(subRootNode, keyPair.second)); ... // iterate through all parent key record collections referred to by subtables parentNode (rootnode + optional subnode) for (treeMap::value_type && parentKeysPair : parentTableTrees.find(lookupNode)->second) { // iterate through all records (ptrees) in key collection for (pt::ptree::iterator parentRecPair = parentKeysPair.second->begin(); parentRecPair != parentKeysPair.second->end(); ++parentRecPair) { pt::ptree &parentRecPtree = parentRecPair->second; ... // SECOND LINE // parentRecPair is an iterator variable from the ptr_map: parentTableTrees[subRootNode].insert(rowsFK, &parentRecPair->second.get_child(subnodeFRec)); ... // THIRD LINE // parentRecPtree is the referenced property tree itself: parentTableTrees[subRootNode].insert(rowsFK, &parentRecPtree.put_child(subnodeOfParent, foreignRecordset)); I already have the suspicion that the memory access violation comes from ptr_map taking ownership of the pointers (references) passed to it in the above three lines and that these are all created on the stack and not the heap, where ptr_map expects them to be. When the ptr_map is finally destroyed, it tries to access memory, that's impossible to access... But I didn't manage to successfully create heap objects that also do NOT copy the property tree objects, which is important as I only need references to the property trees. Any help is appreciated. -regards, Roland

1 0

boost ptr_map memory access violation (stack/heap problem)
by roland kapl 06 Jan '21

06 Jan '21

Dear Boost-Experts! I'm trying to use a ptr_map along with boost property trees. I need to store pointers to specific parts of a built up property tree to access these parts later on for easy insertion there. However, when the ptr_map is cleared/destroyed, I get memory access violations/stack overflow messages for my test cases. The ptr_map (typedef treeMap) is globally defined within a std::map to a ptr_map to property trees as follows (I know this isn't good practice, but passing it as a reference didn't make a difference, so I left it that way): namespace pt = boost::property_tree; typedef boost::ptr_map<std::string, pt::ptree> treeMap; std::map<std::string, treeMap> parentTableTrees; The property tree is defined as follows in this function (this builds a property tree from tabular data in the passed table variable "data" using definitions form table variable "control"): std::string TreeizeRelD::writeTreeAndCreateXML(const std::vector<std::vector<std::string>>& control, const std::vector<std::vector<std::vector<std::string>>>& data, int *result) { std::string resultString; pt::ptree &propTree = pt::ptree(); std::string returnStr = TreeizeRelD::writeTree(control, data, propTree); *result = 0; if (returnStr != "") { *result = 1; return returnStr; } returnStr = TreeizeRelD::createXML(propTree, resultString); if (returnStr != "") { *result = 1; return returnStr; } // here the memory access violation occurs: parentTableTrees.clear(); return resultString; } In function writeTree the propTree (being passed by ref as ptTree) is populated in following three lines: std::string TreeizeRelD::writeTree(const std::vector<std::vector<std::string>> &control, const std::vector<std::vector<std::vector<std::string>>> &data, pt::ptree & ptTree) ... // FIRST LINE parentTableTrees[subRootNode].insert(keyPair.first, &ptTree.add_child(subRootNode, keyPair.second)); ... // iterate through all parent key record collections referred to by subtables parentNode (rootnode + optional subnode) for (treeMap::value_type && parentKeysPair : parentTableTrees.find(lookupNode)->second) { // iterate through all records (ptrees) in key collection for (pt::ptree::iterator parentRecPair = parentKeysPair.second->begin(); parentRecPair != parentKeysPair.second->end(); ++parentRecPair) { pt::ptree &parentRecPtree = parentRecPair->second; ... // SECOND LINE // parentRecPair is an iterator variable from the ptr_map: parentTableTrees[subRootNode].insert(rowsFK, &parentRecPair->second.get_child(subnodeFRec)); ... // THIRD LINE // parentRecPtree is the referenced property tree itself: parentTableTrees[subRootNode].insert(rowsFK, &parentRecPtree.put_child(subnodeOfParent, foreignRecordset)); I already have the suspicion that the memory access violation comes from ptr_map taking ownership of the pointers (references) passed to it in the above three lines and that these are all created on the stack and not the heap, where ptr_map expects them to be. When the ptr_map is finally destroyed, it tries to access memory, that's impossible to access... But I didn't manage to successfully create heap objects that also do NOT copy the property tree objects, which is important as I only need references to the property trees. Any help is appreciated. -regards, Roland

1 0

[statechart] Event dispatching for orthogonal states
by B A 03 Jan '21

03 Jan '21

Hello, I have following code : ///////////////////////////////////////////// #include <iostream> #include <boost/mpl/list.hpp> #include <boost/statechart/simple_state.hpp> #include <boost/statechart/state_machine.hpp> #include <boost/statechart/event.hpp> #include <boost/statechart/transition.hpp> #include <boost/statechart/custom_reaction.hpp> namespace sc = boost::statechart; struct Event1 : sc::event<Event1> {}; struct Event11 : sc::event<Event11> {}; struct Meta; struct Machine : sc::state_machine<Machine, Meta> {}; struct Meta2 : sc::simple_state<Meta2, Machine> { Meta2() { std::cout << "Meta2\n"; } }; struct Inner1; struct Inner2; struct Meta : sc::simple_state<Meta, Machine, boost::mpl::list<Inner1, Inner2>> { Meta() { std::cout << "Meta\n"; } typedef sc::transition<Event1, Meta2> reactions; }; struct Inner11 : sc::simple_state<Inner11, Meta::orthogonal<0>> { Inner11() { std::cout << "Inner11\n"; } typedef sc::custom_reaction<Event1> reactions; sc::result react(const Event1& ev) { std::cout << "Inner1 reaction\n"; return forward_event(); } }; struct Inner1 : sc::simple_state<Inner1, Meta::orthogonal<0>> { Inner1() { std::cout << "Inner1\n"; } typedef sc::transition<Event11, Inner11> reactions; }; struct Inner2 : sc::simple_state<Inner2, Meta::orthogonal<1>> { Inner2() { std::cout << "Inner2\n"; } }; int main() { Machine m; m.initiate(); m.process_event(Event11()); m.process_event(Event1()); return 0; } //////////////////// That gives me output: Meta Inner1 Inner2 Inner11 Meta2 The goal was to get Event1 processed first in Inner11 state and then forward it to outer state Meta for further processing but looks like event dispatching does not work as expected. I can make it work in that way only if I put Event1 handling into Inner1 state. I'm aware of limitation (https://www.boost.org/doc/libs/1_61_0/libs/statechart/doc/rationale.html#Li… #Event dispatch to orthogonal regions) but as far I understand that considers that case when both Inner1 and Inner11 have same event processing, please correct me if I'm wrong. Is current dispatching work correctly for my code? Regards, Andrew

1 0

BOOST_DLL_ALIAS - non Latin AliasName
by Nicholas Yue 02 Jan '21

02 Jan '21

Hi, The examples I have seen so far appears to use ASCII name for AliasName I am considering providing AliasName in languages like Chinese, Korean, Japanese, Thai, Tamil, does Boost DLL allow for wstring/UNICODE as AliasName ? Cheers -- Nicholas Yue Graphics - Arnold, Alembic, RenderMan, OpenGL, HDF5 Custom Dev - C++ porting, OSX, Linux, Windows http://au.linkedin.com/in/nicholasyue https://vimeo.com/channels/naiadtools

1 0

shutdown( ) blocks forever
by Fariya F 30 Dec '20

30 Dec '20

I have a server that accepts ssl connections on port 443. I am using the boost libraries for the server implementation. Below is the code snippet: ssl_socket socket_; { // Open the acceptor with the option to reuse the address (i.e. SO_REUSEADDR). boost::asio::ip::tcp::resolver resolver(io_service_); boost::asio::ip::tcp::resolver::query query(address, port); boost::asio::ip::tcp::endpoint endpoint = *resolver.resolve(query); acceptor_.open(endpoint.protocol()); acceptor_.set_option(boost::asio::ip::tcp::acceptor::reuse_address(true)); acceptor_.bind(endpoint); acceptor_.listen(1000); start_accept(); } void server::start_accept() { new_connection_.reset( new connection(io_service_, connection_manager_, request_handler_, context_)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&server::handle_accept, this, boost::asio::placeholders::error)); } void server::handle_accept(const boost::system::error_code& e) { // Check whether the server was stopped by a signal before this completion // handler had a chance to run. if (!acceptor_.is_open()) { return; } else { start(); } start_accept(); } void server::start() { socket_.async_handshake(boost::asio::ssl::stream_base::server, boost::bind(&connection::handle_handshake, this, boost::asio::placeholders::error)); if (boost::asio::placeholders::error) { // printf("Error is there. Check it\n"); } } void connection::handle_handshake(const boost::system::error_code& error) { if (!error) { DEBUG("async read some"); socket_.async_read_some(boost::asio::buffer(buffer_), boost::bind(&connection::handle_read, shared_from_this(), boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred)); } else { std::cout << "Error number: " << error.value(); std::cout << " Error message: " << error.message(); std::cout << " Error category: " << error.category().name() << std::endl; delete this; } } When I initiate multiple clients to connect to the server, most of the time the connection succeeds i.e TCP handshake followed by SSL handshake succeeds and I am able to do data transfer. However, there are times when the TCP handshake succeeds (as seen by wireshark) but the SSL request isn't responded to by my server. I am able to see the SSL handshake request coming from the client onto my server, but the server doesn't respond, neither is my handle_accept() function gets called. When I check the output of netstat, I see a lot of data pending in the Recv-Q. Below is the output of netstat -atn command on the server during one such scenario: Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 15 0 0.0.0.0:443 0.0.0.0:* LISTEN tcp 0 213 10.110.112.1:443 10.110.112.2:52363 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52624 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52581 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52646 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52749 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52824 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52805 ESTABLISHED tcp 180 0 10.110.112.1:443 10.110.112.2:52645 ESTABLISHED tcp 0 471 10.110.112.1:443 10.110.112.2:52362 ESTABLISHED tcp 1 0 10.110.112.1:443 10.110.112.3:39129 CLOSE_WAIT tcp 183 0 10.110.112.1:443 10.110.112.3:39130 CLOSE_WAIT tcp 180 0 10.110.112.1:443 10.110.112.2:52595 ESTABLISHED tcp 183 0 10.110.112.1:443 10.110.112.3:39131 CLOSE_WAIT Upon further debugging, I see that the call to ssl socket member function shutdown() blocks sometimes forever and sometimes for at least 5 - 10 minutes due to which no further data is taken in and the recv-q indicates pending bytes. This is how my write handler is. Any help on where I am going wrong would be greatly appreciated: void connection::handle_write(const boost::system::error_code& e) { if (!e) { boost::system::error_code ec; socket_.shutdown(ec); //Code blocks here indefinitely } } Any reasons why the socket_.shutdown( ) blocks. How could I specify a timeout such that the shutdown( ) returns back after some time? Any other way of handling this? ( I know there is async_shutdown( ) but I prefer using shutdown( ) blocking handler) Any help would be greatly appreciated. Regards, Fariya

3 3

[json] Using boost.json with boost.beast
by Sean Farrow 22 Dec '20

22 Dec '20

Hi, Are there any samples of using boost.json to deserialize boost.beast responses? Thanks, Sean.

2 2

[Accumulators] How to extract P90, P99?
by Dominique Devienne 21 Dec '20

21 Dec '20

I'm already using the accumulator below to keep track of request processing times of a server. using ProcAcc = acc::accumulator_set< double, acc::stats< acc::tag::min, acc::tag::max, acc::tag::mean, acc::tag::median, acc::tag::sum > >; But now I'm interested in also extracting the P90 and P99 values, to get insight in how much of an outlier the max could be. AFAIK, the median is the P50, but it's not readily obvious to me what tag to use for the P90 or P(N) in general. Could someone help please? Thanks, --DD PS: Does accumulator allow to keep the top(N) values as well? How?

2 2

boost::hash_value not picking up std::hash specialisations
by Marc Mutz 21 Dec '20

21 Dec '20

Hi, I'm trying to move away from Qt containers in my source code, and Qt 5.15 makes that a bit easier by providing specialisations for some its types. Now I have a struct S { QLatin1String a, b; } that I'd like to hash. I can use boost::hash_combine to implement std::hash<S>, but S is actually a nested type, and with qHash(), I could just write an overload as a hidden friend, and I figured I could do the same with hash_value() (at the cost of having to specify boost::hash<S> as a unordered_map template argument). But hash_combine seems to insist on hash_value which in turn doesn't look at std::hash at all: size_t seed = 0; boost::hash_combine(seed, key.a); // ERROR: no hash_value overload for QLatin1String boost::hash_combine(seed, key.b); // Ditto return seed; expected: calls std::hash<QLatin1String>{}::op() (if it exists). It's trivial to write hash_value(QLatin1String), of course, but what's the general strategy here? Shouldn't hash_combine() look for std::hash<T> first? And, speaking as a Qt developer: do I really need to provide qHash(), std::hash<T> _and_ hash_value(T) for every hashable T?! Thanks, Marc

1 1

Weight as a function of distance at source
by Amit Prakash Ambasta 18 Dec '20

18 Dec '20

Hi, Looking at BGL implementations, weight, compare and combine are used in examine edge(e) as compare(combine(old_distance, get(weight, e)), old_distance) Additionally, old_distance = m_zero to check for negative weights etc. In my implementation, edges have a bundled property struct EdgeProp { Distance weight(Distance start) { ... } }; that returns the cumulative weight on iterating over the edge given a start distance. To use this, I should be able to pass a combine function of the form []<typename BinaryFunction>(DistanceType d, BinaryFunction f) { return f(d); } However, this approach falls apart. I've tried putting together a formal example here, but not sure why this fails. (My actual use case is where weights represent nodes in a transport system and for a person arriving at a vertex at some time Tx, there is a variable weight of using the next outbound transport = waiting_time + travel_time, where waiting_time is a function of Tx #include <boost/graph/adjacency_list.hpp> #include <boost/graph/dijkstra_shortest_paths.hpp> #include <limits> struct VProp {}; struct EProp { int multiplier; int weight(int src_distance) { return src_distance * multiplier; } }; typedef boost:: adjacency_list<boost::vecS, boost::vecS, boost::directedS, VProp, EProp> Graph; template<class G> using Vertex = boost::graph_traits<G>::vertex_descriptor; template<class G> using Edge = boost::graph_traits<G>::edge_descriptor; int main() { Graph graph; auto source = boost::add_vertex(VProp{}, graph); auto target = boost::add_vertex(VProp{}, graph); boost::add_edge(source, target, EProp{ 3 }, graph); auto index_map = get(boost::vertex_index, graph); std::vector<Vertex<Graph>> predecessors(boost::num_vertices(graph)); auto predecessor_map = boost::make_iterator_property_map(predecessors.begin(), index_map); std::vector<int> distances(boost::num_vertices(graph)); auto distance_map = boost::make_iterator_property_map(distances.begin(), index_map); auto compare = [](int lhs, int rhs) -> bool { return lhs < rhs; }; auto combine = []<typename BinaryFunction>(int dst, BinaryFunction f) -> int { return f(dst); }; boost::dijkstra_shortest_paths( graph, source, predecessor_map, distance_map, boost::weight_map(boost::get(&EProp::weight, graph)), index_map, compare, combine, std::numeric_limits<int>::min(), std::numeric_limits<int>::max(), boost::default_dijkstra_visitor()); }

2 3

boost::regex: regex meta data or AST for regex?
by Frank Bergemann 16 Dec '20

16 Dec '20

in concrete: Is it possible to not only get the #no of groups (sub-expressions) via basic_regex::mark_count(), but also their position (begin() / end() iterators) in the given regex string (c'tor argument)? more general: Can an AST be created for a the given regex string (c'tor argument)? E.g with boost::spirit?

1 0

how to get progress indicator during phrase_parse() of boost::spirit::x3?
by Lane 15 Dec '20

15 Dec '20

I am using boost::spirit:x3, shown below. I use it to import/parse some very large files and am trying to find a way to get some sort of progress from it so that I can use for a progress bar in app, but so far I've been unable to see a way. namespace parser { namespace x3 = boost::spirit::x3; using namespace x3; auto const string = lexeme[+graph]; auto const point = rule<struct _p, ast::point> {"point"} = int_ >> double_ >> double_ >> double_; auto const element = rule<struct _e, ast::element> {"element"} = int_ >> string >> int_ >> int_ >> int_ >> eol >> *(int_ >> int_ >> int_); auto lines = [](auto p) { return *(p >> eol); }; auto const input = lines(point) >> lines(element); } And I import a file like so. { namespace x3 = boost::spirit::x3; mapped_file_source map(filePath); std::istringstream iss(map.data()); map.close(); boost::spirit::istream_iterator iter(iss >> std::noskipws), eof; bool ok = phrase_parse(iter, eof, smf::parser::input, x3::blank, types); if (iter != eof) { // ... } } Would anyone have thoughts on how to get some sort of indicator during the parsing?

1 0

[Beast] Client connection times out after 5s. How to avoid?
by Dominique Devienne 15 Dec '20

15 Dec '20

Hi. Continuing my explorations of Boost.Beast. So now I have: * a small Boost.Beast HTTP server (based on async_http_server example), and * a small Boost.Beast HTTP client (based on sync_http_client example). This works, in that I can start the client (wrapped in a helper class), make several requests, and close the client. But until now, the client was short-lived, so no problem. (the server was started too of course, and long lived as expected). Now I'm embedding that HTTP client into another (WebSocket) server, which uses that same wrapper class above, successfully connects to the HTTP server on startup, and now waits for WebSocket connections, at which time it's supposed to make HTTP requests to the HTTP server for various reasons. Problem is, the HTTP client times out after 5s. While I want it to be long lived. I've read Vinnie's answer in https://stackoverflow.com/a/56880415/103724 about sync operation not having timeouts, but I don't want to control timeouts of individual requests here (for now at least), I want to not have the underlying socket not timeout on its own, the same way a WebSocket does not timeout either. Is that possible? Who controls that 5s timeout? Would switching to an async HTTP client make any difference? Sounds to me that it would allow me to control timeouts of requests, but would it make any difference to the socket timeout described above? Thanks, --DD

2 4

[Beast] Slow HTTP connect on Windows. Normal?
by Dominique Devienne 14 Dec '20

14 Dec '20

Hi. I'm writing a small interaction between: 1) a Boost.Beast-based HTTP server (based on http_server_async.cpp example) 2) a Boost.Beast-based HTTP client (based on http_client_sync.cpp example) When I log the time it takes to resolve the server address and connect to it, then issue my HTTP request, it takes over 1 second (Win10, VS2019, C++17, Release, localhost for both client and server): 2020-11-25T09:46:37.473936 Prologue in 0.001s 2020-11-25T09:46:37.482470 Resolved in 0.008s 2020-11-25T09:46:38.506180 Connected in 1.023s 2020-11-25T09:46:38.509607 OK: Authenticated; in 0.003s While the same on Linux (RH7.5) is just over 2ms: 2020-11-25T09:45:45.515926 Prologue in 0.000s 2020-11-25T09:45:45.517083 Resolved in 0.001s 2020-11-25T09:45:45.517550 Connected in 0.000s 2020-11-25T09:45:45.518010 OK: Authenticated; in 0.000s That's a huge difference! Almost 500x... And when I contact the same HTTP server on Windows, but from Chrome this time, it takes about 300ms, and if I hit reload rapidly, the time jumps around to as low as 3ms, and as high as 300ms (the same initial delay), with ~50ms and ~100ms in between. (I also see several different connections being established, for some reason...) I've seen similar differences between Windows and Linux connection times, but with WebSocketPP-based client and server this time, also based on Boost.ASIO. Q1: Am I doing anything wrong? I.e. is this "normal" somehow? Q2: How come Chrome, on Windows too, is 3x faster than ASIO-based clients, with the same Beast-based server? Given the high connect time on Windows, I thought I'd try to keep the connection open on the client, and issue several send-request/read-response pairs, using the Beast-based (sync) HTTP client, but only the first one works correctly, the 2nd errors out with: Error: An established connection was aborted by the software in your host machine I suspect it is a user-error, with the server closing the connection, despite the Keep-Alive HTTP header being present server-side on the request. Q3: Given the Beast HTTP examples I'm based off, would anyone have suggestions on what changes are necessary to allow the client issuing several (non-overlapping) requests to the server, on the same connection? (since connecting is so expensive). Q4: And for better performance, what about overlapping request/response pairs, with HTTP pipelining. How to set that up on the client and server with Beast? Thanks for any help on this. --DD PS: Great examples in Boost.Beast BTW. Thanks Vinnie.

4 9

boost.python and python3 64 bit
by Cruz, Petagay [USA] 14 Dec '20

14 Dec '20

Does Boost Python support using python 3 64 bit? if yes, how do i build say Boost.Python 1.74? What is the 'b2' command and then the bjam command for the tutorial example? Thank you

1 0