February 2011 - Boost - lists.preview.boost.org

Preventing Boost.Pool from allocating too much memory - need your help.
by John Maddock 16 Feb '11

16 Feb '11

There is a long standing bug against Boost.Pool that points out that it allocates far too much memory when the size of the requested block is oddly sized: https://svn.boost.org/trac/boost/ticket/1252 For example if you request blocks of size 1501 bytes then you actually get 6004 byte sized blocks because Pool uses LCM to calculate a size that can store both a pointer and the requested block. There's some dispute on the Trac ticket whether the correct solution is to just round up the block size to a multiple of the pointer size. I'd like to run my take on this past you all and check I have the right fix ;-) ! So here goes - if we round up to the nearest multiple of pointer-alignment-size then: 1) The actual alignment requirements of the block (which only the client of Boost.Pool knows) must be a factor of the block size (otherwise arrays of blocks would be incorrectly aligned). 2) If the requested block has an alignment requirement >= pointer type, then of necessity alignment_of<void*>::value will be a factor of the block size and no rounding occurs or is necessary. 3) If the requested block has an alignment requirement < pointer type, then we may round up the block size, but this will only *increase* the alignment of the returned blocks so is always safe. Note that (2) assumes that all alignments are powers of 2, if that isn't true, then all bets are off. But I'm assuming there are no real-world architectures that violate that assumption? Thanks in advance, John.

1 0

[Serialization] extra_detail namespace breaks ability to multiply export classes
by Number Cruncher 16 Feb '11

16 Feb '11

I've been using serialization in a client-server application where the client and server support different archive types, and the same "message" classes need to be exported for a different set of archives in the client and server cases. For example: *Client*: links to libNetwork. libNetwork supports export of Message class by, e.g. Message.hpp: struct Message { }; BOOST_CLASS_EXPORT_KEY(Message) network.cpp: #include <network_oarchive.hpp> #include <network_iarchive.hpp> #include <Message.hpp> BOOST_CLASS_EXPORT_IMPLEMENT(Message) template void Message::serialize<network_oarchive>(.......); template void Message::serialize<network_iarchive>(.......); On the server, I also need serialization support for Message over a new archive type which only the server knows about: *Server*: links to libNetwork AND libServer. libServer supports export of Message class by, e.g. server.cpp: #include <server_oarchive.hpp> #include <server_iarchive.hpp> #include <Message.hpp> BOOST_CLASS_EXPORT_IMPLEMENT(Message) template void Message::serialize<server_oarchive>(.......); template void Message::serialize<server_iarchive>(.......); For boost 1.43 and earlier, this was all fine; the client had sufficient serialization code for its archives and so did the server. With 1.45, boost/export.hpp dropped the anonymous namespace used for GUID instatiation so now the boost::archive::detail::extra_detail::init_guid<Message>::g symbol is multiply defined (first in libNetwork, then in libServer) and I can't link. If drop the BOOST_CLASS_EXPORT_IMPLEMENT from libServer, I get unregistered class exceptions on the server, because the guid_initializer in libNetwork never "knew" about my server_oarchive. What was wrong with the anonymous namespace usage? Any help appreciated, Simon

1 0

[Tokenizer]Usage and documentation
by Max 16 Feb '11

16 Feb '11

Hello, I'm using boost::tokenizer to do some simple parsing of data file in a format specified by the following rules: - One record of several fields in a single line - Adjacent data fields in a record separated by space char's(space or tab), with or without "," - String without space(s), with or without quotation marks - String with space(s), with quotation marks One example of a 4-field-per-record file is like: "string 2" 3 4 5 4.3 "String", 2, 3.04 4 3 AnyOtherText, 2, 3.04 4 3 I am using the following code to get a line at first, supposing 'input' has the contents of the data file: typedef boost::tokenizer<boost::char_separator<char> > tokenizer; boost::char_separator<char> sep("\n", " "); tokenizer tokens(input, sep); for(tokenizer::iterator beg=tokens.begin(); beg!=tokens.end(); ++beg) { } Then for each *beg, I parse each line with this typedef boost::tokenizer<boost::char_separator<char> > tokenizer; tokenizer tokens (*beg, boost::char_separator<char>(", ")); tokenizer::iterator it= tokens.begin(); But I cannot get the expected output. And, at the mean time, I found the doc of boost::tokenizer quite slim and not easy to find the information that I need. Does anybody else have the same feeling, or, is the fact that nobody is actually using it but turning to any other better lib? Thanks for any help. B/Rgds Max

4 12

Re: [boost] Boost.Dynamic Config
by Ivan Le Lann 16 Feb '11

16 Feb '11

----- "Dan Weber" <dan(a)marketsoup.com> a écrit : > Hi Mathias, > > I've found that property_tree and program_options are deficient in > solving this problem. > > For instance, property_tree requires full serialization back and > forth > for each time you write to the underlying data store, so it could not > guarantee that it has the most accurate data all the time. > Maybe property_tree can be extended to support this use case. It seems Boost.MultiIndex usage is isolated in implementation. Is there anything that prevents basic_ptree to use an adaptor to external data source ? Regards, Ivan.

2 1

Re: [boost] [Boost-interest] C++ library for runtime-concepts (type-erasure)
by Andrzej Krzemienski 16 Feb '11

16 Feb '11

> DEFINE_ERASURE( Calculator, (add)(sub), > virtual double add(double) = 0; > virtual double sub(double) = 0; > ) > I would like feedback on how such an implementation would be received by the community. Hi, I think this macro would not work for the following interface: DEFINE_ERASURE( Dict, (getPair), virtual pair<string, string> getPair() = 0; ) This is because the comma comma 'pair' will be interpreted as separatinh third from the fourth macro param. Regards, &rzej

2 1

Initial review of Boost Process
by Edward Diener 16 Feb '11

16 Feb '11

This is an initial review of Boost Process based just on reading the documentation, I hope to find some time to try out the library within the next few days. I am pretty knowledgable about child processes, more on Windows than Linux, but I understand well how starting another process works in general. I was able to generate the documentation from the jamfile in the doc directory of the library. The documentation updated the targets successfully but there were a few messages which need to be fixed: "xslt-xsltproc.windows bin\msvc-10.0\debug\threading-multi\autodoc-xml.doxygen xslt-xsltproc.windows bin\msvc-10.0\debug\threading-multi\autodoc-xml.boostbook common.copy autodoc.xml bin\msvc-10.0\debug\threading-multi\autodoc-xml.boostbook 1 file(s) copied. xslt-xsltproc.windows bin\msvc-10.0\debug\threading-multi\process.docbook Cannot find method named 'boost::process::detail::basic_status::async_wait' Cannot find method named 'boost::process::detail::basic_status::async_wait' Cannot find class named 'boost::system::system_error' Cannot find class named 'boost::system::system_error' Cannot find class named 'boost::filesystem::filesystem_error' Reference to class 'pipe' is ambiguous. Found: Match in namespace ::boost::process:: Match in namespace ::boost::process::behavior:: Reference to class 'pipe' is ambiguous. Found: Match in namespace ::boost::process:: Match in namespace ::boost::process::behavior:: xslt-xsltproc-dir.windows html\standalone_HTML.manifest Computing chunks... Writing standalone_HTML.manifest ...updated 13 targets..." In the "Configuring runtime contexts" topic I did not feel there is a clear explanation of some of the context members. What is the 'env' member ? The synopsis says it is of type 'environment' but there is no explanation what this means. Is the parent's environment inherited by the child process ? Are there any environment variables automatically inherited by the child process ? The context's streams member and its meaning is incomplete. What is the stream_type ? The various built-in functors, which are described as behaviors are also incomplete in their explanation. What these are, and what they do, needs to be explained. The synopsis doc for the stream_id says it is a "NativeStreamId" but it does not tell one what that is. Are streams other than standard input, standard output, or standard error supported ? In the "Communicating with Child Processes" documentation, it mentions the various pipe behaviors. An example is given of synchronous I/O using 'pipe'. But then the last line tells us that for asynchronous I/O one uses 'pipe'. This is confusing. In the "Asynchronous I/O" section it repeats again that one uses 'pipe' for ansynchronous I/O but then the example uses 'async_pipe' instead. Again this is very confusing. In the "Terminating a Process" section it does not explain how a process is terminated. I think that needs to be in there. In the "Creating new stream behaviors" section I am shown example code for redirecting a stream but there is no explanation of how this works. There needs to be. It would be nice if some mention were made about Boost.Interprocess and if there is any redundancy between Process and the facilities in the Interprocess library. Also any needed explanation of how one uses Process with the facilities in Boost.Interprocess would be helpful. I think the documentation is organized well but is too sparse in places. I will try to use Process and hopefully I can review it further. But I would not initially recommend it with documentation which is this incomplete. The design does seem both clean and focused on what creating, communicating, and controlling a child process is supposed to do, and I like it that the library seeks to do specific things and not try to overextend itself. But it clearly needs a fuller doc. Edward Diener

1 0

[string] Yet another Unicode string class
by Anders Dalvander 15 Feb '11

15 Feb '11

I'm working on yet another Unicode string class/library from another set of features and requirements. * It is designed around the codepoint concept. * It uses (currently forward-) iterators for encoding and decoding. * It has a minimal interface, mostly constructors and iterator access. * Most other functions can (hopefully) be free functions. * It uses basic_string as backend. * It has fast access to underlying basic_string. * It is (currently) using some C++0X features (mainly decltype). * It is (currently) immutable and shares data, and thus fast to copy. Some of these features and requirements may be unacceptable to some of you, but I'm open to suggestions and comments. // Copyright (c) 2011 Anders Dalvander. // // Distributed under the Boost Software License, Version 1.0. // (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) template <typename encoding> class basic_text { public: typedef encoding encoding_type; typedef typename encoding_type::codeunit_type codeunit_type; typedef typename encoding_type::codepoint_type codepoint_type; typedef std::basic_string<codeunit_type> string_type; typedef typename string_type::const_iterator codeunit_iterator; typedef typename encoding_type::decode_iterator<codeunit_iterator> codepoint_iterator; typedef codepoint_iterator const_iterator; typedef codepoint_iterator iterator; basic_text() : s(std::make_shared<string_type>()) { } template <typename other_encoding> basic_text(const basic_text<other_encoding>& text) : s(std::make_shared<string_type>( encoding_type::encode_iterator<decltype(std::begin(text))> (std::begin(text), std::begin(text), std::end(text)), encoding_type::encode_iterator<decltype(std::begin(text))> (std::end(text), std::begin(text), std::end(text)))) { } // TODO: Use some default_encoding traits type. template <typename container> explicit basic_text(const container& c) : s(std::make_shared<string_type>( encoding_type::encode_iterator<decltype(std::begin(c))> (std::begin(c), std::begin(c), std::end(c)), encoding_type::encode_iterator<decltype(std::begin(c))> (std::end(c), std::begin(c), std::end(c)))) { } template <typename codepoint_iterator> basic_text(codepoint_iterator first, codepoint_iterator last) : s(std::make_shared<string_type>( encoding_type::encode_iterator<codepoint_iterator> (first, first, last), encoding_type::encode_iterator<codepoint_iterator> (last, first, last))) { } codepoint_iterator begin() const { return codepoint_iterator (codeunit_begin(), codeunit_begin(), codeunit_end()); } codepoint_iterator end() const { return codepoint_iterator (codeunit_end(), codeunit_begin(), codeunit_end()); } codeunit_iterator codeunit_begin() const { return std::begin(*s); } codeunit_iterator codeunit_end() const { return std::end(*s); } const string_type& str() const { return *s; } const codeunit_type* c_str() const { return s->c_str(); } private: typedef std::shared_ptr<const string_type> pointer_type; pointer_type s; }; typedef undefined-type utf8_encoding; typedef basic_text<utf8_encoding> u8text; typedef undefined-type utf16_encoding; typedef basic_text<utf16_encoding> u16text; typedef undefined-type utf32_encoding; typedef basic_text<utf32_encoding> u32text; typedef undefined-type wchar_encoding; typedef basic_text<wchar_encoding> wtext; typedef undefined-type ascii_encoding; typedef basic_text<ascii_encoding> ascii_text; Usage: int main() { const uint32_t cps[] = {0x41,0x42,0x80,0x800,0x10000,0x10ffff}; // construct from codepoint range u8text u8txt(std::begin(cps), std::end(cps)); // construct from encoded container, // currently treats each element as a codepoint u8text u8txt2("test"); // sharing is caring u8text u8txt3 = u8txt; // construct from codepoint range u16text u16txt(std::begin(cps), std::end(cps)); // construct from text, transcodes range u16text u16txt2 = u8txt; // construct from text, transcodes range u32text u32txt = u8txt; // using policy (possible extension) ascii_text ascii(u8txt, replace_policy(0xff)); } void OpenFileWin32(const u8text& txt) { CloseHandle(CreateFileW(wtext(txt).c_str(), ...)) } typedef undefined-type posix_encoding; typedef basic_text<posix_encoding> posixtext; void OpenFilePosix(const u8text& txt) { close(open(posixtext(txt).c_str(), ...)) } Regards, Anders Dalvander -- WWFSMD?

10 14

Re: [boost] [Boost-users] Brainstorming [WAS: Subject: Formal Review of Proposed Boost.Process library starts tomorrow]
by Darren Garvey 15 Feb '11

15 Feb '11

On 11 February 2011 23:30, Boris Schaeling <boris(a)highscore.de> wrote: > On Fri, 11 Feb 2011 16:39:35 +0100, Jeff Flinn < > TriumphSprint2000(a)hotmail.com> wrote: > > [...]I think with the right abstractions the seemingly contradictory goals >> can be accommodated. That's what I'm attempting to find. As stated at a >> previous boostcon: "... abstractions are discovered not invented..." >> >> >> IMHO, previous version of process have not discovered the proper >> abstractions. I'm sure that the entirely different paradigms between windows >> and posix have contributed to the difficulties so far. It's difficult to be >> a master of the subject matter in both domains. >> >> I think building upon existing boost filesystem and iostreams libraries >> and their abstractions that do successfully address platform differences >> will make the effort more successful. Additionally I think it's mandatory >> for a design to directly acknowledge the multi-phase constraints of the >> fork/exec paradigm. Also it's mandatory to break down the historical >> monolithic approaches into more tenable pieces. >> > > Thanks for your comments, Jeff, Ilya and Nat! How would you (and others) > feel about a Boost.Process library which only aims to replace std::system() > with a more powerful function? You could configure the child process a bit > (its streams, environment variables and a few more things) and that's all. > This would be a less ambitious goal but obviously more easily to reach. I > was under the impression that we want more? Maybe we still want more but > settle realistically for this goal? I've not caught up on all of the discussion about the pending review yet, but it seems to be hitting a few familiar issues this time around, unfortunately. I'd like Boost.Process to either mimic the Python subprocess module (as Nat mentioned) or to provide a few low-level pieces: 1. std::system replacement allowing me to do the call in an async way and capture the output. Ideally implemented in such a way that a "process pipeline" library could be implemented on top of Boost.Process. Pipelining input / output between different work items is a more general domain than fits in Boost.Process, IMHO. 2. Way to get process id in a platform independent way. 3. Way to look up location of processes and get a Boost.Filesystem path. 4. Version of "std::system" that works with Boost.Filesystem 5. Version of std::system that allows passing a custom set of environment variables 6. Expose a "native_handle_type" from the library, whatever that native handle type is. There seems to be a lot of justified disagreement about what the high-level view of the library should be, yet all but point 1 are almost trivially implemented in a cross-platform way and entirely orthogonal to the rest of any Boost.Process library. I'd like to see just points 2-6 addressed, reviewed and released before tackling the bigger picture. The disagreements over scope are a showstopper for any progress and threaten to lead Boost.Process astray for another another couple of years unless they are worked around, ie. by reducing scope, perhaps to the point that Boost.Process starts life as a basic utility library. Evolutionary rewrites and breaking changes are much better than nothing. After all, we're up to V2.x of Boost.Spirit and V3 of Boost.Filesystem and all versions have been immensely useful for us end users. Kind Regards, Darren

5 24

Implicit Convert from own Typ to Boost::Variant Type
by a.mueller 15 Feb '11

15 Feb '11

Hi, i'm new here and i hope you can help me. My problem i have a class with a Boost Variant type as member and a implicit convert operator from my class to this Variant type. But when i want use the conversion in my programm the Compiler give me a error. Compiler Version gcc 4.4.1 on Ubuntu system Boost version 1.45 Example Program: #include <iostream> #include <boost/variant.hpp> #include <boost/lexical_cast.hpp> using namespace std; 　 typedef boost::variant<float,int> Testtyp; class Testtypcontainer { public: Testtypcontainer():m_testtyp((float) 5.5){} Testtyp m_testtyp; template<class T> void settesttyp(T testvalue){m_testtyp =testvalue;} Testtyp gettesttyp(){return m_testtyp;} operator Testtyp() const{ return m_testtyp; } }; int main(){ Testtyp help1; Testtypcontainer help3; cout<<help3.gettesttyp()<<endl; help1=(Testtyp)help3; // generate compile error return 0; } P.s. Sorry for my bad english

2 2

Boost.Dynamic Config
by Dan Weber 15 Feb '11

15 Feb '11

Hi guys, I feel as part of the Boost community and having worked on multiple platforms that we're missing something to manage reading/writing configuration information for an application thread safely across platforms. The best example I know of that handles this job is Qt's QSettings which writes to the registry on windows, inis on linux, and some sort of xml on Mac OS X. The inspiration for this comes from the fact that I am looking to build web browser plugins that run on multiple platforms. I started by looking at how I could extend boost::program_options, and decided it wasn't the right tool because it was primarily for reading configurations, but not really writing to them. Likewise, it doesn't appear to provide anything in terms of thread safety. The interface should model after several things already in existence such as: QSettings, std::map, and tbb::concurrent_hash_map. It should also take into consideration how systems such as the Windows Registry works, as well as browser specific configuration systems like the one that comes with Mozilla Firefox, and of course file based storage. It should be easily extensible with a simple interface to implement a new backend with minimal work. At a later date, I think it should also be able to support things like boost::property_map or boost::fusion to make it easy to insert and extract objects from the underlying configuration system with automatic type conversion. I'm not sure how to combine it with boost::fusion or boost::property_map yet, so I'm hoping that's where you guys come in. Thanks, Dan Here is a preliminary interface: namespace boost { namespace dynamic_config { template<typename charT> class backend { public: enum operation_performed { none, // e.g. no value changed insert, update }; bool insert(basic_string<charT> key, (int, basic_string<charT>, bool, int64_t, char) value); // These values may be easily implementable with boost::variant, or separately if enable_if is used. bool update(basic_string<charT> key, (see above..) value); // Note this returns operation performed because optional locking will be done around these functions, so combining insert with update replaces a possible race condition over those 2 operations performed separately operation_performed replace(basic_string<charT> key, (...) value); template <T> bool select(basic_string<charT> key, T& value); // returns success if retrieved, this can also use boost::optional otherwise bool delete(basic_string<charT> key, bool cascade); // cascade takes effect if key takes a whole tree with it }; template <typename charT, typename Backend, typename Mutex> class property_config : boost::noncopyable { public: property_config(); property_config(basic_string<charT> organizationName, basic_string<charT> applicationName); // Pretty much required for use in the windows registry and in firefox ~property_config(); size_type erase(basic_string<charT> key); void clear(); template<typename T> bool find(basic_string<charT> key, T & value); template<typename T> bool insert(basic_string<charT> key, T value); template<typename T> bool update(basic_string<charT> key, T value); template<typename T> backend::operation_performed replace(basic_string<charT> key, T value); template<typename T> bool select(basic_string<charT> key, T& value); template<typename T> bool delete(basic_string<charT> key, T& value); }; template <typename charT, typename Backend, typename Mutex> template <typename T> backend::operation_performed property_config<charT, Backend, Mutex>::replace(basic_string<charT> key, T val) { scoped_lock(mutex_); backend_.replace(key,val); } } }

4 6