September 2011 - Boost - lists.preview.boost.org

[chrono][stopwatches] Request for interest in NEW stopwatch/timer reporting facilities
by Vicente J. Botet Escriba 16 Sep '11

16 Sep '11

Hi, I has been reconsidering the reporting facilities of proposed Boost.Stopwatches. I have started to make some examples that make use Boost.Format to make the reports. Default reporting stopclock<> sw; prints the duration 29308.350 µs You can make a boost::format and pass it to the stopclock. void f1(long j) { boost::format fmt("%1%[%2%] f1(%3%) - Elapsed time: %4%"); fmt % __FILE__ % __LINE__ % j; stopclock< > sw(fmt); // burn some time } ../example/scoped_stopwatch_example.cpp[42] f1(1000000) - Elapsed time: 29834.769 µs ../example/scoped_stopwatch_example.cpp[51] void f2(long int) - Elapsed time: 0.979 µs You can change the precision, the output stream, the units and the unit text using a specific formatter. For example typedef elapsed_formatter<milli > formatter; formatter fmt("Elapsed time: %1%", std::cerr); fmt.set_duration_style(text); fmt.set_precision(6); stopclock<high_resolution_clock, formatter> sw(fmt); could print Elapsed time: 87.829801 milliseconds This formatter could be shared between several stopclocks (thread unsafe however). Even if the reporting facilities will not be completly 'localizable', I think that this interface goes one step towards in this direction. Before continuing the adaptation of the whole library I would like to hear some comments from you. Do you think this is an acceptable interface? What are the parameters that do you want to manage? procession, unit, short/long units, ... Best, Vicente

2 4

[iostreams] #5904: Patch to suppress "unreachable code" warning from Visual Studio 2008/2010 (VC90/VC100)
by Sylvester-Bradley, Gareth 16 Sep '11

16 Sep '11

I've added ticket #5904 with a patch to suppress a warning in Boost.Iostreams. Please would someone apply this for Boost 1.48.0? Ticket URL: <https://svn.boost.org/trac/boost/ticket/5904> Many thanks, Gareth ************************************************************************ The information contained in this message or any of its attachments may be confidential and is intended for the exclusive use of the addressee(s). Any disclosure, reproduction, distribution or other dissemination or use of this communication is strictly prohibited without the express permission of the sender. The views expressed in this email are those of the individual and not necessarily those of Sony or Sony affiliated companies. Sony email is for business use only. This email and any response may be monitored by Sony to be in compliance with Sony's global policies and standards

1 0

utility-container_gen in the sandbox
by Cromwell Enage 16 Sep '11

16 Sep '11

Hi, all. After a bit of a programming hiatus, I've managed to follow through on the container_gen discussion we had last May. I've created a branch in the sandbox called utility-container_gen that places the metafunction in Boost.Utility, changes detail::is_random_access to is_random_access_selector, provides an is_unique_associative_selector metafunction, and changes adjacency_list and parallel_edge_traits to use said utilities. Plus, there's Quickbook documentation. I've tested a few example programs in the BGL to make sure they still work, but I feel that's far from being thorough. No selectors for Boost.Intrusive have been implemented, yet. Feedback welcome! Cromwell D. Enage

1 0

[Review] Boost.Endian by BEman Dawes starts today
by Joel Falcou 15 Sep '11

15 Sep '11

According to the schedule, the review of the Boost.Endian library by Beman Dawes starts today. =========== What is it? =========== Boost.Endian provides facilities to manipulate the byte ordering of integers. * The primary use case is binary I/O of integers for portable exchange with other systems, via either file or network transmission. * A secondary use case is to minimizing storage size via integers of sizes and/or alignments not supported by the built-in types. Integers 1, 2, 3, 4, 5, 6, 7, and 8 bytes in length are supported. * Two distinct approaches to byte ordering are provided. Each approach has a long history of successful use, and each approach has use cases where it is superior to the other approach. * The explicit approach provides explicit functions to reorder bytes. All four combinations of non-modifying or modifying, and unconditional or conditional, functions are provided. * The implicit approach provides integer classes that mimic the built-in integers, implicitly handling all byte reordering. =================== Getting the Library =================== Docs are available at http://boost.cowic.de/rc/endian/doc/index.html A zip file is available at http://boost.cowic.de/rc/endian/endian-rc1.zip INSTALL instructions at http://boost.cowic.de/rc/endian/INSTALL Alternately, the whole library can be checked out of the sandbox: svn co http://svn.boost.org/svn/boost/sandbox/endian endian ================ Writing a Review ================ The reviews and all comments should be submitted to the developers list, and the email should have "[Endian] Review" at the beginning of the subject line to make sure it's not missed. Please explicitly state in your review whether the library should be accepted. The general review checklist: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With what compiler? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? And finally, every review should answer this question: - Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion.

4 6

[container] Issues with non-copyable mapped-types
by onlyone＠senet.com.au 15 Sep '11

15 Sep '11

Hi, I have been trying to use Boost.Container to make a map<copyable_but_not_moveable, moveable_but_not_copyable>. Unfortunately I have been unable to do this because the value_type of boost::container::map is std::pair, which does not support move semantics (in C++03). This causes problems for the moveable_but_not_copyable type. It means that, for example: myMap[copyable_but_not_moveable()]; does not compile. This behaviour is fully supported in C++11 mode, and I see no reason why a custom C++03 pair class would be unable to use the Boost.Move emulation to achieve the same effect. Regards, Evan Wallace

3 3

[boost/filesystem] Access to a shared folder on other computer
by Demetrio Car 15 Sep '11

15 Sep '11

Can someone send me a example of how to copy a file from a remote folder to other remote folder? For example: I am running MyProgram on MyComputer and this program has to copy a file from a folder inside OtherComputer. This folder need user and password to allow MyProgram to copy the file. I am using the following sample. The main point in this example is to make my program to work both in linux and windows, but locally. Now I want to do the same with a path like this: \\192.168.x.x\mySharedFolder\fileToBeCopied.text . Maybe I need another library but I don't know. //***http://www.ibm.com/developerworks/aix/library/au-boostfs/ #include <stdio.h> #include <boost/filesystem.hpp> int main() { boost::filesystem::path path("/usr/local/include"); bool result = boost::filesystem::is_directory(path); return 0; }

1 0

[Review] Boost.Endian Review
by joel falcou 15 Sep '11

15 Sep '11

A gentle reminder that the Boost.Endian review is scheduled to end today. Any participant that wish to submit review can do so before tonight Regards

1 0

[signals2] Problem compiling with Intel V12.1 (Windows)
by Hartmut Kaiser 15 Sep '11

15 Sep '11

Compiling code using Boost.Signals2 fails using Intel V12.1 on Windows. The reason is a forward declaration of std::weak_ptr and std::shared_ptr in the signals2 code which creates a problem as Intel seems to import std::tr1::weak_ptr into namespace std. This patch fixes the issue: --- forei-revBASE.svn002.tmp.hpp Wed Oct 07 15:56:56 2009 +++ boost/boost/signals2/detail/foreign_ptr.hpp Thu Sep 15 06:10:24 2011 @@ -12,16 +12,19 @@ #define BOOST_SIGNALS2_FOREIGN_PTR_HPP #include <algorithm> +#include <boost/config.hpp> #include <boost/assert.hpp> #include <boost/scoped_ptr.hpp> #include <boost/smart_ptr/bad_weak_ptr.hpp> #include <boost/utility/swap.hpp> +#if !defined(BOOST_INTEL_STDCXX0X) namespace std { template<typename T> class shared_ptr; template<typename T> class weak_ptr; } +#endif namespace boost { Ok to commit? Regards Hartmut --------------- http://boost-spirit.com

1 0

[phoenix] compile-time performance
by Eric Niebler 15 Sep '11

15 Sep '11

On 9/12/2011 4:06 AM, Thomas Heller wrote: > On Monday, September 12, 2011 02:57:21 PM Joel de Guzman wrote: >> For example, here is the current CT status of Phoenix2 vs Phoenix3 >> comparing the elapsed (CT) time for the phoenix2 vs. phoenix3 >> lambda_tests.cpp (**): >> >> MSVC 10: >> Phoenix2: 00:04.5 >> Phoenix3: 00:29.9 >> >> G++ 4.5: >> Phoenix2: 00:02.6 >> Phoenix3: 00:04.7 > > I wasn't aware that Phoenix3 was so bad under MSVC 10. Me neither. If that's the case, we have work to do. >> You all know that Phoenix2 uses Fusion exclusively. Phoenix3 uses >> proto, which according to Eric does not use Fusion, although IIRC >> the core of Phoenix3 uses some Fusion still (quick check: Thomas >> uses an optimized-PP version of fusion:: vector for phoenix3). >> >> Heller did a helluva perf-tweaks for Phx3 to get that number for >> g++ (alas, not MSVC). In fairness, I did absolutely no CT >> perf-tweaks for both Phoenix2 and Fusion. >> >> (** I made sure both tests have exactly the same code, so I removed >> the last test. I can post the exact code if need be) OK, I retract everything I said. Fusion rulez, and Proto needs help. Sorry for the blind accusation. Let's stop this and figure out what needs to be fixed. > FWIW, there are some unit tests that outperform the compile times of > Phoenix2 (with gcc), the current bad hit on compile times seem to > only occur with let, lambda and switch/case expressions. This /suggests/ to me that the core of Phx3 is sound, but that the implementation of let, lambda, and switch are heavy. But I'm not blaming Thomas. I'm just suggesting a logical place to begin an investigation. And regardless, we should point Steven's template profiler at the code and see what it says. Thomas, do you want to give that a shot? -- Eric Niebler BoostPro Computing http://www.boostpro.com

5 9

[concept] EqualityComparable metafunction
by Lorenzo Caminiti 14 Sep '11

14 Sep '11

Hello all, What is the best way to check if a type T is EqualityComparable? Essentially, I would need a is_equality_comparable<> metafunction: struct mytype {}; is_equality_comparable<int>::value // true is_equality_comparable<mytype>::value // false Can I re-use the boost::EqualityComparable concept to implement is_equality_comparable<>? Or, shall I just try to use introspection-SFINAE techniques? Thanks in advance. --Lorenzo

7 11

intrusive_ptr to a map containing intrusive_ptr object
by shri kumar 14 Sep '11

14 Sep '11

Hi, I am facing a problem while making the raw pointer to an intrusive_ptr inside a map whose pointer is already an intrusive pointer so typedef goes like: typedef boost::intrusive_ptr< protocolLibraryLoader> ProtocolLibraryLoaderPtr; typedef std::pair<ProtocolLibraryLoaderPtr, ProtocolRecognizer_type*> ProtocolLibEnginePair; typedef boost::intrusive_ptr<ProtocolLibEnginePair> ProtocolLibEnginePairPtr; Now inside code when I try creating map and assign with ptr as given in below: .. *boost::intrusive_ptr<protocolLibraryLoader> libraryLoader = getProtoLoader();* .. m_activeRecEnginePtr = ProtocolLibEnginePairPtr(libraryLoader,newRecognizer); .. I get the following error: ProtocolManager.cpp: In member function 'void ProtocolManager::InitializeProtocolsLibrary(ProtocolManager*)': ProtocolManager.cpp:234: error: no matching function for call to 'boost::intrusive_ptr<std::pair<boost::intrusive_ptr<protocolLibraryLoader>, ProtocolRecognizer_type*> >::intrusive_ptr(boost::intrusive_ptr<protocolLibraryLoader>&, ProtocolRecognizer_type*&)' ../../../include/boost/intrusive_ptr.hpp:94: note: candidates are: boost::intrusive_ptr<T>::intrusive_ptr(const boost::intrusive_ptr<T>&) [with T = std::pair<boost::intrusive_ptr<protocolLibraryLoader>, ProtocolRecognizer_type*>] ../../../include/boost/intrusive_ptr.hpp:70: note: boost::intrusive_ptr<T>::intrusive_ptr(T*, bool) [with T = std::pair<boost::intrusive_ptr<protocolLibraryLoader>, ProtocolRecognizer_type*>] ../../../include/boost/intrusive_ptr.hpp:66: note: boost::intrusive_ptr<T>::intrusive_ptr() [with T = std::pair<boost::intrusive_ptr<protocolLibraryLoader>, ProtocolRecognizer_type*>] ../../../include/boost/intrusive_ptr.hpp: In constructor 'boost::intrusive_ptr<T>::intrusive_ptr(T*, bool) [with T = protocolLibraryLoader]': ProtocolManager.cpp:225: instantiated from here ../../../include/boost/intrusive_ptr.hpp:72: error: cannot convert 'protocolLibraryLoader*' to 'Tracker*' for argument '1' to 'void intrusive_ptr_add_ref(Tracker*)' ../../../include/boost/intrusive_ptr.hpp: In destructor 'boost::intrusive_ptr<T>::~intrusive_ptr() [with T = protocolLibraryLoader]': ProtocolManager.cpp:225: instantiated from here ../../../include/boost/intrusive_ptr.hpp:101: error: cannot convert 'protocolLibraryLoader*' to 'Tracker*' for argument '1' to 'void intrusive_ptr_release(Tracker*)' ../../../include/boost/intrusive_ptr.hpp: In copy constructor 'boost::intrusive_ptr<T>::intrusive_ptr(const boost::intrusive_ptr<T>&) [with T = protocolLibraryLoader]': ProtocolManager.cpp:225: instantiated from here ../../../include/boost/intrusive_ptr.hpp:96: error: cannot convert 'protocolLibraryLoader*' to 'Tracker*' for argument '1' to 'void intrusive_ptr_add_ref(Tracker*)' make: *** [FREEBSD_6_i686/ProtocolManager.o] Error 1 Basically worrying part is "no matching function for call to 'boost::intrusive_ptr<std::pair<boost::intrusive_ptr<protocolLibraryLoader>, ProtocolRecognizer_type*> >::intrusive_ptr(boost::intrusive_ptr<protocolLibraryLoader>&, ProtocolRecognizer_type*&)' I wonder if it is kind of limitation or do I have to do something else to achieve? Actually I am dealing with legacy code so I don't want to remove the intrusiveness on the pair which actually increases/decreases ref_count of ProtocolRecognizer_type and not on protocolLibraryLoader. Please help me as I am kind of stuck with this. Thanks Shri

2 1

[endian] Review
by Stewart, Robert 14 Sep '11

14 Sep '11

Herewith is my review of Beman's Boost.Endian library. ____________________ ____________________ What is your evaluation of the design? _____ The two-argument overloads of the [un]conditional swapping functions seem awkward at first blush, but are really the best interface for those operations. Changing to value returning functions opens the door to mistakes like the following: uint32_t const in(123); int32_t const out(reorder(in)); There might be a warning, but many programmers routinely ignore such. As is, the compiler will complain about the conditional functions: uint32_t const in(123); int32_t out; native_to_big(in, out); // Error: ambiguous The unconditional functions do not provide that benefit because they aren't function templates: uint32_t const in(123); int32_t out; reorder(in, out); _____ An interesting variation that I've used in the past is a set of cast-style function templates that permit byte swapping one type and returning the result as a different type, after verifying that the resulting value is in range of the target type. Thus, uint32_t const in(std::numeric_limits<uint32_t>::max()); int32_t const out(to_host<int32_t>(in)); // throws TMP allows streamlining the range checks done, of course. _____ Another variation I've used is a checked, two-argument function template with each potentially having a different type: template <class T, class U> void to_host(T & _result, U _value); (I always prefer output parameters to be first because their position is consistent and it leaves open the possibility of defaulted parameters in the general case. I'd prefer that in Boost.Endian, but not everyone likes that convention.) There are two possibilities for dealing with the case when T and U are different. One is that they be required to have the same, supported size and that they are both signed or both unsigned. The other is to do a runtime check that the swapped value is within the range of the target type. _____ I like that the aligned integer types have the longer names because they incur padding and are non-portable. _____ The overhead implied by the comments in the "Comment on naming" section of the Endian Integer Types page is an excellent reason for splitting the endian class template into endian_buffer and endian_integer as previously suggested. For C++11, the latter can derive from the former. For C++03, if POD-ness is important, you'll need to duplicate the former's code in the latter, but it's worthwhile. ____________________ ____________________ What is your evaluation of the implementation? There's already been a good deal of discussion on efficiency and reuse during the review, so I won't rehash it. The 64b reorder(), for example, looks like a lot of code for an inline function. I haven't compiled it to inspect the resulting assembly to see how much or little results. My concern is for the possibility of code bloat arising from the desire to keep this a header-only library. If those were function templates, then the compiler can choose whether to inline or not, despite the definition being in a header. The implementation of class endian is uncommented, yet non-trivial. A future maintainer, even Beman, will benefit from some helpful information inserted now. The implementation of class endian would be simplified, it appears to me, by passing the endianness along to a detail::store() function template and letting it select the big/little endian store logic. ____________________ ____________________ What is your evaluation of the documentation? It's incomplete. There are no examples of the conversion functions. There is no tutorial. There is no guidance on selecting between the alternatives. There is no comparison with traditional htnol()-style code. __________ Endian Home _____ Introduction to endianness s/modern Apple, Linux, or Windows computer/computer/ (the OS doesn't matter) s|a[sic] Oracle/Sun Solaris computer| (the OS doesn't matter) s/CPU's/CPUs/ (plural, not possessive) _____ Introduction to the Boost.Endian library The first and last sentences should be merged: "Boost.Endian is a header only library providing facilities for managing integer endianness." Endian conversions for native integers: The last sentence makes both absolute and relative statements. The relative statements are, presumably, relative to endian integer types, but that's not made clear. I suggest rewording as, "This approach is simple and efficient, but when compared to endian integer types, is less flexible regarding size and alignment, can be harder to manage, is more error prone because of the need to apply endian conversions in unrelated logic paths, all of which can lead to logic errors that are difficult to debug." Endian integer types: The second sentence has a similar problem, and the last sentence could be combined to correspond with the list of the previous paragraph. Try this variation: "This approach is simple and, while it can be less efficient than endian conversions, it avoids strict alignment requirements allowing opportunities to pack data more tightly." I'd follow with this variation of the third sentence: "Furthermore, types of length from 1 to 8 bytes are supported, rather than just the usual 1, 2, 4, and 8 byte lengths." __________ Conversion Functions This page is odd. The synopsis does not include links to the description of the functions. When I first saw the page, I thought there simply was no documentation for using the various functions. That documentation does appear in the latter part of the page, but it isn't obvious. There should be information explaining why conditional and unconditional functions are useful and each should refer to its counterpart. That is, reorder(uint16_t &) should note that if the intent is not to reorder the bytes regardless of the platform, that bit_to_native(uint16_t &), for example, is appropriate. The unconditional nature or reorder() escaped me until I realized that these functions always swap bytes, unlike the htonl() family, which I'd never before encountered as necessary. Thus, use cases for unconditional byte swapping are warranted. __________ Endian Integer Types The Conversion Functions page links to the description of big and little endian on the Endian Home page, but the Integer Types page does not. The example shown on the Integer Types page should be recreated using the Conversion Functions to show the difference in usage. That example should use sizeof(h) rather than sizeof(header) since that is a safer practice. _____ Limitations This section, while important, should not appear so soon on the page. s/compilers do layout memory/compilers lay out memory/ s/C++0x/C++11/ s/it will be possible to specify the default constructor as trivial/the default constructor is marked as trivial/ s/will no longer disqualify/do not disqualify/ s/will no longer be relying/does not rely/ _____ Feature set s/Big endian | little endian | native endian byte ordering./Big, little, and native endian byte ordering/ s/Signed | unsigned/Signed and unsigned/ s/Unaligned | aligned/Data may be aligned or unaligned/ s/1-8 byte (unaligned) | 2, 4, 8 byte (aligned)/2, 4, and 8 byte aligned types and 1-8 byte unaligned types/ _____ Typedefs The column order should correspond with the order of information in the typedef names: alignment, sign, endianness, size. The discussion of the difference between aligned and unaligned types should not be "buried" in the Typedefs section. _____ Comment on naming s/these type grows/these types grows/ Why does "the realization creep in" that the endian integer types are lousy arithmetic integers? Only profiling would reveal that. Consequently, you should call that out, perhaps in a "Design Details" section along with the aligned/unaligned discussion, thereby explicitly stating rather than leaving to each library user the discovery of the overhead of these types. _____ Synopsis The link from "cover_operators" to a header is surprising. _____ Members The description of the default constructor fails to indicate whether the value is left uninitialized, which I presume to be the case. The description of the conversion operator mixes T and value_type. While those two are the same, one has to examine the class definition to see that. Either s/T/value_type/ or s/value_type/T/. _____ FAQ s/and both speed and/plus both speed and/ s/disk records/records/ (applies to networking, too) s/Yes, for sure, compared/There is overhead compared/ s/are usually be faster/are usually faster/ s/types POD's?/types POD?/ s/endian integer types not being POD's/of endian types not being POD/ s/data files are portable/data files that are portable/ The "These types are really just byte-holders" entry should not be a FAQ. That information, along with a discussion of the overhead of the simple syntax, should be part of the "Design Details" section suggested above. _____ Design considerations for Boost.Endian integers Only "Design" is underlined in the section heading. s/memcpyable/memcpy-able/ _____ Experience s/used very successful/used very successfully/ _____ Compilation s/This is ensures that , and/This ensures that class endian is POD, even in C++03, and/ s/POD's/POD/ ____________________ ____________________ What is your evaluation of the potential usefulness of the library? Highly useful, but could be more so. ____________________ ____________________ Did you try to use the library? No. Sorry. ____________________ ____________________ How much effort did you put into your evaluation? I spent about an hour reading the documentation plus a lot of time on the mailing list participating in endian discussions. ____________________ ____________________ Are you knowledgeable about the problem domain? Yes. I've used the old standby functions, htonl(), ntohl(), and friends over many years. I've created template functions to automatically select the right function based upon the size of the type being swapped, etc. ____________________ ____________________ Do you think the library should be accepted as a Boost library? There are issues that keep me from saying yes at this time. There are too many suggested variations and ideas under consideration to accept the library in its present state. However, a mini-review should be sufficient to evaluate the final form, once Beman determines a course of action, and determine whether to accept it or not. _____ Rob Stewart robert.stewart(a)sig.com Software Engineer using std::disclaimer; Dev Tools & Components Susquehanna International Group, LLP http://www.sig.com ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

2 1

[proto] Looong compile times and other issues
by John Maddock 14 Sep '11

14 Sep '11

So.... I thought I had all the expression template stuff pretty much done, and then I tried some real world use cases (compiling all the students_t distribution functions with my extended-precision FP type) and everything fell apart: * VC10 wouldn't compile the code at all - more or less runs the system out of swap space (takes about 10 minutes or more!), then exits with an internal compiler error (if I break the code down into it's parts and instantiate each part separately it does compile though - that's not a solution though!). * GCC-4.4.x fails to compile the code due to clashes between boost::math::complement (a function) and boost::proto::complement (a class). I suspect this is an old gcc bug (finding structures via ADL) - I guess the solution is to not derive my number type from a proto-type so ADL can't find proto:: classes? Or will I hit this from some other unforeseen lookup? * GCC-4.5.0 Fails with an internal compiler error :-( * GCC-4.6.0 Builds the code OK, but takes a long time - though possibly just barely acceptable. I suspect this is a "triple template" problem: * proto is complex template library. * my number class is a fairly large/complex template in it's own right. * the code above gets instantiated from deep within Boost.Math's internals. Other than using proto::switch in the grammar (which I'm doing already), is there anything I can do to reduce the template load on the proto side of things? Otherwise since I'm only using a tiny fraction of proto's capabilities, the only other option I can see is to rip it out and replace with a mini-proto designed to minimalize template instantiations within this particular use case. Thanks in advance, John.

7 26

[C++11 Feature List]
by Sid Sacek 13 Sep '11

13 Sep '11

Does boost have a spec (document) where it mentions which new language features in C++11 are permitted to be used in new libraries? -Sid Sacek

2 2

[Endian] Performance
by Phil Endecott 13 Sep '11

13 Sep '11

Dear All, I've just done some quick benchmarks of Beman's proposed byte-swapping code using the following test program: #include <cstdint> #include <iostream> #include <arpa/inet.h> // for htonl() using std::uint32_t; static inline uint32_t byteswap(uint32_t src) { #if defined(USE_GCC_BUILTIN) return __builtin_bswap32(src); #elif defined(USE_HTONL) return htonl(src); #elif defined(USE_REV_INSTR) uint32_t target; __asm__ ( "rev %0, %1\n" : "=r" (target) : "r" (src) ); return target; #elif defined(USE_BEMAN) const char* s (reinterpret_cast<const char*>(&src)); uint32_t target; char * t (reinterpret_cast<char*>(&target) + sizeof(target) - 1); *t = *s; *--t = *++s; *--t = *++s; *--t = *++s; return target; #else #error "Define USE_*" #endif } int main() { uint32_t s = 0; for (uint32_t i = 0; i < 1000000000; ++i) { s += byteswap(i); } std::cout << s << "\n"; // Expect 506498560 } I've tested this on two systems: (A): Marvell ARMv5TE ("Feroceon") @ 1.2 GHz, g++ 4.4 (B): Freescale ARMv7 (i.MX53, Coretex A8) @ 1.0 GHz, g++ 4.6.1 Compiled with: --std=c++0x -O4 Execution times in seconds are: A B USE_GCC_BUILTIN 29.4 3.0 USE_HTONL 12.6 3.0 USE_REV_INSTR n/a 3.0 USE_BEMAN 17.6 8.0 ARM architecture version 6 (IIRC) introduced a "rev" byte-swapping instruction. On system B, which supports this, it is used by the gcc builtin and glibc's htonl() giving better performance than Beman's code. On system A which does not support this instruction, the gcc builtin makes a library call; I've not looked at what htonl() does. What do people see on other platforms? How important do we consider performance to be for this library? Regards, Phil.

11 37

[stopwatches] About reducing the scope of the library
by Vicente J. Botet Escriba 13 Sep '11

13 Sep '11

Hi, I'm not completly satisfied with the reporting interface/design I did for Boost.Stopwatches. I need to think more about the design so it can provide Locale facilities to the user. Do you think that it is worth proposing an addition to Boost.Chrono that provides only stopwatches (without reporting)? Best, Vicente

4 9

Re: [boost] [Submission] RawMemory (24-bit integers and runtime-dispatch)
by Adder 13 Sep '11

13 Sep '11

Hi there ! And thank you for your time and for your questions. Before I answer the first question regarding 24-bit integers, etc., please allow me to kindly ask you: Where are you going to store the 24-bit integer that you read from "raw memory" (= untyped memory = array of bytes) ? Or where is the 24-bit integer that you plan to write to "raw memory" stored ? (In what type of variable ?) Regarding the second question, regarding run-time dispatching of operations (as opposed to compile-time dispatching), I will start with a couple of examples that supports your question even more: It is not just the endian-ness that might not be known at compile-time. The alignment of data might also not be known before explicitly checking it at run-time. Let me quote from the ugly documentation that I have written (http://adder.iworks.ro/Boost/RawMemory/#UsersManual) "When we are dealing with an array of values (i.e. contiguous values) and performance is critical, we might want to check whether the data is aligned and specifically demand unchecked_memory_xfer if it is." Ultimately, the number of bytes that are used to represent the integer may be variable. Such is the case with the "mapping pair offsets" used by NTFS to locate the contiguous runs of clusters that make up the (non-MFT resident portion of the) file. In order to save precious MFT (Master File Table) entry space, signed offsets are stored in variable-sized (1 to 8-byte long) integers. My answer is probably obvious by now. I am thinking about letting the programmer use the ultimate weapon she has in her toolchest: The mighty "if" instruction. For the specific example you kindly gave, I am thinking that the following syntax might be useful for reading UCS-2 characters from a file saved with Notepad (in "Unicode" or "Unicode Big Endian" format): const uint16_t nByteOrderMask = raw_memory <uint16_t, little_endian>::peek (pFileBuffer); const bool bBigEndian = nByteOrderMask == 0xFEFF; ... // The loop: { const uint16_t cCurrent = bBigEndian ? raw_memory <uint16_t, big_endian>::peek (pCurrent) : raw_memory <uint16_t, little_endian>::peek (pCurrent); ... } I await your thoughts... Thank you again ! (-: -- Yours truly, Adder On 9/7/11, Joe Mucchiello <jmucchiello(a)yahoo.com> wrote: > For a library designed to deal with the weirdness of binary file formats, > I'm surprised it does not support 24-bit integers of arbitrary > alignment/sign. > >> uint32_t x = raw_memory <uint32_t, big_endian>::peek (&vbBuffer [5]); > > Do you also support a non-template "endianness" parameter? Sometimes you > don't know the endian-style of the file until you open it and read some > value out of it. > > Joe > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost >

2 5

[Endian] Review
by Tomas Puverle 12 Sep '11

12 Sep '11

Hi Beman, > - What is your evaluation of the design? The design of the integer types seems reasonable to me, even though I would prefer not to have the overloaded operators on them. I feel that having the operators has the potential to break the boundary between program input and the program's processing. What do you I mean by that? Typically, I try to structure my programs as follows: 1) Read input data 2) Verify input invariants, construct internal representation 3) Process internal representation 4) Output result The problem with having operators in the endian types is that they encourage the user to continue using them past the point 2). However, as I said, that's my personal preference and last year plenty of people wanted to see the operators included. Perhaps they could be a policy? endian<..., bool EnableOperators_ > ? As for the rest of the library, I have the following comments: - As many other people have mentioned, floating point support and user defined types are a must. - I am happy that the "conversion functions" have been added, Unfortunately, I find them lacking in several departments: - (Note: I am going to try very hard not to get drawn into another naming discussion. That's pretty much what ended my efforts last year - 50 people commenting, each proposing a different set of names :D ) - While the endian types support unaligned data, the endian functions don't appear to do so. I don't see how the endian types could be implemented using them. - No support for user defined types, arrays/ranges of user defined types etc. This is extremely useful. A frequent example I come across is the following (using our syntax): //load a file into memory //... swapInPlace<BigToNative>(data.begin(), data.end()); Depending on the platform, that last statement may get compiled out to a no-op. This is also the big reason why our version of "reorder()" is not unconditional but depends on the "swap type". - reorder(x) will work fine for integral types, however, has potential to cause issues for e.g. floating types. This has been discussed extensively during my proposal last year. Also, from my practical experience, having a floating point number as an input, I find the assembly typically includes (unncecessary) floating point load/store instructions. Something along the lines of e.g.: template<typename T_> T_ big_to_native(raw_memory<T_>) and template<typename T_> swapped_data<T_> native_to_big(T_) might be better and result in better performance. Of course, those two function signatures don't solve the "reorder" signature, but that could be changed correspondingly. - reorder() should use platform specific instructions (such as bswap) for doing endian swapping, otherwise falling back on one of the (many) variants which are currently being discussed. - I think the alternatives such as little_to_big() and big_to_little() are also needed. - I would like to see a few standard traits/typedefs so that I don't have to keep checking pre-processor macros, such as typedef boost::mpl::true_/false_ platform_is_big_endian; or something similar. - While I haven't needed this myself, last year someone mentioned the need for a "general reorder" functionality. Rather than simply dealing with little/big endianness, allow the user to specify ANY byte ordering for the specific data size. - I would like to see an endian_swap_iterator (and a raw_endian_iterator, which can iterate over unaligned data, too) > - What is your evaluation of the implementation? Other than the implementation of reorder() and its performance, which is already being discussed on the list, (and the inclusion of platform specific instructions, as per above) I would like to see endian<> to be implemented in terms of the "free" functions. It seems strange to have two implementations of the same task. > - What is your evaluation of the documentation? Fine. > - What is your evaluation of the potential usefulness of the > library? Very useful, long overdue in boost. > - Did you try to use the library? With what compiler? Did you > have any problems? Tried the integer types last year, no problems. Didn't try the "new" swapping functions. > - How much effort did you put into your evaluation? A glance? A > quick reading? In-depth study? I spent a fair bit of time looking through the docs, the code and reading the followup posts etc. > - Are you knowledgeable about the problem domain? Reasonably. > And finally, every review should answer this question: > > - Do you think the library should be accepted as a Boost library? I see the library as having two pieces, both useful to different people. I would say conditionally accept the endian<> class if floating point support is added. I would vote no for the inclusion of the free function interfaces in their current form. Thanks! Tom

6 14

[graph][iterators] csr graph and reverse iterator problem
by Takatoshi Kondo 11 Sep '11

11 Sep '11

Hi, All I'm using Boost.Graph and Boost.Iterators. When I use the compressed_sparse_row_graph with reverse_iterator, I ran into a problem. When I dereference the reverse_iterator of compressed_sparse_row_graph's iterator, invalid memory access occurs. On g++ version 4.6.0 with -Os option, The output is below. 0 1 1 0 0 1 134526756 3218710936 1 0 And I also checked where the invalid memory access occurred using valgrind. ==2341== Invalid read of size 4 ==2341== at 0x8048DE7: test_csr_graph() (rev_iter.cpp:45) ==2341== by 0x8048FB3: main (rev_iter.cpp:59) ==2341== Address 0xbe825f88 is just below the stack ptr. To suppress, use: --workaround-gcc296-bugs=yes ==2341== 1 ==2341== Invalid read of size 4 ==2341== at 0x8048E31: test_csr_graph() (rev_iter.cpp:46) ==2341== by 0x8048FB3: main (rev_iter.cpp:59) ==2341== Address 0xbe825f88 is just below the stack ptr. To suppress, use: --workaround-gcc296-bugs=yes ==2341== This result indicates the code below is suspicious. std::cout << *rib++ << std::endl; // NG Why? Line 45 std::cout << *rib++ << std::endl; // NG Why? Line 46 But... boost::tie(ib, ie) = boost::vertices(g); std::cout << *--ie << std::endl; // OK std::cout << *--ie << std::endl; // OK It is OK. Here is the source code that reproduces the problem. ------------ begin ------------- #include <boost/iterator/reverse_iterator.hpp> #include <boost/graph/adjacency_list.hpp> #include <boost/graph/compressed_sparse_row_graph.hpp> #include <iostream> #include <cassert> void test_graph() { typedef boost::adjacency_list<> graph_type; graph_type g(2); boost::graph_traits<graph_type>::vertex_iterator ib, ie; { boost::tie(ib, ie) = boost::vertices(g); std::cout << *ib++ << std::endl; // OK std::cout << *ib++ << std::endl; // OK assert(ib == ie); } { boost::tie(ib, ie) = boost::vertices(g); boost::reverse_iterator<boost::graph_traits<graph_type>::vertex_iterator> rib(ie); boost::reverse_iterator<boost::graph_traits<graph_type>::vertex_iterator> rie(ib); std::cout << *rib++ << std::endl; // OK std::cout << *rib++ << std::endl; // OK assert(rib == rie); } } void test_csr_graph() { typedef boost::compressed_sparse_row_graph<> graph_type; std::pair<int, int> e(0, 1); std::pair<int, int> es[] = { e }; graph_type g(boost::edges_are_unsorted, &es[0], &es[1], 2); boost::graph_traits<graph_type>::vertex_iterator ib, ie; { boost::tie(ib, ie) = boost::vertices(g); std::cout << *ib++ << std::endl; // OK std::cout << *ib++ << std::endl; // OK assert(ib == ie); } { boost::tie(ib, ie) = boost::vertices(g); boost::reverse_iterator<boost::graph_traits<graph_type>::vertex_iterator> rib(ie); boost::reverse_iterator<boost::graph_traits<graph_type>::vertex_iterator> rie(ib); std::cout << *rib++ << std::endl; // NG Why? Line 45 std::cout << *rib++ << std::endl; // NG Why? Line 46 assert(rib == rie); } { boost::tie(ib, ie) = boost::vertices(g); std::cout << *--ie << std::endl; // OK std::cout << *--ie << std::endl; // OK assert(ib == ie); } } int main() { test_graph(); test_csr_graph(); } ------------ end ------------- Why does the invalid memory access occur? Am I misusing the libraries? Thanks, Takatoshi

5 10

[type_erasure] ODR violation promotion
by Vicente J. Botet Escriba 10 Sep '11

10 Sep '11

Hi Steven, I'm interested in your TypeErasure library. Where can I find the on going version? The example for concept map seams not complete, as the specialization for less_than_comparable<std::type_info> is not used in any type erasure type. Could you complete it? Before the Boost.Conversion review someone signaled that the Boost.Conversion review was promoting ODR violations as two libraries could specialize a conversion for types S and T and make them incompatible for an end user as the ODR is violated. I think that your library suffer from the same issue, when two libraries need to specialize a concept provided by the library for a 3rd part concrete class using the proposed concept map mechanism. Do you agree? If not, how do the library avoids this kind of issue? Best, Vicente

3 10

[fusion] Algorithm/auxillary documentation
by Daniel James 10 Sep '11

10 Sep '11

Hi, The generated documentation for the algorithm/auxillary section hasn't been checked in. It should be at: http://svn.boost.org/svn/boost/trunk/libs/fusion/doc/html/fusion/algorithm/… http://svn.boost.org/svn/boost/trunk/libs/fusion/doc/html/fusion/algorithm/… Daniel

4 10

[Review] Boost.Endian i/o
by Vicente J. Botet Escriba 10 Sep '11

10 Sep '11

Hi, previous releases of Boost.Endian included a binary stream. Is there something else that can be used to get the same behavior? Best, Vicente

2 1

[Endian] Unaligned fields in packed structs
by Phil Endecott 10 Sep '11

10 Sep '11

Dear All, One of my all-time worst bugs was due to calling htonl() on an unaligned field in a packed struct. It worked in debug mode but not in production code, and there were no compiler warnings about the broken alignment requirement. So I thought I'd try this: inline void inplace_byteswap(uint32_t& val) { val = (val>>24) | ((val>>8) & 0x0000ff00) | ((val<<8) & 0x00ff0000) | (val<<24); } struct S { uint16_t a; uint32_t b; } __attribute__ ((packed)); int main() { S s; s.b = 0x12345678; byteswap(s.b); } This fails to compile: "error: cannot bind packed field ‘s.S::b’ to ‘uint32_t& {aka unsigned int&}’". So it's good that it doesn't just fail unpredictably at runtime like my old nightmare, but it's not ideal. If instead I write: inline uint32_t byteswap(uint32_t val) { return (val>>24) | ((val>>8) & 0x0000ff00) | ((val<<8) & 0x00ff0000) | (val<<24); } ... s.b = byteswap(s.b); ... then of course it does work; the unaligned accesses don't "escape" from where the compiler knows about them and can apply fixups. Anyway, before I tried this I was already wondering about the choice of function signatures: void endian::reorder(T&); // in-place byteswap void endian::reorder(T src, T& dest); // out-of-place byteswap What is the rationale (or precedent) for these and not T endian::reorder(T); ? If sin(theta) returns its result, why shouldn't reorder(val) do so too? Regards, Phil.

3 4

[boost/foreach] What is wrong with BOOST_FOREACH(const std::string &name, m_modules)?
by Demetrio Car 10 Sep '11

10 Sep '11

I am trying to compile this example http://www.boost.org/doc/libs/1_42_0/doc/html/boost_propertytree/tutorial.h… and I am getting the error "/usr/local/boost_1_47_0/boost/property_tree/detail/ptree_implementation.hpp:795:54: error: request for member ‘put_value’ in ‘tr’, which is of non-class type ‘bool’" in this line: BOOST_FOREACH(const std::string &name, m_modules){ pt.put("debug.modules.module", name, true); } My full error message is: make all Building file: ../src/Boost_property_tree.cpp Invoking: GCC C++ Compiler g++ -I/home/mori/Demetrio/WSs/Test/VanManagerParser/json_spirit -I/usr/local/boost_1_47_0/ -I/usr/local/boost_1_47_0/boost/spirit -I/usr/local/curl-7.21.7/ -O0 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"src/Boost_property_tree.d" -MT"src/Boost_property_tree.d" -o "src/Boost_property_tree.o" "../src/Boost_property_tree.cpp" ../src/Boost_property_tree.cpp: In member function ‘void debug_settings::save(const std::string&)’: ../src/Boost_property_tree.cpp:80:5: error: invalid initialization of reference of type ‘std::string&’ from expression of type ‘const std::basic_string<char>’ In file included from /usr/local/boost_1_47_0/boost/property_tree/ptree.hpp:516:0, from ../src/Boost_property_tree.cpp:4: /usr/local/boost_1_47_0/boost/property_tree/detail/ptree_implementation.hpp: In member function ‘void boost::property_tree::basic_ptree<Key, Data, KeyCompare>::put_value(const Type&, Translator) [with Type = std::basic_string<char>, Translator = bool, Key = std::basic_string<char>, Data = std::basic_string<char>, KeyCompare = std::less<std::basic_string<char> >]’: /usr/local/boost_1_47_0/boost/property_tree/detail/ptree_implementation.hpp:817:13: instantiated from ‘boost::property_tree::basic_ptree<K, D, C>& boost::property_tree::basic_ptree<Key, Data, KeyCompare>::put(boost::property_tree::basic_ptree<Key, Data, KeyCompare>::path_type&, const Type&, Translator) [with Type = std::basic_string<char>, Translator = bool, Key = std::basic_string<char>, Data = std::basic_string<char>, KeyCompare = std::less<std::basic_string<char> >, boost::property_tree::basic_ptree<Key, Data, KeyCompare>::path_type = boost::property_tree::string_path<std::basic_string<char>, boost::property_tree::id_translator<std::basic_string<char> > >]’ ../src/Boost_property_tree.cpp:81:50: instantiated from here /usr/local/boost_1_47_0/boost/property_tree/detail/ptree_implementation.hpp:795:54: error: request for member ‘put_value’ in ‘tr’, which is of non-class type ‘bool’ make: *** [src/Boost_property_tree.o] Error 1 **** Build Finished ****

4 5

[Endian] Review / Comments
by Tim Blechmann 10 Sep '11

10 Sep '11

hi, documentation the documentation in general is pretty short, but sufficient: the concepts of the library are easy, so there is no need to go into more details. however i would prefer if the documentation uses quickbook/boostbook. most of the recent boost libraries have a very similar layout of the documentation, but the boost.endian docs do not follow these conventions. also the docs for `conversion functions' and for `integer types' are separate. personally i'd prefer if they could go into the same document, maybe having a structure like: introduction, integer types, conversion functions, examples, reference (preferably doxygen-generated). section `experience': this section gives no insights for people who use or read the code. it mainly tells: "we are not the first and the domain of the library is important.". imo this section can be removed (maybe the part that it is not related to any c library can go to the `design considerations' section "motivating use cases": this is more a marketing blurb/testimonial. maybe this could be changed to a section about possible use cases, listing `communicating between different devices' and `reading/writing of binary file formats'. provided files i suppose, the msvc project file will go away when the library is merged into trunk. besides, i'd suggest to move "boost/endian/support/timer.hpp" to "lib/endian/support", because it only seems to be used by the benchmarks. testsuite scoped_enum_emulation_test.cpp ... not related to boost.endian, please remove endian_operations_test.cpp and endian_in_union_test.cpp ... maybe rename from _test.cpp to _compile_test.cpp? they don't seem to do any run-time tests. they also should not include <cassert> since no assertion statement is needed, this might speed up the compilation time of the testsuite by something like 50ms ;) conversion functions personally, i find the interface for the conversion functions rather odd, as they don't actually return a value, but take a reference to the return value. so they cannot really be used in a functional style. if the reorder functions would have a signature like: int16_t reorder(int16_t x); it would be possible to use it as unary operator e.g. in with std::transform and the like. this would also avoid the need for both in-place and out-of- place functions. iac, i don't see a reason for the in-place functions. for the reason, see below. performance considerations the current implementation seems to perform horribly on x86_64 hardware. i ran phil's nice benchmark program inside the `perf' profiler on my core i7 920 ("g++-4.6 -O3 -std=gnu++0x -march=native"), where the algorithm takes MUCH more time than htonl, gcc's builtins or a mask/shift implementation. the main reason seems to be a huge amount of byte-wise load/stores, which lead to a huge number of stalls. a mask/shift implementation will be computed completely in registers, making use of instruction-level parallelism on out- of-order cpus and requiring less memory transfers. as a side effect, boost.endian will compose much better, when using the reorder functions on values, which are already in registers, or which need to be placed in registers later in the algorithm. modification for bitmasks: return (src<<24) | ((src<<8) & 0x00ff0000) | ((src>>8) & 0x0000ff00) | (src>>24); modification for boost.endian types (assuming little-endian hardware): uint32_t n; boost::detail::store_big_endian<uint32_t, sizeof(uint32_t)>(&n, src); return n; USE_GCC_BUILTIN: Performance counter stats for './a.out': 986.666145 task-clock # 1.000 CPUs utilized 0 context-switches # 0.000 M/sec 7 CPU-migrations # 0.000 M/sec 307 page-faults # 0.000 M/sec 2,637,245,698 cycles # 2.673 GHz 633,384,188 stalled-cycles-frontend # 24.02% frontend cycles idle 281,512,771 stalled-cycles-backend # 10.67% backend cycles idle 6,007,671,727 instructions # 2.28 insns per cycle # 0.11 stalled cycles per insn 1,001,451,092 branches # 1014.985 M/sec 31,112 branch-misses # 0.00% of all branches 0.987133571 seconds time elapsed USE_HTONL Performance counter stats for './a.out': 938.226124 task-clock # 1.000 CPUs utilized 0 context-switches # 0.000 M/sec 3 CPU-migrations # 0.000 M/sec 307 page-faults # 0.000 M/sec 2,507,753,527 cycles # 2.673 GHz 504,694,743 stalled-cycles-frontend # 20.13% frontend cycles idle 53,006,181 stalled-cycles-backend # 2.11% backend cycles idle 6,006,376,475 instructions # 2.40 insns per cycle # 0.08 stalled cycles per insn 1,001,180,324 branches # 1067.099 M/sec 23,212 branch-misses # 0.00% of all branches 0.938684989 seconds time elapsed USE_BEMAN Performance counter stats for './a.out': 7494.906291 task-clock # 1.000 CPUs utilized 0 context-switches # 0.000 M/sec 12 CPU-migrations # 0.000 M/sec 307 page-faults # 0.000 M/sec 20,033,609,533 cycles # 2.673 GHz 15,018,356,865 stalled-cycles-frontend # 74.97% frontend cycles idle 11,372,397,445 stalled-cycles-backend # 56.77% backend cycles idle 15,030,059,048 instructions # 0.75 insns per cycle # 1.00 stalled cycles per insn 1,005,600,935 branches # 134.171 M/sec 88,848 branch-misses # 0.01% of all branches 7.497572144 seconds time elapsed USE_BITMASK Performance counter stats for './a.out': 986.085196 task-clock # 0.999 CPUs utilized 0 context-switches # 0.000 M/sec 0 CPU-migrations # 0.000 M/sec 307 page-faults # 0.000 M/sec 2,635,658,193 cycles # 2.673 GHz 631,672,783 stalled-cycles-frontend # 23.97% frontend cycles idle 269,536,819 stalled-cycles-backend # 10.23% backend cycles idle 6,008,101,435 instructions # 2.28 insns per cycle # 0.11 stalled cycles per insn 1,001,523,773 branches # 1015.656 M/sec 31,020 branch-misses # 0.00% of all branches 0.986601856 seconds time elapsed USE_TYPES Performance counter stats for './a.out': 7121.796026 task-clock # 1.000 CPUs utilized 0 context-switches # 0.000 M/sec 12 CPU-migrations # 0.000 M/sec 307 page-faults # 0.000 M/sec 19,035,784,897 cycles # 2.673 GHz 15,019,771,466 stalled-cycles-frontend # 78.90% frontend cycles idle 1,182,185,735 stalled-cycles-backend # 6.21% backend cycles idle 14,032,012,033 instructions # 0.74 insns per cycle # 1.07 stalled cycles per insn 1,005,916,931 branches # 141.245 M/sec 84,751 branch-misses # 0.01% of all branches 7.124340206 seconds time elapsed What is your evaluation of the potential usefulness of the library? very important, very useful How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? roughly 3 hours, reading code, documentation, running some benchmarks. besides, i'm using an older version of the library for about 2 years without having any problems. Are you knowledgeable about the problem domain? yes Do you think the library should be accepted as a Boost library? the library should be accepted, if (a) the interface of the conversion functions is changed (b) the performance can be improved (c) the documentation integrates better with the rest of the boost documentation. cheers, tim

5 8