April 2010 - Boost - lists.preview.boost.org

Idea: Boost.Allocator
by joel falcou 18 Apr '10

18 Apr '10

I recently had to use Boost Iterator Facade and its relatives and found the idea great. It occured to me whiel lookign at my X different, hand written allocaotrs for various project that maybe such a system for writing std allocator could be great. Buildign a new allcoator will just be impelemnt some allocator_facade<> adn fill some small scale methods. Moreover, it could be the opportunity to add new allocator concept like allocator supporting resize calls, allcoator with state, allcoator with a allocated() method etc... and reality-check them in the boost framework. How does it sounds ? -- ___________________________________________ Joel Falcou - Assistant Professor PARALL Team - LRI - Universite Paris Sud XI Tel : (+33)1 69 15 66 35

4 5

Re: [boost] [GGL] [geometry] Inexplicable speed benefit when using Visual C++ 2010
by Stephan T. Lavavej 18 Apr '10

18 Apr '10

[Christian Buchner] > We found that GCC 4.4 on Linux was about 100% faster than Visual C++ > 2008 on Linux without modifying the code. Wow, I didn't know that VC9 could target that platform. ;-) > we attributed much of the performance difference to sub-optimal memory > heap management of Visual C++ 2008. VC's default implementation of ::operator new() calls malloc(), which calls HeapAlloc() from the Windows API. HeapAlloc()'s implementation is mostly a mystery to me, but there are a couple of things about it that you should probably be aware of. First, on WinXP and higher (note that VC9 supported targeting Win2K, but VC10 doesn't), Windows implements the Low Fragmentation Heap (LFH). The LFH services "small" allocations (definitely sub-KB; I'd have to ask around in order to figure out the precise limit) from buckets, so applications that allocate lots of small chunks of memory (like nodes) can benefit significantly from the LFH. I believe it performs other wizardry which I'm not aware of. However, on some platforms, the LFH is not enabled by default. You can request it by calling HeapSetInformation() on the CRT heap. There are two situations where the LFH is enabled by default. First, VC enables the LFH for its CRT heap when targeting x64. (For obscure reasons, we can't do this for x86.) Second, WinVista and higher automagically enable the LFH when they detect allocation patterns that would benefit from this. I've spoken to the LFH's maintainers, and they say that this automagic machinery is sufficiently smart, that enabling the LFH manually is no longer necessary (i.e. they have never seen that produce an observable benefit). > Then we tried recompiling the project with Visual C++ 2010 Ultimate > Release Candidate (RC). The speed gain of the algorithm was 900% (not > joking) and the results still appear to be correct. This greatly amuses me, as I talked about seeing order-of-magnitude performance improvements but never expected to actually see one in the real world. This is almost certainly due to rvalue references, as nothing else that we did could possibly have had so large of an effect. VC10 implemented rvalue references (v2, but not the stuff in the FCD that automatically generates move ctors/assigns), and we updated the Standard Library to take advantage of them (both by providing move ctors/assigns on Standard Library types like vector, and by taking advantage of move ctors/assigns during operations like vector reallocation). > Is anyone else seeing similar speedups in boost or in the geometry > library when compiling with Visual C++ 2010 RC (HINT: it's a free > download, so anyone can try it out until end of June 2010). Note that VC10 RC is perfectly representative of VC10 RTM. The compiler and linker are unchanged, and the Standard Library received exactly one bugfix (for obnoxious compiler errors in shared_ptr). Also note that while GCC 4.3 and higher support rvalue references (v1 in 4.3 and 4.4, v2 in 4.5), they are not enabled by default. You must compile with either -std=c++0x or -std=gnu++0x in order to activate their C++0x mode. I'd be interested to learn how VC10 fares against GCC in C++0x mode. (I'm unsure as to when and to what extent rvalue references were applied to GCC's Standard Library implementation libstdc++. As GCC 4.5 was released mere days after VC10, that would generate the most fair comparison.) About _SECURE_SCL, the worst performance penalty that I've ever seen it exact is 2x. It could have been responsible (along with allocator mumbo-jumbo) for the 100% difference you observed between GCC 4.4 and VC9, but not for the 900% difference between VC9 and VC10 (unless you have discovered a truly astounding pathological case). _SECURE_SCL (now combined with _HAS_ITERATOR_DEBUGGING into _ITERATOR_DEBUG_LEVEL) in VC10 is now disabled by default in release mode. (In debug mode, _HAS_ITERATOR_DEBUGGING, aka _ITERATOR_DEBUG_LEVEL=2, is still enabled by default - it can affect performance significantly, but provides exhaustive correctness checks, and debug performance isn't especially important unless it's 1000x slower and renders the program too slow to debug.) VC10's linker now deterministically detects when you're mixing translation units compiled with different settings of _ITERATOR_DEBUG_LEVEL, so that (combined with the new default of 0 in release mode) will put an end to the bad old days of incomprehensible ODR violations triggered by attempting to disable this machinery in some but not all libraries. Do note that mixing major compiler versions (e.g. VC9 and VC10) is still completely forbidden when you're using the C++ Standard Library, as we break binary compatibility between every major version and will continue to do so for the foreseeable future. Unfortunately, the linker's new feature (#pragma detect_mismatch) is unable to detect VC9 and VC10 mixing, so you'll just have to be careful. (The machinery is in place for us to detect VC10 and VC11 mixing.) [Barend Gehrels] > I've asked last year during BoostCon to Microsoft, present there, if > they knew this issue, but apparently they didn't, and told me it would > surprise him because the compiler basically was the same. (That was me.) > When I used after that once the VC 2008 command line compiler (so not > from Visual Studio) the problem disappeared. So it was fast. So it was > not the compiler itself, it was some setting in the IDE or VPROJ. I've > never managed to find which setting, though I studied it carefully. According to my understanding, the VS 2008 IDE had a bug when upgrading VS 2005 projects, where optimization settings would be lost. I'm unsure of what the exact symptoms of the bug looked like (I thought it was just the /O options), but your experience (that it disappeared when you used the command line) perfectly fits with this hypothesis. Alternatively, you could create a VS 2008 project from scratch. I believe that VS 2010's project system (which underwent a massive overhaul) does not suffer from this bug when upgrading either VS 2005 or VS 2008 projects, but I don't know what happens when you take a project that was damaged by the VS 2005 to VS 2008 upgrade, and then upgrade it to VS 2010. I suspect that in that case, the damage persists. My understanding of this is vague because I don't use IDE build systems. Stephan T. Lavavej Visual C++ Libraries Developer

2 1

[call-traits][param_type] why is sizeof(void*) chosen as the limit for passing by value?
by Thorsten Ottosen 17 Apr '10

17 Apr '10

Hi Boosters, It seems to me that many other slightly larger types could benefit from pass by value. For example, I rarely write foo( const double& ); but always foo( double ); So param_type<T>::type don't even follow this guideline. I tried a simple example program for the auto_buffer (vector like) class I'm implementing: #include <boost/auto_buffer/auto_buffer.hpp> int main() { boost::auto_buffer<double> buf( 2001u ); buf.push_back(42.); } When I pass by value in push_back(), the assembler is somewhat different, and as far as I can tell, better. Basically my compiler (vc9) can inline the load of 42. into push_back() whereas in the call by reference, the parameter is passes by a pointer (in a register). In the by-value version the compiler is able to optimize away completely the call to alllocate and delete memory whereas this is not so for the by-reference version. This makes me wonder why we shouldn't expect e.g. std::pair<int,int> and std::bitset<64> to be easily passable in registers. With this in mind, I think we should consider changing the limit for param_type to sizeof(long double) or even 2*sizeof(int). Of course, we should only pass types with a trivial assignment operator by value. Any thoughts? -Thorsten

4 4

Re: [boost] [Bind] How do I get bind to deduce the return type of for_each?
by Thomas Jordan 17 Apr '10

17 Apr '10

> Message: 8 > Date: Thu, 15 Apr 2010 10:51:27 +0100 > From: "Sylvester-Bradley, Gareth" > <Gareth.Sylvester-Bradley(a)eu.sony.com> > To: <boost(a)lists.boost.org> > Subject: Re: [boost] [Bind] How do I get bind to deduce the return > type offor_each? > Message-ID: > <84883B934769D3428C4AE2E403B188800A1A0F9A(a)GBISE0BASMSX02.eu.sony.com> > Content-Type: text/plain; charset="us-ascii" > >> I have been trying to use Boost.bind to create an 'STL algorithm > function >> object' to pass into another, 'higher order,' STL algorithm (e.g., to > pass >> a for_each into a for_each). However, I am struggling to get bind to >> deduce the correct return type of the for_each function object. > > The following might help: > > http://www.boost.org/doc/libs/1_42_0/doc/html/lambda/le_in_details.html# > lambda.nested_stl_algorithms > > > Best regards, > Gareth > Thanks a lot for the suggestion, I did take a look at Lambda library, and I saw that it used some extra type information in its STL algorithm wrappers (the 'sig') to help it figure out the return types. However, I am against using the Lambda library for other reasons, I only want to use the bind library. I'd appreciate some specific advice on how to get the type deduction with the Boost.bind library. Or is this just not possible with Boost.bind?

4 3

(no subject)
by Dan Jiang 16 Apr '10

16 Apr '10

Hi All, I have a question regarding CSR graph. Does anyone know if a CSR graph can be used as input to push_relabel_max_flow or edmonds_karp_max_flow algorithm in BOOST? The existing examples only use adjacency_list with the maximum flow algorithms. I need to use CSR graph with the max flow implementations, but do not know how and if it is feasible at all. If answer is yes, can anyone give a simple example? I am having trouble in setting capacity map and reverse_edge map in CSR before passing it to the max flow algorithms. -Thanks a lot for your time. Dan Jiang _________________________________________________________________ Live connected. Get Hotmail & Messenger on your phone. http://go.microsoft.com/?linkid=9724462

2 10

[xint] Question about suitability, portability, and "Boostiness"
by Chad Nelson 16 Apr '10

16 Apr '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Right now, the XInt library uses a pointer to a (privately defined) struct. The struct has several items, including a small array of "quick digits" and a pointer to a dynamically allocated array of digit_t types, both for containing the magnitude of the integer. While trying to find a more efficient way to store things, I came up with this: struct new_data_t { size_t used, allocated; // (other data items) digit_t digits[1]; // Last item in the struct }; It's an old C trick, apparently legitimized in the C99 standard as the "struct hack". For those not familiar with it, the idea is that I'd allocate a single array of digit_t types, of the required size plus sizeof(new_data_t), then just typecast it as a pointer of type new_data_t. Does anyone know if there's any reason not to do this? Maybe some portability or alignment problem with it? - -- Chad Nelson Oak Circle Software, Inc. * * * -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvD3ZQACgkQp9x9jeZ9/wQLdQCgklN2Xcs0mDAGhsy4n3TX/Bgw JVoAoK+/YEoawXT7Yd0R3++eWLPYZhjr =Jx8O -----END PGP SIGNATURE-----

4 15

Re: [boost] How to construct a reverse_edge_map from CSR
by Dan Jiang 16 Apr '10

16 Apr '10

Hi Jeremiah, Here is the code for this error. Hope it is more readable. typedef compressed_sparse_row_graph<directedS> Traits; struct EdgeProp{ float capacity; float residual_capacity; Traits::edge_descriptor reverse_edge; }; typedef compressed_sparse_row_graph<directedS, property<vertex_index_t, DWORD32>, EdgeProp, // bundled property no_property, // Graph Property DWORD32, // Vertex Type DWORD32> // EdgeIndex Type BOOST_Graph; Errors: C:\Program Files (x86)\boost\boost_1_42\boost/graph/compressed_sparse_row_graph.hpp(1321) : error C2780: 'V boost::target(const incidence_graph_archetype<Vertex,Directed,ParallelCategory,Base>::edge_descriptor &,const boost::incidence_graph_archetype<Vertex,Directed,ParallelCategory,Base> &)' : expects 2 arguments - 1 provided C:\Program Files (x86)\boost\boost_1_42\boost/graph/graph_archetypes.hpp(64) : see declaration of 'boost::target' .\NetFlow.cpp(209) : see reference to function template instantiation 'std::pair<_Ty1,_Ty2> boost::edge<boost::directedS,boost::property<Tag,T>,EdgeProp,boost::no_property,unsigned int,DWORD32>(Vertex,Vertex,const boost::compressed_sparse_row_graph<Directed,VertexProperty,EdgeProperty,GraphProperty,Vertex,EdgeIndex> &)' being compiled with [ _Ty1=boost::detail::csr_edge_descriptor<size_t,size_t>, _Ty2=bool, Tag=boost::vertex_index_t, T=DWORD32, Vertex=unsigned int, Directed=boost::directedS, VertexProperty=boost::property<boost::vertex_index_t,DWORD32>, EdgeProperty=EdgeProp, GraphProperty=boost::no_property, EdgeIndex=DWORD32 ] C:\Program Files (x86)\boost\boost_1_42\boost/graph/compressed_sparse_row_graph.hpp(1321) : error C2780: 'Vertex boost::target(compressed_sparse_row_graph<Directed,VertexProperty,EdgeProperty,GraphProperty,Vertex,EdgeIndex>::edge_descriptor,const boost::compressed_sparse_row_graph<Directed,VertexProperty,EdgeProperty,GraphProperty,Vertex,EdgeIndex> &)' : expects 2 arguments - 1 provided Error line is: if (target(*range.first) == j) C:\Program Files (x86)\boost\boost_1_42\boost/graph/compressed_sparse_row_graph.hpp(1241) : see declaration of 'boost::target' C:\Program Files (x86)\boost\boost_1_42\boost/graph/compressed_sparse_row_graph.hpp(1321) : error C2780: 'Vertex boost::target(const boost::detail::edge_base<Directed,Vertex> &,const boost::adjacency_list<OutEdgeListS,VertexListS,DirectedS,VertexProperty,EdgeProperty,GraphProperty,EdgeListS> &)' : expects 2 arguments - 1 provided C:\Program Files (x86)\boost\boost_1_42\boost/graph/adjacency_list.hpp(530) : see declaration of 'boost::target' C:\Program Files (x86)\boost\boost_1_42\boost/graph/compressed_sparse_row_graph.hpp(1321) : error C2780: 'filtered_graph_base<G>::vertex_descriptor boost::target(filtered_graph_base<G>::edge_descriptor,const boost::filtered_graph_base<G> &)' : expects 2 arguments - 1 provided: Error line is: if (target(*range.first) == j) > > Could you please reformat your code to be more readable? What is the > exact type of your graph, and what are the definitions of the vertex and > edge bundled property classes? > > -- Jeremiah Willcock > _______________________________________________ > Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost _________________________________________________________________ Live connected. Get Hotmail & Messenger on your phone. http://go.microsoft.com/?linkid=9724462

2 1

How to construct a reverse_edge_map from CSR
by Dan Jiang 16 Apr '10

16 Apr '10

Hi All, I am having difficulty in constructing a reverse_edge_map from an existing CSR. My CSR was created as "edges_are_unsorted", I need to create a reverse_edge_map out of it? Here is my attempt:1 BGL_FORALL_EDGES(e, g, BOOST_Graph)2 {3 // Get source and target of the edge4 BOOST_Graph::vertex_descriptor src = boost::source(e, g);5 BOOST_Graph::vertex_descriptor dst = boost::target(e, g);678 // Get reverse edge9 graph_traits<BOOST_Graph>::out_edge_iterator ei, ei_end;10 for (tie(ei, ei_end) = edge_range(dst, src, g); ei != ei_end; ++ei) 11 {12 rev[*e] = *ei;13 break; // I know there is only one reverse edge in my graph14 } 15 } I got compile error on line 10: error C2784: 'std::pair<Config::out_edge_iterator,Config::out_edge_iterator> boost::edge_range(Config::vertex_descriptor,Config::vertex_descriptor,const boost::adj_list_helper<Config,Base> &)' : could not deduce template argument for 'const boost::adj_list_helper<Config,Base> &' from 'BOOST_Graph' where BOOST_Graph is a CSR graph that has been instantiated as "edges_are_unsorted" prior to the loop. Any help is appreciated. -Thanks, Dan _________________________________________________________________ Got a phone? Get Hotmail & Messenger for mobile! http://go.microsoft.com/?linkid=9724464

2 1

Re: [boost] Crypto Proposal
by Chad Seibert 16 Apr '10

16 Apr '10

Hello, > Does this mean you will be converting the source-code of Botan to Boost > standards or you are going to wrap it? > 1. If it is conversions two issues may rise: > a) License Botan is licensed under BSD license with is different from > Boost and all the code you will write for boost would not have > correct license. > b) Conversion to Boost means fork... How do you expect to synchronize > changes in two libraries and most important security updates. > This means lots of work in the live cycle of Boost.Botan I'll fork it, as stated in my proposal (which I sent sometime after the abstract). It'll be relicensed under BSL. As for synchronization, this will have to be done by hand (which I will do for some time). > Additional points: > - Botan uses GNU gmp library licensed under LGPL... Does it fit to > Boost licensing guidelines? It doesn't require gmp and is perfectly capable of working without it. It's available as a plugin, but a default implementation is already included. > - I think that you should be really specific what would be the advantage > of using Boost.Botan library over original Botan one or OpenSSL that > Boost.Asio uses. Here are some: * Tighter integration with Boost (ASIO, and possibly iostreams) * Some functions are considerable faster (such as RSA, which the maintainer purports to be several times faster than OpenSSL. SHA-1 is SSE2 optimized). * There is plugin support for OpenSSL, so ASIO would be modified to use the plugin system (which would use the default implementation or OpenSSL). > You might want to consider adding some discussion to your proposal about the > crypto library in the vault. What features are in the vault Crypto that > might be useful? That aren't useful? Why do you feel that Botan is a better > starting point. The crypto library in the vault contains mostly hash algorithms, which Botan already contains (and some of them e.g. SHA-1 have a SSE optimized version). And Botan contains better documentation and testing. Also, Botan was engineered mostly by a security expert, meaning it is likely to be more secure. It is also being maintained, so security and feature patches will be made. > Your proposal is very ambitious! IMHO, that's not doable in full over the > summer. If you trimmed it down into a more focused feature set (for instance > replacing SSL in ASIO, or adding TLS to ASIO) it would give you a more > realistic chance to finish what you are suggesting. It is very ambitious! But as I say in my draft, it'll be done in two parts. I don't think that porting is unrealistic for the summer part of the project. Many thanks for the feedback, Chad Seibert _________________________________________________________________ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28…

3 6

Re: [boost] Crypto Proposal
by Chad Seibert 16 Apr '10

16 Apr '10

Here's a response to a question asked on the private review system: The design and architecture is based on dynamic polymorphism, for better or worse. In the process of porting it to Boost.Botan, it would be made static. On the mailing list, someone created a simple project that creates a certificate and the exe grew from 40kb to 1mb. Having static polymorphism would solve this problem. Also, we won't have to pay the runtime penalties for dp (especially since it's not needed - something that Boost libraries have shown time and time again). Looking at other libraries under similar licenses yielded similar results. Crypto++ also uses DP (and has even larger hierarchies) and for simple uses, the entire library get's built and included with project. For example, http://www.cryptopp.com/docs/ref/class_s_h_a1.html. SP will solve this problem, and Botan uses less DP which should make it easier to convert. I couldn't find a valid libtomcrypt homepage, so I don't know how it's structured. The interface of Botan is simple and intuitive. Looking at their sample programs (and other samples), there doesn't seem to be a simpler method to implement the various techniques and still remain flexible. Crypto++ samples seem to be a bit difficult to digest, requiring manual creation of secure memory blocks and ugly dynamic casting. More can be seen here: http://www.cryptopp.com/wiki/Diffie-Hellman. All in all, Botan has a simpler interface and a DP hierarchy that shouldn't be (as) difficult to convert to SP. I wanted to compare against libtomcrypt, but couldn't find a valid implementation. An example of how something might get ported would be the following. Botan.AES_128 has the inheritance hierarchy Botan->SymmetricAlgorithm->BlockCipher->AES->AES_128. Writing this as SP would look like the following (if I understand SP correctly): Before: class SymmetricAlgorithm { void set_key(...) } class BlockCipher: SymmetricAlgorithm { virtual void encrypt(...) virtual void decrypt(...) } class AES: BlockCipher { void encrypt(...); void decrypt(...); } class AES_128: AES { } There is no reason for AES. I'd first remove that class and have AES_128 derive from BlockCipher directly. Converting this to SP yields (sorry, I don't have a compiler to test this atm): template<class BlockCipher> struct SymmetricAlgorithm { void set_key_impl() { static_cast<*BlockCipher>(this)->set_key(); } } template <class AES_128> struct BlockCipher: SymmetricAlgorithm<BlockCipher>{ void set_key(); //More stuff } However, there is one concern. SymmetricAlgorithm is the base class of three major hierarchies: BlockCipher, MessageAuthenticationCode, and StreamCipher. If this inheritance hierarchy stays, it will mean lot's of template metaprogramming (some of which I don't know how to do). In any case, it's easier to work with than Crypto++. For example, SHA1 in Crypto++ has a hierarchy depth of 9; Botan 5. Hope this answers some questions! Glad you are interested in my work! Chad Seibert _________________________________________________________________ The New Busy is not the old busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:…

4 3