[BGL] Distributed adiacency_list memory allocation
Hi to all, I'm working on WinXP, VS2010, Boost 1.46.1. I've writed this short program: //---------------------------------- #include <boost/graph/use_mpi.hpp> #include <boost/graph/distributed/mpi_process_group.hpp> #include <boost/graph/distributed/adjacency_list.hpp> using boost::graph::distributed::mpi_process_group; typedef boost::property<boost::vertex_distance_t, long, boost::property<boost::vertex_name_t, std::string > > vertex_prop; typedef boost::property<boost::edge_weight_t, long > edge_prop; typedef boost::adjacency_list< boost::vecS, boost::distributedS<mpi_process_group, boost::vecS>, boost::directedS, vertex_prop, edge_prop > Graph; typedef boost::graph_traits<Graph>::edge_descriptor EdgeDescriptor; typedef boost::graph_traits<Graph>::vertex_descriptor VertexDescriptor; int main(int argc, char* argv[]) { boost::mpi::environment env(argc,argv); Graph g( 10 ); return 0; } //---------------------------------- When I launch it, every process allocates about 1 GB of memory. Why it allocates all that memory? Is it normal? Is there any tuning to the PBGL framework that I must made? I use it on a 2-core machine, and the command that I use is: mpiexec -n 2 .\test.exe I've tried with openMPI 1.4.3 and Microsoft HPC Pack 2008, but it seems irrilevant. Thanks in advance, Cosimo Calabrese.
Hi Cosimo, I run into the same problem as you, and I'm so disturbed by the question. Have you got the answer now? I'll appreciate for some ideas. Yan -- View this message in context: http://boost.2283326.n4.nabble.com/BGL-Distributed-adiacency-list-memory-all... Sent from the Boost - Users mailing list archive at Nabble.com.
Hi Cosimo, I run into the same problem as you, and I'm so disturbed by the question. Have you got the answer now? I'll appreciate for some ideas.
Yan
Sorry, I can't remember which of these "PBGL consumes lots of memory" questions I've replied to and which I haven't but the key point here is that you have to consider not only the distributed graph itself, but other auxiliary data structures that may be created and most importantly in the "my test case is small but it allocates lots of space" case, the memory needed by the Process Group to buffer MPI_Isend and MPI_Irecv data. Check out libs/graph_parallel/src/mpi_process_group.cpp... In there you'll see a #define for PREALLOCATE_BATCHES which tells the MPI Process Group how many "batches" to preallocate, where a batch is the size of the message buffer which, when full, invokes an MPI_Isend. This is all defined in mpi_process_group.tpp in the mpi_process_group::impl object. So... before you even create a single distributed data structure you have PREALLOCATE_BATCHES * (mpi_process_group::impl::batch_buffer_size + mpi_process_group::impl::batch_message_size) bytes allocated for buffering MPI communication... and those are just the *large* objects in the mpi_process_group. I'm sure you can find plenty of other moderately sized ones that will add up as well. The OOB Bsend buffer cache is a good example if you're using SEND_OOB_BSEND (off by default I believe). The adjacency list is by no means the most compact data structure and you'll certainly find some unexpected uses of memory in there, especially if you're using undirected edges, but for small problem instances most of the memory consumption is in the process group. If you're concerned about memory utilization and don't need mutable graphs I'd definitely recommend the CSR graph type though. Hope that helps, Nick
Thank you very much for your reply and the details. This is exactly what I was looking for, though I don't understand all of your words, but most. At least now I'm sure its not an error but normal. By the way, I'm new to PBGL, would you please tell me some forums or websites that good for beginers of PBGL. Thank you ! Best regards, Yan -- View this message in context: http://boost.2283326.n4.nabble.com/BGL-Distributed-adiacency-list-memory-all... Sent from the Boost - Users mailing list archive at Nabble.com.
Hi all, I am working on an application where we have a plugin written to an application, and then a second app that communicates with the plugin using boost::ipc. For this we are using boost 1.46. The 2012 release of the application we are plugging into has released their SDK and included boost 1.35 as part of the SDK (so they can hook into boost python.) Is there any way for us to continue using boost 1.46 for our plugin even though it is compiling in an sdk which includes boost 1.35? If not, is there a good alternative to boost ipc which we can use to handle this cross-application communication? Maybe by limiting ourselves to the boost 1.35? (The developer who actually implemented the boost::ipc switching our code from using qt signals and slots [who isn't with us anymore] told us that boost 1.46 was needed for ipc as it contained a necessary fix to make it work.) Any advice would be appreciated. Thanks, Liron Software Developer Image Metrics Liron.Kopinsky@image-metrics.com
I am working on an application where we have a plugin written to an application, and then a second app that communicates with the plugin using boost::ipc. For this we are using boost 1.46.
The 2012 release of the application we are plugging into has released their SDK and included boost 1.35 as part of the SDK (so they can hook into boost python.)
Is there any way for us to continue using boost 1.46 for our plugin even though it is compiling in an sdk which includes boost 1.35?
What do you mean by saying that your plugin "is compiling in an sdk"? Is this SDK a collection of source files, static libs, DLLs?
The sdk is a collection of source files and libs. We include those libs and source when we statically compile our dll, and then we add the compiled dll to the application's plugin directory to add it to the app. -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Igor R Sent: Tuesday, September 20, 2011 4:23 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Multiple versions of boost at once/IPC Alternative
I am working on an application where we have a plugin written to an application, and then a second app that communicates with the plugin using boost::ipc. For this we are using boost 1.46.
The 2012 release of the application we are plugging into has released their SDK and included boost 1.35 as part of the SDK (so they can hook into boost python.)
Is there any way for us to continue using boost 1.46 for our plugin even though it is compiling in an sdk which includes boost 1.35?
What do you mean by saying that your plugin "is compiling in an sdk"? Is this SDK a collection of source files, static libs, DLLs?
This is exactly what I was looking for, though I don't understand all of your words, but most. At least now I'm sure its not an error but normal. By the way, I'm new to PBGL, would you please tell me some forums or websites that good for beginers of PBGL.
I think this list is the best place I can suggest. I'm not sure how many people are using the Boost release but most of the people actively using the development version are people I know personally or professionally and they tend to just contact me directly. The folks on this list ought to be able to help with some of the basic parts though and I'll try to chime in when something about the internals comes up (i.e. your question about memory consumption). Cheers, Nick
Hi Nick, I'm sorry for disturbing you again for some more help. These days I have done some more test using the distributed adjacency list and distributed dijkstra SP-algorithm. The test data contains about 3,800,000 edges and 2,800,000 vertices. I use a pair of iterators to pass the data to the adjacency list, like that: Graph g(reader.begin(), reader.end(), reader.weight_begin(), reader.num_vertices(), pg, dist); "reader" is an user-defined class which supplys required data (edges, vertices, weights). "pg" is defined like that "mpi_process_group pg;" "dist" is supplied by the reader like that "graph::metis_distribution dist = reader.distribution();". It is based on a metis partition file coordinating the test data. I run 2 processes on my notebook computer(Intel core2 2.0G, Ram 2G, WinXPsp3). I also test the same data using a sequential bgl dijkstra SP-algorithm on this computer. The result is that: Parallel bgl: Creating of Graph g costs 10080 seconds; dijkstra computing costs 28.281 seconds. Sequential bgl: Creating of Graph g costs 4.563seconds; dijkstra computing costs 1.703 seconds. As you see, when using distributed adjacency list, it costs so much time to create the graph. Is it nomal? I have done some more comparison between 2-processes parallel dijkstra on one core2 computer and sequential dijkstra on the same computer, using different scale data set. Always, the sequential bgl dijkstra costs less time than the 2-processes parallel dijkstra. What's the problem? I have read the boost document on "boost_1_46_1\libs\graph_parallel\doc\html\dijkstra_shortest_paths.html", which shows a good performance and speedup in the charts. Doesn't that mean less time costing than the sequential program? By the way, when using distributed dijkstra, how to invoke the delta-stepping algorithm? Is it also through the dijkstra_shortest_paths(g, start, ...)? Thank you very much for all your help. Best regards! Yan ________________________________ 发件人： Nick Edmonds [via Boost] <ml-node+s2283326n3837765h68@n4.nabble.com> 收件人： Yan <yan_z_y@yahoo.com.cn> 发送日期： 2011年9月24日, 星期六, 上午 3:03 主题: Re: [BGL] Distributed adiacency_list memory allocation
This is exactly what I was looking for, though I don't understand all of your words, but most. At least now I'm sure its not an error but normal. By the way, I'm new to PBGL, would you please tell me some forums or websites that good for beginers of PBGL.
I think this list is the best place I can suggest. I'm not sure how many people are using the Boost release but most of the people actively using the development version are people I know personally or professionally and they tend to just contact me directly. The folks on this list ought to be able to help with some of the basic parts though and I'll try to chime in when something about the internals comes up (i.e. your question about memory consumption). Cheers, Nick _______________________________________________ Boost-users mailing list [hidden email] http://lists.boost.org/mailman/listinfo.cgi/boost-users ________________________________ If you reply to this email, your message will be added to the discussion below:http://boost.2283326.n4.nabble.com/BGL-Distributed-adiacency-list-memory-all... To unsubscribe from [BGL] Distributed adiacency_list memory allocation, click here. -- View this message in context: http://boost.2283326.n4.nabble.com/BGL-Distributed-adiacency-list-memory-all... Sent from the Boost - Users mailing list archive at Nabble.com.
participants (5)
-
Cosimo Calabrese -
Igor R -
Liron Kopinsky -
Nick Edmonds -
Yan