newbie BGL problem: core dump due to graph size
Hi all. I am calculating the connected components of af graph with a c++ code written with BGL. My graph is huge (4E+05 vertexes, 10^6 ~ 10^8 edges). The code uses the connected_components algorithm (or, in a different version, directly the depth_first_visit one) and works well up to a "reasonable" number of edges in the graph. Over this threshold it core dumps. I have the same problem both on SPARC Solaris 9 g++3.2.2 and on Intel Linux 2.4 g++3.1 . I know that it should not be a problem of the RAM memory available on the system (or addressable form a single istance). I am able to use up to 6-7 GBs of RAM, and it tipically core dumps at 2.5-5 GBs. I suspect that I'm violating some intrinsic size limit in the BGL adjacency_list class or similar. Are there such limits, and can they be extended? Any clue would be really appreciated. Thanks in advance! Duccio Medini
hi Duccio, I'm BGL true newbie, too.
I suspect that I'm violating some intrinsic size limit in the BGL adjacency_list class or similar. Are there such limits, and can they be extended? Any clue would be really appreciated.
Sorry, I don't know how large the BGL's limitation is. But I can show you my experience. I've tried to treat a graph for a few weeks. succeed: 5E+5 vertices and 5E+6 edges (required 500MB memory) failure: 1E+6 vertices and 1E+7 edges (runs short of the memory and incredibly slow, so I stopped it manually.I think it will succeed, if I have more large memory.) The environment I tried is.. windowsXP ram 512MB(paging size:2GB) vc++6.0 with STL it includes. I wonder these limitation also is related to size of property map. otsusan
On Thu, 2003-12-11 at 18:03, duccio.medini@tin.it wrote:
Hi all.
<snip>
I suspect that I'm violating some intrinsic size limit in the BGL adjacency_list class or similar. Are there such limits, and can they be extended? Any clue would be really appreciated.
What does the backtrace say? -- Tarjei
Hi Duccio, The only intrinsic size limitations, say for adjacency_list, that I am aware of are in the range of numbers representable by an "int". As for the algorithms, I am not aware of any size limitations. Of course, there's always the possibility of a bug in there somewhere... Cheers, Jeremy On Thu, 11 Dec 2003 duccio.medini@tin.it wrote: duccio> Hi all. duccio> duccio> I am calculating the connected components of af graph with a c++ code duccio> written with BGL. duccio> My graph is huge (4E+05 vertexes, 10^6 ~ 10^8 edges). duccio> The code uses the connected_components algorithm (or, in a different duccio> version, directly the depth_first_visit one) and works well up to a duccio> "reasonable" number of edges in the graph. duccio> Over this threshold it core dumps. duccio> I have the same problem both on SPARC Solaris 9 g++3.2.2 and on Intel Linux duccio> 2.4 g++3.1 . duccio> I know that it should not be a problem of the RAM memory available on the duccio> system (or addressable form a single istance). I am able to use up to 6-7 duccio> GBs of RAM, and it tipically core dumps at 2.5-5 GBs. duccio> duccio> I suspect that I'm violating some intrinsic size limit in the BGL duccio> adjacency_list class or similar. duccio> Are there such limits, and can they be extended? duccio> Any clue would be really appreciated. duccio> duccio> Thanks in advance! duccio> duccio> Duccio Medini ---------------------------------------------------------------------- Jeremy Siek http://php.indiana.edu/~jsiek/ Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu C++ Booster (http://www.boost.org) office phone: (812) 856-1820 ----------------------------------------------------------------------
-- Messaggio originale -- Date: Fri, 12 Dec 2003 09:24:13 -0500 (EST) From: Jeremy Siek
To: Boost Users mailing list Subject: Re: [Boost-users] newbie BGL problem: core dump due to graph size Reply-To: Boost Users mailing list Hi Duccio,
The only intrinsic size limitations, say for adjacency_list, that I am aware of are in the range of numbers representable by an "int". As for
Thx a lot for your help. Giving a look in the CVS I've found a "Nonrecursive version" of the depth_first_search.hpp . It should fix a problem in the Stack generated in the boost 1.30.2 version (that I was using). Since substituting the old implementation with this new file perfectly fixed my problem (the code now uses less RAM and is perfectly stable also with my largest network), I conclude that I was experiencing exactly that Stack bug in the routine. Thanks again, and especially thanks A LOT to Bruce Barr who wrote the new implementation!! Duccio the
algorithms, I am not aware of any size limitations. Of course, there's always the possibility of a bug in there somewhere...
Cheers, Jeremy
On Thu, 11 Dec 2003 duccio.medini@tin.it wrote: duccio> Hi all. duccio> duccio> I am calculating the connected components of af graph with a c++ code duccio> written with BGL. duccio> My graph is huge (4E+05 vertexes, 10^6 ~ 10^8 edges). duccio> The code uses the connected_components algorithm (or, in a different duccio> version, directly the depth_first_visit one) and works well up to a duccio> "reasonable" number of edges in the graph. duccio> Over this threshold it core dumps. duccio> I have the same problem both on SPARC Solaris 9 g++3.2.2 and on Intel Linux duccio> 2.4 g++3.1 . duccio> I know that it should not be a problem of the RAM memory available on the duccio> system (or addressable form a single istance). I am able to use up to 6-7 duccio> GBs of RAM, and it tipically core dumps at 2.5-5 GBs. duccio> duccio> I suspect that I'm violating some intrinsic size limit in the BGL duccio> adjacency_list class or similar. duccio> Are there such limits, and can they be extended? duccio> Any clue would be really appreciated. duccio> duccio> Thanks in advance! duccio> duccio> Duccio Medini
---------------------------------------------------------------------- Jeremy Siek http://php.indiana.edu/~jsiek/ Ph.D. Student, Indiana Univ. B'ton email: jsiek@osl.iu.edu C++ Booster (http://www.boost.org) office phone: (812) 856-1820 ----------------------------------------------------------------------
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (4)
-
duccio.medini@tin.it
-
Jeremy Siek
-
otsusan
-
Tarjei Knapstad