Storing a Huge Graph
Hi, Regarding the Boost Graph Library, Will I be able to convert a huge text file to a graph with millions of nodes and edges? Can I store the graph somewhere in the hard disk and then upload it whenever I test my code (i.e. I don't have to convert the text file into a graph each time I run my program).
Boost has the serialization library. While I dont know, I bet it can
store your graph, have a look at it!
Best,
Dee
On Sat, Oct 15, 2011 at 14:28, Nouf M.
Hi, Regarding the Boost Graph Library, Will I be able to convert a huge text file to a graph with millions of nodes and edges? Can I store the graph somewhere in the hard disk and then upload it whenever I test my code (i.e. I don't have to convert the text file into a graph each time I run my program). _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Hi,
Hi, Regarding the Boost Graph Library, Will I be able to convert a huge text file to a graph with millions of nodes and edges? Can I store the graph somewhere in the hard disk and then upload it whenever I test my code (i.e. I don't have to convert the text file into a graph each time I run my program). You should first decide whether you want to use some kind of serialization (-> boost serialization) or memory mapped files.
Serialization: http://www.boost.org/doc/libs/1_47_0/libs/graph/doc/adjacency_list.html#seri... Memory mapped files: I haven't used memory mapped files with boost graph yet. Hmm, just a thought - you can customize boost graph storage http://www.boost.org/doc/libs/1_47_0/libs/graph/doc/using_adjacency_list.htm... Good Luck Amanjit Gill
On Oct 15, 2011, at 2:28 AM, Nouf M. wrote:
Hi, Regarding the Boost Graph Library, Will I be able to convert a huge text file to a graph with millions of nodes and edges? Can I store the graph somewhere in the hard disk and then upload it whenever I test my code (i.e. I don't have to convert the text file into a graph each time I run my program).
"millions" of nodes and edges really isn't all that large (unless you're talking 100's of millions), if your graph is in a halfway sensible format you can probably get away with using one of the existing graph readers or slightly modifying one of them to read your data. If your data is ordered in some way there are CSR constructors that will respect the ordering, and in some cases construct the graph in-place with regard to the memory utilized by the raw graph data once you load it from disk. The adjacency list class is less compact/efficient in this respect. Using Boost.Serialization to convert your text file to a binary format may improve your load times by making your data more compact on disk however. Cheers, Nick
participants (4)
-
Amanjit Gill
-
Diederick C. Niehorster
-
Nick Edmonds
-
Nouf M.