
Hi there, my plan for modularizing and modernizing Boost was roughly this: * Phase 0 - remove dead weight by bumping compiler feature requirements * Phase 1 - move some files around so that the modularized repos form a mostly directed graph * Phase 2 - Form some kind of 'boost core library' or 'boost feature normalization library' from the guts of existing libraries like type_traits, static_assert, config mpl and utilities. * Phase 3 - Try to port the mpl to variadic templates so that the dependency on Boost.PP is not needed when variadic templates are available. I recommend completing up to phase 2 before migrating to git, as otherwise files will have to be deleted from one repo, and added to another without history. Phase 0 is aborted, so let's look at phase 1. [Note: This should go without saying, but my experience on this list tells me it needs to be said: * I do not propose making many commits with the commit message 'migrate' * The below is scripted for your understanding only. Scripting gives an exactness which prose does not, and allows reproducibility. Don't take it too literally. * Of course, forwarding headers should be understood to be left behind when a file is moved, where possible. * Don't cry too loudly about where I'm moving files to. See Phase 2 above, and understand that this is partly only an experiment to see how modularization can be done. EndNote] If the dependencies between repositories are analysed, the result is this graph: http://steveire.com/boost/graph_all.dot http://steveire.com/boost/graph_all_small.png There are 104 nodes and 1159 edges. Each edge is a real dependency which exists between repos, and which will exist between boost modularized packages. They should not be considered optional. If we remove any nodes which are not strongly connected, we are left with a new graph of nodes which are all strongly connected: http://steveire.com/boost/graph_strong.dot http://steveire.com/boost/graph_strong_small.png There are 68 nodes and 675 edges. Move enable_if from utility to type_traits: git submodule foreach 'git grep -l -P utility/enable_if.hpp -- include/boost | xargs sed -i "s|utility/enable_if.hpp|type_traits/enable_if.hpp|" || echo' mv libs/utility/include/boost/utility/enable_if.hpp libs/type_traits/include/boost/type_traits git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 670 edges and 68 nodes. Next, move boost/detail/workaround.hpp to the config library/repo: git submodule foreach 'git grep -l -P boost/detail/workaround.hpp -- include/boost | xargs sed -i "s|boost/detail/workaround.hpp|boost/config/workaround.hpp|" || echo' mv libs/detail/include/boost/detail/workaround.hpp libs/config/include/boost/config/ git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 661 edges and 68 nodes. Next, move boost/limits.hpp to the config library/repo: git submodule foreach 'git grep -l -P boost/limits.hpp -- include/boost | xargs sed -i "s|boost/limits.hpp|boost/config/limits.hpp|" || echo' mv libs/detail/include/boost/limits.hpp libs/config/include/boost/ git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 653 edges and 68 nodes. Next, move boost/detail/iterator.hpp and boost/iterator/iterator_traits.hpp to the type_traits library/repo: git submodule foreach 'git grep -l -P boost/detail/iterator.hpp -- include/boost | xargs sed -i "s|boost/detail/iterator.hpp|boost/type_traits/detail/iterator.hpp|" || echo' git submodule foreach 'git grep -l -P boost/iterator/iterator_traits.hpp -- include/boost | xargs sed -i "s|boost/iterator/iterator_traits.hpp|boost/type_traits/iterator_traits.hpp|" || echo' mv libs/detail/include/boost/detail/iterator.hpp libs/type_traits/include/boost/type_traits/detail/ mv libs/iterator/include/boost/iterator/iterator_traits.hpp libs/type_traits/include/boost/type_traits git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 650 edges and 68 nodes. Next, move boost/iterator.hpp from iterator to the utility library/repo: mv libs/iterator/include/boost/iterator.hpp libs/utility/include/boost/ git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 649 edges and 68 nodes. utility no longer depends on iterator. Move boost/version.hpp to config: mv libs/detail/include/boost/version.hpp libs/config/include/boost/ git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 512 edges and 64 nodes. The config, integer, io and static_assert libraries are no longer part of the mesh Next, move boost/pointee.hpp from iterator to detail: mv ./libs/iterator/include/boost/pointee.hpp libs/detail/include/boost/ git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' Move exception/detail/attribute_noreturn.hpp into config: git submodule foreach 'git grep -l -P boost/exception/detail/attribute_noreturn.hpp -- include/boost | xargs sed -i "s|boost/exception/detail/attribute_noreturn.hpp|boost/config/attribute_noreturn.hpp|" || echo' mv libs/exception/include/boost/exception/detail/attribute_noreturn.hpp libs/config/include/boost/config/ git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' Move boost/throw_exception.hpp and exception.hpp into utility: mv libs/exception/include/boost/throw_exception.hpp libs/utility/include/boost mv libs/exception/include/boost/exception/exception.hpp libs/utility/include/boost/utility git submodule foreach 'git grep -l -P boost/exception/exception.hpp -- include/boost | xargs sed -i "s|boost/exception/exception.hpp|boost/utility/exception.hpp|" || echo' git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' There are now 485 edges and 64 nodes. Several libraries form an inner mesh: $ grep -P "^\s*(type_traits|mpl|detail|utility|smart_ptr|typeof)" output.dot detail->mpl detail->type_traits detail->smart_ptr detail->utility utility->mpl utility->type_traits utility->detail type_traits->detail type_traits->mpl type_traits->typeof type_traits->utility smart_ptr->type_traits smart_ptr->detail smart_ptr->utility mpl->type_traits mpl->detail mpl->utility typeof->mpl typeof->type_traits If we treat them as one element for now, we get a new graph from the rest http://steveire.com/boost/graph_with_corelib.dot http://steveire.com/boost/graph_with_corelib.png There are now 138 edges and 39 nodes Move parts of {vector_,}property_map into graph_parallel: sed '1,/BOOST_GRAPH_USE_MPI/d;/BOOST_GRAPH_USE_MPI/,$d' libs/property_map/include/boost/property_map/property_map.hpp > tmp.hpp sed -i '/BOOST_GRAPH_USE_MPI/,/BOOST_GRAPH_USE_MPI/d' libs/property_map/include/boost/property_map/property_map.hpp sed '1,/BOOST_GRAPH_USE_MPI/d;/BOOST_GRAPH_USE_MPI/,$d' libs/property_map/include/boost/property_map/vector_property_map.hpp >> tmp.hpp sed -i '/BOOST_GRAPH_USE_MPI/,/BOOST_GRAPH_USE_MPI/d' libs/property_map/include/boost/property_map/vector_property_map.hpp sed -i '/#undef PBGL_DISTRIB_PMAP/r tmp.hpp' libs/property_map/include/boost/property_map/parallel/distributed_property_map.hpp rm tmp.hpp git submodule foreach 'git grep -l -P boost/property_map/parallel/distributed_property_map.hpp -- include/boost | xargs sed -i "s|boost/property_map/parallel/distributed_property_map.hpp|boost/graph/parallel/distributed_property_map.hpp|" || echo' git submodule foreach 'git grep -l -P boost/property_map/parallel/impl/distributed_property_map.ipp -- include/boost | xargs sed -i "s|boost/property_map/parallel/impl/distributed_property_map.hpp|boost/graph/parallel/detail/distributed_property_map.hpp|" || echo' mv libs/property_map/include/boost/property_map/parallel/distributed_property_map.hpp libs/graph_parallel/include/boost/graph/parallel mv libs/property_map/include/boost/property_map/parallel/impl/distributed_property_map.ipp libs/graph_parallel/include/boost/graph/parallel/detail git submodule foreach 'git add . || echo' git submodule foreach 'git commit -am migrate || echo' http://steveire.com/boost/graph_after_parallel.dot http://steveire.com/boost/graph_after_parallel.png There are now 95 edges and 33 nodes The remaining problematic edges are: conversion->range conversion->math range->algorithm math->multiprecision concept_check->parameter Assuming they can be broken by moving some files aroung (I believe they can be), we end up with a small graph of strongly connected components: http://steveire.com/boost/graph_after_remaining.dot http://steveire.com/boost/graph_after_remaining.png There are now 18 edges and 11 nodes. Looking at the entire graph again, we get this: http://steveire.com/boost/graph_final.dot http://steveire.com/boost/graph_final_small.png Obviously, this is not perfect, but it is a beginning, and it is mostly now a directed graph. Any comments? Thanks, Steve.