I don't know of the top of my head, but the Boost.Core documentation is currently built as part of the combined documentation which means that it can't use its own settings in the boostbook to html translation stage. We'll have to change that if you want to customise these things.
I don't insist on customising them - it's just that the local Jamfile.v2 already had the options and I wanted to change the ToC appearance, but couldn't figure out how. It wouldn't matter if the docs are part of the combined documentation.
Is the combined documentation for develop or master currently live somewhere?
If section ToCs can't have more than one level, I think that it'd be better to flatten the structure and not nest the sections the way I've structured them.
I believe toc.max.depth is the param you want, for example like this: http://www.boost.org/doc/libs/1_55_0/libs/math/doc/html/dist.html See http://docbook.sourceforge.net/release/xsl/current/doc/html/toc_index.html for all the TOC options. HTH, John.