RFC: Automatic indexing of quickbook/boostbook/docbook docs.
I've just committed a new tool to the sandbox under tools/auto_index which helps automate the indexing of quickbook/boostbook/docbook documentation. Obviously the tool is biased toward C/C++ code documentation, as among other things it can scan C/C++ headers looks for definitions of classes/typedefs/macros/functions which should be indexed. The actual index can be produced either by the final XSL transform, or by auto_index itself (ie using regular docbook markup for the index), the former is better for PDF/print docs as it allows page numbers to be inserted, the latter works better for html docs, and is also faster (thankfully - the XSLT index generation code is glacially slow). There are some examples taken from type_traits and Boost.Math here: http://svn.boost.org/svn/boost/sandbox/tools/auto_index/example/readme.html Some rather basic documentation for the tool itself is here: http://svn.boost.org/svn/boost/sandbox/tools/auto_index/doc/html/index.html note that currently there is no Boost.Build integration - I really need some help with that - so the tool is much harder to use than it should be. Any comments, especially on the usability of the indexes generated would be much appreciated! Cheers, John.
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of John Maddock Sent: 28 November 2008 18:01 To: Boost Subject: [boost] RFC: Automatic indexing of quickbook/boostbook/docbook docs.
I've just committed a new tool to the sandbox under tools/auto_index which helps automate the indexing of quickbook/boostbook/docbook documentation.
Obviously the tool is biased toward C/C++ code documentation, as among other things it can scan C/C++ headers looks for definitions of classes/typedefs/macros/functions which should be indexed. The actual index can be produced either by the final XSL transform, or by auto_index itself (ie using regular docbook markup for the index), the former is better for PDF/print docs as it allows page numbers to be inserted, the latter works better for html docs, and is also faster (thankfully - the XSLT index generation code is glacially slow).
There are some examples taken from type_traits and Boost.Math here:
http://svn.boost.org/svn/boost/sandbox/tools/auto_index/example/readme.html
Some rather basic documentation for the tool itself is here:
http://svn.boost.org/svn/boost/sandbox/tools/auto_index/doc/html/index.html
note that currently there is no Boost.Build integration - I really need some help with that - so the tool is much harder to use than it should be.
Any comments, especially on the usability of the indexes generated would be much appreciated!
Looks complicated - before I look at the docs - but the results excellent (though I did find some index items missing and wanted to add them - will try). However, this is what we need to provide a step-change in Boost docs quality. Brilliant! Paul
AMDG John Maddock wrote:
Some rather basic documentation for the tool itself is here: http://svn.boost.org/svn/boost/sandbox/tools/auto_index/doc/html/index.html note that currently there is no Boost.Build integration - I really need some help with that - so the tool is much harder to use than it should be.
I've worked out how to insert an extra tool in the middle of the boostbook toolchain. See attached. In Christ, Steven Watanabe import feature ; import generators ; import "class" ; feature.feature modify-docbook : off "on" ; class modify-docbook-generator : generator { rule run ( project name ? : property-set : sources * ) { if [ $(property-set).get <modify-docbook> ] = "on" { return [ generator.run $(project) $(name) : $(property-set) : $(sources) ] ; } } } generators.register [ class.new modify-docbook-generator modified-docbook.modify-docbook : DOCBOOK : DOCBOOK(%.modified) ] ; generators.override modified-docbook.modify-docbook : boostbook.boostbook-to-docbook ; actions modify-docbook { copy $(>) $(<) }
Steven Watanabe wrote:
AMDG
John Maddock wrote:
Some rather basic documentation for the tool itself is here: http://svn.boost.org/svn/boost/sandbox/tools/auto_index/doc/html/index.html note that currently there is no Boost.Build integration - I really need some help with that - so the tool is much harder to use than it should be.
I've worked out how to insert an extra tool in the middle of the boostbook toolchain. See attached.
That would be way cool, but it doesn't work for me: I cut and pasted your code into the type_traits Jamfile.v2, and without "<modify-docbook>on" I see: c:/data/boost/boost/trunk/tools/build/v2/tools\boostbook.jam:359: in object(boostbook-generator)@73.run warning: rulename $(base-target).depends expands to empty string c:/data/boost/boost/trunk/tools/build/v2/build\generators.jam:994: in try-one-generator-really c:/data/boost/boost/trunk/tools/build/v2/build\generators.jam:1056: in try-one-generator c:/data/boost/boost/trunk/tools/build/v2/build\generators.jam:1268: in construct-really c:/data/boost/boost/trunk/tools/build/v2/build\generators.jam:1340: in generators.construct c:/data/boost/boost/trunk/tools/build/v2/build\targets.jam:1431: in construct c:/data/boost/boost/trunk/tools/build/v2/build\targets.jam:1244: in object(typed-target)@428.generate c:/data/boost/boost/trunk/tools/build/v2/build\targets.jam:767: in generate-really c:/data/boost/boost/trunk/tools/build/v2/build\targets.jam:739: in object(main-target)@435.generate c:/data/boost/boost/trunk/tools/build/v2/build\targets.jam:257: in object(project-target)@424.generate c:/data/boost/boost/trunk/tools/build/v2\build-system.jam:682: in load c:\data\boost\boost\trunk\tools\build\v2/kernel\modules.jam:283: in import c:\data\boost\boost\trunk\tools\build\v2\kernel\bootstrap.jam:134: in boost-build And with "<modify-docbook>on" I see: c:/data/boost/boost/trunk/tools/build/v2/kernel\modules.jam:103: in modules.call-in rule modified-docbook.modify-docbook unknown in module modified-docbook. c:/data/boost/boost/trunk/tools/build/v2/util\indirect.jam:90: in indirect.call c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:758: in object(action)@467.actualize c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:301: in actualize-action c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:489: in actualize-no-scanner c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:130: in object(file-target)@468.actualize c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:779: in actualize-source-type c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:800: in actualize-sources c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:740: in object(action)@472.actualize c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:301: in actualize-action c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:489: in actualize-no-scanner c:/data/boost/boost/trunk/tools/build/v2/build\virtual-target.jam:130: in object(file-target)@473.actualize What am I doing wrong? Cheers, John.
John Maddock wrote:
Steven Watanabe wrote:
AMDG
John Maddock wrote:
Some rather basic documentation for the tool itself is here: http://svn.boost.org/svn/boost/sandbox/tools/auto_index/doc/html/index.html note that currently there is no Boost.Build integration - I really need some help with that - so the tool is much harder to use than it should be.
I've worked out how to insert an extra tool in the middle of the boostbook toolchain. See attached.
Thanks for making this!
That would be way cool, but it doesn't work for me:
I cut and pasted your code into the type_traits Jamfile.v2,
...
What am I doing wrong?
Pasting the code :-) Please save Steven's attachment into a file, tools/build/v2/tools/modified-docbook.jam and add import modified-docbook ; in your jamfile. You need to this it this way because the names of actions are qualified by the module name, or if you put this code into a different file, you need to change this code.
And with "<modify-docbook>on" I see:
c:/data/boost/boost/trunk/tools/build/v2/kernel\modules.jam:103: in modules.call-in rule modified-docbook.modify-docbook unknown in module modified-docbook.
This should be gone if you save into right file.
and without "<modify-docbook>on" I see: > c:/data/boost/boost/trunk/tools/build/v2/tools\boostbook.jam:359: in object(boostbook-generator)@73.run warning: rulename $(base-target).depends expands to empty string
Please try the attached revision. Steven, your original code did nothing when modify-docbook=off -- however, Boost.Build first processes generator overrides, and then runs generator. The standard docbook generator gets overriden, and if your custom generator returns nothing, no attempt to use the standard generator is made. The attached version explicitly tries to construct DOCBOOK again. Another alternative is to add '<modify-docbook>on' to generator requirements, so it's not even considered if not needed. Thanks, Volodya
Vladimir Prus wrote:
I cut and pasted your code into the type_traits Jamfile.v2, ... What am I doing wrong?
Pasting the code :-) Please save Steven's attachment into a file, tools/build/v2/tools/modified-docbook.jam and add
import modified-docbook ;
in your jamfile. You need to this it this way because the names of actions are qualified by the module name, or if you put this code into a different file, you need to change this code.
Aha!, with your version, I have things almost working now, just need to figure out how to get the tool automatically built like quickbook is... Thanks, John.
John Maddock wrote:
Vladimir Prus wrote:
I cut and pasted your code into the type_traits Jamfile.v2, ... What am I doing wrong?
Pasting the code :-) Please save Steven's attachment into a file, tools/build/v2/tools/modified-docbook.jam and add
import modified-docbook ;
in your jamfile. You need to this it this way because the names of actions are qualified by the module name, or if you put this code into a different file, you need to change this code.
Aha!, with your version, I have things almost working now, just need to figure out how to get the tool automatically built like quickbook is...
Do you have sources checked in already? I'm thinking the the easiest approach would be to add something like: <docbook-indexer>tools/indexer//indexer to Jamroot (and it will be hooked in more-or-less automatically). And if you want to use prebuilt version, you would have to modify Jamroot to use the path to the prebuilt binary. This is slightly less fancy than setting things in user-config.jam, but easier to implement. - Volodya
Vladimir Prus wrote:
Do you have sources checked in already? I'm thinking the the easiest approach would be to add something like:
<docbook-indexer>tools/indexer//indexer
to Jamroot (and it will be hooked in more-or-less automatically). And if you want to use prebuilt version, you would have to modify Jamroot to use the path to the prebuilt binary. This is slightly less fancy than setting things in user-config.jam, but easier to implement.
Well... I think I almost have a version working that works the same as quickbook - built from source if you don't put anything in user-config.jam. Just some strange assertion failing when the tool is run from bjam that doesn't happen when I run from the IDE :-( Still investigating yours, John.
AMDG Vladimir Prus wrote:
Do you have sources checked in already? I'm thinking the the easiest approach would be to add something like:
<docbook-indexer>tools/indexer//indexer
to Jamroot (and it will be hooked in more-or-less automatically).
Doesn't this have the effect of making /everything/ in the project depend on tools/indexer//indexer? In Christ, Steven Watanabe
Vladimir Prus wrote:
Please try the attached revision.
Many thanks to Steven and Vladimir for helping me get Boost.Build integration sorted out. I've committed a bunch of fixes for the tool, along with full Boost.Build integration now (setup process is the same as quickbook), anyone who checked out an earlier version should really update. Enjoy... John.
John Maddock skrev:
I've just committed a new tool to the sandbox under tools/auto_index which helps automate the indexing of quickbook/boostbook/docbook documentation. ... Any comments, especially on the usability of the indexes generated would be much appreciated!
I think it looks good. I think it might be useful to have the header or headers that one should include listed also (that is, the header of definition), albeit it might not be completely possible in all cases. I just want talk about something related. Finding information in a fast manner is very important IMO. I PHP, one could search the online documentation very fast, and often one was presented with not just a list of alternatives, but with the exact documentation you were looking for directly. For example, try and go to http://www.php.net/ and type "strstr" in the search box. Something similar to this could be very convenient for Boost. -Thorsten
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Thorsten Ottosen Sent: 30 November 2008 15:31 To: boost@lists.boost.org Subject: Re: [boost] RFC: Automatic indexing of quickbook/boostbook/docbook docs.
I just want talk about something related. Finding information in a fast manner is very important IMO.
I PHP, one could search the online documentation very fast, and often one was presented with not just a list of alternatives, but with the exact documentation you were looking for directly.
For example, try and go to
and type "strstr" in the search box.
Something similar to this could be very convenient for Boost.
Indeed - excellent. I wonder if this will work using Google's indexing of the Boost site especially once we have documents in pdf. 'Index' & 'BOOST_MACRO' should take you to the index with the places where it is mentioned? Paul --- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
participants (5)
-
John Maddock -
Paul A. Bristow -
Steven Watanabe -
Thorsten Ottosen -
Vladimir Prus