[scope] Mr. Docs rendered reference
Hi everyone, My name is Krystian, and I'm a Staff Engineer at The C++ Alliance. I’ve been working on Mr. Docs, a clang based C++ documentation generator intended to serve as an alternative to Doxygen. Although Mr. Docs is not yet ready for release, I noticed that there was an issue with rendering the exception specification of destructors due to a bug in Boostbook. Thus, I would like to present Mr. Docs as a potential future alternative, and have generated a multi-page reference for Boost.Scope to demonstrate its capabilities: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... The output was created using the built-in Asciidoc generator and converted to HTML with asciidoctor. Since the Asciidoc generator uses handlebars templates to format the output, customizing the rendered documentation is a matter of modifying the templates. Unlike Doxygen, Mr. Docs operates directly on the clang AST. This means that support for new/obscure language features is only limited by the extent of clang’s conformance to the latest C++ standard. Any feedback is greatly appreciated. 🙂 Best, Krystian Stasiowski
On 4. Dec 2023, at 15:08, Krystian Stasiowski via Boost <boost@lists.boost.org> wrote:
My name is Krystian, and I'm a Staff Engineer at The C++ Alliance. I’ve been working on Mr. Docs, a clang based C++ documentation generator intended to serve as an alternative to Doxygen.
That's super great, I am looking forward to drop doxygen and the ugly hacks I use in Boost-Histogram to produce reasonable-looking documentation. Best regards, Hans
On 12/4/23 17:08, Krystian Stasiowski via Boost wrote:
Hi everyone,
My name is Krystian, and I'm a Staff Engineer at The C++ Alliance. I’ve been working on Mr. Docs, a clang based C++ documentation generator intended to serve as an alternative to Doxygen. Although Mr. Docs is not yet ready for release, I noticed that there was an issue with rendering the exception specification of destructors due to a bug in Boostbook. Thus, I would like to present Mr. Docs as a potential future alternative, and have generated a multi-page reference for Boost.Scope to demonstrate its capabilities: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope...
The output was created using the built-in Asciidoc generator and converted to HTML with asciidoctor. Since the Asciidoc generator uses handlebars templates to format the output, customizing the rendered documentation is a matter of modifying the templates.
Unlike Doxygen, Mr. Docs operates directly on the clang AST. This means that support for new/obscure language features is only limited by the extent of clang’s conformance to the latest C++ standard.
Any feedback is greatly appreciated. 🙂
Thanks. Looks like the tool might become a good alternative to Doxygen in the future, if it is configurable enough and integrates well with QuickBook. I have a few notes and complaints to the generated output: 1. In general, avoid wasting space on the sides of the content. Make the output span the majority of the display width. 2. https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... I would prefer if there was a separation of components by headers instead of alphabetical lists of types, functions, etc. Libraries are typically built such that a given header typically contains a group of components (i.e. types, functions, variables) that pertain to a given functionality. This association is lost when the library contents are presented as flat lists like on the page above. Flat lists may be useful as a symbol index for quickly searching the symbol you want *when you know what you want*. Such an index could be useful as a secondary addition to the main reference that should be presented per header. Also, I don't think deduction guides need to be listed on that page. 3. In class descriptions such as this: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... 3.1. The Synopsis should show the class definition, with all its public members. 3.2. The synopsis should be followed by the class Description, not by functions. 3.3. In the description of this class, the description seems to be incorrectly parsed. The list ends after the end of the first sentence of the last entry. "The condition function object operator() must..." should be displayed as a new paragraph after the list, not as part of the last entry of the list. 3.4. Member functions should show the complete function signatures, not just names. Also, it should probably separate constructors, destructor, assignment operrators, other operators and regular member functions. A simple alphabetical list doesn't make sense. 3.5. The list of functions should not contain deleted functions. If the synopsis already shows them as deleted, there's no need to show them separately in the function list without any description. In the linked page, this pertains to the "operator=" and "scope_exit" entries. 3.6. Links to headers should probably lead to a documentation generated for that header rather than GitHub. 3.7. Any links to GitHub should use permalinks rather than links to branches. 4. I hope, there is a way to detect Mr. Docs in code by testing a predefined macro. This is useful in order to cleaning up the code for documentation. For example, I don't want private stuff like types in detail namespace to be present in the generated docs. I would also like to hide SFINAE tricks. For example, this page: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... should look more like this: https://lastique.github.io/scope/libs/scope/doc/html/boost/scope/scope_exit.... Also, all of the documentation for the constructors seems to be missing. 5. In function description such as this: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... 5.1. In the Description, part of the description (the "Throws:" part) seems to have been incorrectly merged with the Note. 5.2. I don't think there's a point to the Return Value section. The return type should already be visible in the Synopsis. 5.3. Parameter descriptions are missing. 6. On this page: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... as well as these: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... swap functions appear undocumented, but they are in the source code. 7. I hope there is a way to export information from the reference documentation in an XML or QuickBook format so that it is possible to add links to specific classes and functions reference pages from the main documentation in QuickBook. (Yes, QuickBook is a requirement from me, plain AsciiDoc doesn't offer some very useful features for me to consider it a format to write in.)
Andrey Semashev wrote:
4. I hope, there is a way to detect Mr. Docs in code by testing a predefined macro. This is useful in order to cleaning up the code for documentation. For example, I don't want private stuff like types in detail namespace to be present in the generated docs. I would also like to hide SFINAE tricks. For example, this page:
https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope...
should look more like this:
https://lastique.github.io/scope/libs/scope/doc/html/boost/scope/scope_exit....
Note that the way mrdocs works is by compiling the code with Clang and extracting the information from the AST, so if you #ifdef MRDOCS and put something arbitrary in the e.g. noexcept, it won't work because the compilation will fail. There should probably exist other @means to override what appears in the noexcept (and the explicit, and the requires, and the enable_ifs...) instead of ifdef-ing them to something that doesn't compile. (Unless you want to maintain entirely separate, syntactically valid, things just for the documentation.)
On Mon, Dec 4, 2023 at 8:49 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:
Thanks. Looks like the tool might become a good alternative to Doxygen in the future, if it is configurable enough and integrates well with QuickBook.
The kind words are appreciated! I will address some points, leaving the other responses to Krystian. 1. In general, avoid wasting space on the sides of the content. Make the
output span the majority of the display width.
That's a function of the templates used by the asciidoc tool and not something that mrdocs controls. What you are seeing in this demo is the result of running asciidoc on the output of mrdocs. The mrdocs tool produces Asciidoc output. For example here is one of the output files produced by our tool: https://mrdocs.com/demos/develop/boost-scope/multi/adoc/boost/scope.adoc 2.
https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope...
I would prefer if there was a separation of components by headers instead of alphabetical lists of types, functions, etc.
For our first mrdocs deliverable we are targeting libraries like Boost.URL, Boost.JSON, Boost.Beast, et. al. which present the reference in the same style as Boost.Asio from which they were copied. I recognize that this is completely different from the other model of presenting the reference which works on an individual header file basis and also shows the full declaration of the class. Personally I prefer the Asio style over the per-header/full declaration but to each their own. A core principle of mrdocs is that it is not opinionated and will permit full customization of the output and style. Therefore our roadmap includes a set of mrdocs templates and features to be able to completely replicate the per-header/full declaration style found in other Boost documentation. 3.7. Any links to GitHub should use permalinks rather than links to
branches.
This is configured by the author in the mrdocs.yml file I don't want private stuff like types in detail namespace to be present in the generated docs. The mrdocs.yml permits namespace blacklists and whitelists with wildcards.
7. I hope there is a way to export information from the reference documentation in an XML
Currently there are three Generators: Asciidoc, HTML, and XML. The XML generator is intended to capture the entire metadata generated during AST traversal at full fidelity. We use the XML generator to test metadata output. or QuickBook format Well no we do not have a Quickbook generator, nor is there one planned. We chose Asciidoc because the Quickbook toolchain is old, unmaintained, and represents an existential risk to libraries which depend on it. Mrdocs has a plugin system using shared libraries which allows anyone to write a generator without the need to recompile mrdocs or to install and build clang/llvm locally.
so that it is possible to add links to specific classes and functions reference
pages from the main documentation in QuickBook. (Yes, QuickBook is a requirement from me, plain AsciiDoc doesn't offer some very useful features for me to consider it a format to write in.) We are positioning Antora as the premiere solution for modernization of Boost documentation. It is important to distinguish Asciidoc from Asciidoctor. Asciidoc is the markdown format while Asciidoctor is the tool which converts Asciidoc files into HTML. Antora ingests Asciidoc files and emits HTML. The Antora system has a more rich set of features than straight Asciidoc. If you have specific features from Quickbook that are not present when running Asciidoctor, I would ask that you open issues or identify these features so we can ensure they are available in the Antora solution which we are building. This is an example of Antora output. The input files are Asciidoc: https://www.preview.boost.org/doc/contributor-guide/index.html Thanks
Also, I don't think deduction guides need to be listed on that page.
The plan is for deduction guides and template specializations to be associated with the primary template and enumerated on its corresponding documentation page. 3.1. The Synopsis should show the class definition, with all its public
members.
This is a planned feature and should be present upon release.
3.3. In the description of this class, the description seems to be incorrectly parsed. The list ends after the end of the first sentence of the last entry. "The condition function object operator() must..." should be displayed as a new paragraph after the list, not as part of the last entry of the list.
Thank you for pointing this out, I will address it.
3.4. Member functions should show the complete function signatures, not just names.
Only the names are shown because overloads are grouped together and have their signatures displayed on a separate page. This keeps the list of members compact. Regardless, wouldn't this be made redundant by showing the full class definition in the synopsis?
Also, it should probably separate constructors, destructor, assignment operrators, other operators and regular member functions. A simple alphabetical list doesn't make sense.
3.5. The list of functions should not contain deleted functions. If the synopsis already shows them as deleted, there's no need to show them separately in the function list without any description. In the linked page, this pertains to the "operator=" and "scope_exit" entries.
It is impossible to have built-in support for every grouping and ordering of member functions a user may desire. The handlebars templates that ship with Mr. Docs will support a few variations, but it's ultimately up to the user to customize them to their liking. 4. I hope, there is a way to detect Mr. Docs in code by testing a
predefined macro. This is useful in order to cleaning up the code for documentation.
You can define macros in the config file. That being said, the resulting code should still be (mostly) valid, otherwise clang will not create AST nodes in more extreme cases. For example, I don't want private stuff like types in
detail namespace to be present in the generated docs.
The config allows for declarations whose qualified name matches a pattern (e.g. "boost::*::detail") to be blacklisted. By default, these filters are ignored when the "interface" of a documented declaration depends on another (e.g. class A : public detail::B), but this behavior can be disabled.
I would also like to hide SFINAE tricks.
In the future we plan to detect the enable_if idiom (and others) and convert them to equivalent constraints, but this is way down the road. For now, macros will have to suffice.
Also, all of the documentation for the constructors seems to be missing.
swap functions appear undocumented, but they are in the source code.
This is a clang bug. I will be submitting a patch to address this.
5.1. In the Description, part of the description (the "Throws:" part) seems to have been incorrectly merged with the Note.
5.3. Parameter descriptions are missing.
I'll address these issues, thanks :)
On 12/5/23 16:48, Krystian Stasiowski via Boost wrote:
Also, I don't think deduction guides need to be listed on that page.
The plan is for deduction guides and template specializations to be associated with the primary template and enumerated on its corresponding documentation page.
I think, there's not much value in listing deduction guides anywhere at all. Also, if you're going to show the deduction guides, then please show them in full, instead of just the resulting type.
3.4. Member functions should show the complete function signatures, not just names.
Only the names are shown because overloads are grouped together and have their signatures displayed on a separate page. This keeps the list of members compact. Regardless, wouldn't this be made redundant by showing the full class definition in the synopsis?
Different function overloads should be presented separately. Their documentation often differ, too. Function signature should be near the function description so that the user can see its parameters and return type, which are described in the function documentation. Having to seek between the synopsis and function description is a terrible user experience.
Also, it should probably separate constructors, destructor, assignment operrators, other operators and regular member functions. A simple alphabetical list doesn't make sense.
3.5. The list of functions should not contain deleted functions. If the synopsis already shows them as deleted, there's no need to show them separately in the function list without any description. In the linked page, this pertains to the "operator=" and "scope_exit" entries.
It is impossible to have built-in support for every grouping and ordering of member functions a user may desire. The handlebars templates that ship with Mr. Docs will support a few variations, but it's ultimately up to the user to customize them to their liking.
It doesn't need to be customizable, a fixed grouping like the one I described would be enough. Re. deleted functions, that's not about grouping at all.
4. I hope, there is a way to detect Mr. Docs in code by testing a
predefined macro. This is useful in order to cleaning up the code for documentation.
You can define macros in the config file. That being said, the resulting code should still be (mostly) valid, otherwise clang will not create AST nodes in more extreme cases.
I think the documentation tool should be more tolerant to code validity than a plain compiler. Hiding stuff from documentation for readability, where the resulting code is not a conforming C++ is common practice; even the C++ standard does this.
For example, I don't want private stuff like types in
detail namespace to be present in the generated docs.
The config allows for declarations whose qualified name matches a pattern (e.g. "boost::*::detail") to be blacklisted. By default, these filters are ignored when the "interface" of a documented declaration depends on another (e.g. class A : public detail::B), but this behavior can be disabled.
Such filters don't always work. For example, here: https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope... you can see the type `data` is used in the documentation. This is a private type (i.e. it is defined in a private section of the unique_resource class) that should not be present in the docs.
I would also like to hide SFINAE tricks.
In the future we plan to detect the enable_if idiom (and others) and convert them to equivalent constraints, but this is way down the road. For now, macros will have to suffice.
I don't think auto-detection is needed. Just make it easy to hide stuff from docs and make the tool more tolerant to non-conforming C++, that would be enough, IMHO.
participants (5)
-
Andrey Semashev -
Hans Dembinski -
Krystian Stasiowski -
Peter Dimov -
Vinnie Falco