Pretty printer module

older
[release] Master branch closes for...

Дмитрий Архипов

11 Sep 2024 11 Sep '24

7:46 p.m.

Recently Niall Douglas posted about his idea to embed GDB pretty printers into binaries using a header. At the time I was thinking about a good way to deploy pretty printers for JSON that facilitates their autoloading. After Niall's post I experimented a little bit more, and I came to the conclusion that indeed nothing beats embedding via a C++ header in terms of convenience. At the same time I was also thinking about testing that pretty printers do in fact work. Given that pretty printers (usually) are implemented as a Python module, it initially looked like one could just run a Python program and import GDB Python modules. But actually you cannot. It turns out, though, you can run a Python script with GDB as the interpreter. After several iterations I arrived at this setup: https://github.com/boostorg/json/blob/develop/test/printers.cpp. A script converts this file into a Python script that uses GDB to break on certain lines and check if a particular expression produces a particular output when printed. While doing all this I thought that most of this work can be useful to other libraries. So, I moved the reusable parts into a separate directory (https://github.com/boostorg/json/tree/develop/pretty_printers) and added both a b2 module, and a CMake module. For B2. Importing: https://github.com/boostorg/json/blob/develop/build.jam#L8-L9 https://github.com/boostorg/json/blob/develop/build/Jamfile#L13 (Re)generating the header: https://github.com/boostorg/json/blob/develop/build/Jamfile#L88-L95 Testing: https://github.com/boostorg/json/blob/develop/test/Jamfile#L96-L101 For CMake: Importing: https://github.com/boostorg/json/blob/develop/CMakeLists.txt#L54-L55 (Re)generating the header: https://github.com/boostorg/json/blob/develop/CMakeLists.txt#L56-L63 Testing: https://github.com/boostorg/json/blob/develop/test/CMakeLists.txt#L80-L88 My questions to the community: do you find this useful? Would you use it in your Boost or non-Boost library? If you do, the module should be moved to tools rather than stay in libs/json.

Show replies by date

Seth

11 Sep 11 Sep

7:55 p.m.

On Wed, Sep 11, 2024, at 9:46 PM, Дмитрий Архипов via Boost wrote:

...

While doing all this I thought that most of this work can be useful to other libraries. So, I moved the reusable parts into a separate directory (https://github.com/boostorg/json/tree/develop/pretty_printers) and added both a b2 module, and a CMake module.

As a boost user, thank you (and Niall) for spearheading this. It's about time that debuggability is no longer arcane art, using forbidden wisdom scattered across a half-dozen haphazardly maintained websites/repositories (try comparative debugging builds with different boost versions for even more fun). This is probably going to be about as transformative as the CMake support, once it picks up. Just my round of applause to cheer on anyone who might be on the fence. Seth

Julien Blanc

12 Sep 12 Sep

1:57 p.m.

11 septembre 2024 à 21:46 "Дмитрий Архипов via Boost" a écrit:

...

Recently Niall Douglas posted about his idea to embed GDB pretty printers into binaries using a header. At the time I was thinking about a good way to deploy pretty printers for JSON that facilitates their autoloading. After Niall's post I experimented a little bit more, and I came to the conclusion that indeed nothing beats embedding via a C++ header in terms of convenience.

I mostly agree with that statement, with a few caveats though. The main issue i've seen is that with some linkers, the .debug_gdb_scripts section is duplicated several times when the file is included several (although not exactly once per inclusion…), and this causes issues later with gdb (types registering multiple times, etc.). While i observed these only with compilers for embedded targets and pretty old toolchains, we finally used a different approach, based on the inclusion of a single cpp file in the project. This is not feasible for boost, however, i think the following can be done quite easily: * the macro to opt out from gdb pretty printing already exists, move its usage from inside gdb_printers.hpp to config.hpp * provide a public (ie, not in details) json/debug_printers.hpp file, which unconditionnaly includes gdb_printers.hpp, to opt-in again. This file shall not be included by any library code. This allows client code to circumvent the limitations, and is fully transparent for users not affected by the duplicated python code issue. I can file a PR if you are ok with this approach.

...

My questions to the community: do you find this useful? Would you use it in your Boost or non-Boost library? If you do, the module should be moved to tools rather than stay in libs/json.

Testing printers is indeed part of the job (we've had our bunch of broken printers where i work), so yes, this is definitely useful. Thanks for sharing this. Regards, Julien

Дмитрий Архипов

16 Sep 16 Sep

4:25 p.m.

чт, 12 сент. 2024 г. в 17:29, Julien Blanc via Boost :

...

i think the following can be done quite easily:

* the macro to opt out from gdb pretty printing already exists, move its usage from inside gdb_printers.hpp to config.hpp * provide a public (ie, not in details) json/debug_printers.hpp file, which unconditionally includes gdb_printers.hpp, to opt-in again. This file shall not be included by any library code.

This allows client code to circumvent the limitations, and is fully transparent for users not affected by the duplicated python code issue.

I can file a PR if you are ok with this approach.

Sure, sounds like a good idea.

Julien Blanc

20 Sep 20 Sep

5:50 a.m.

Le lundi 16 septembre 2024 à 19:25 +0300, Дмитрий Архипов via Boost a écrit :

...

чт, 12 сент. 2024 г. в 17:29, Julien Blanc via Boost :

...
I can file a PR if you are ok with this approach.

Sure, sounds like a good idea.

Done: https://github.com/boostorg/json/pull/1043 I noticed that Boost.Outcome, which also provides pretty printers for gdb, uses different macros (it does not seem to check for BOOST_ALL_NO_EMBEDDED_GDB_SCRIPTS ). It seems to be in use by the following libraries : * json * unordered * interprocess Ideally, we need some consistency here accross the libraries and/or some guidelines for maintainers on which pattern to follow. Regards, Julien

Braden Ganetsky

28 Sep 28 Sep

11:09 p.m.

On Fri, Sep 20, 2024, 12:50 AM Julien Blanc via Boost wrote: I noticed that Boost.Outcome, which also provides pretty printers for gdb, uses different macros (it does not seem to check for BOOST_ALL_NO_EMBEDDED_GDB_SCRIPTS ). It seems to be in use by the following libraries : * json * unordered * interprocess Ideally, we need some consistency here accross the libraries and/or some guidelines for maintainers on which pattern to follow. The name `BOOST_ALL_NO_EMBEDDED_GDB_SCRIPTS` came from a conversation between Niall and myself, with assent from other members of the community including Dmitry. (I can't remember whether here or on the Slack.) I put that macro into use in Unordered and Interprocess, so I assume Niall will ultimately add this same option to Outcome. As far as I remember, this was the decision. The hope was, eventually this macro will be used in more libraries, as they create more pretty printers for their types. Then we can disable the script embedding in all libraries all at once. Thanks, Braden

Julien Blanc

30 Sep 30 Sep

12:57 p.m.

Le samedi 28 septembre 2024 à 18:09 -0500, Braden Ganetsky via Boost a écrit :

...

The name `BOOST_ALL_NO_EMBEDDED_GDB_SCRIPTS` came from a conversation between Niall and myself, with assent from other members of the community including Dmitry. (I can't remember whether here or on the Slack.)

I don't think it was here, because i would probably remember. But i think it's a good idea.

...

I put that macro into use in Unordered and Interprocess, so I assume Niall will ultimately add this same option to Outcome. As far as I remember, this was the decision. The hope was, eventually this macro will be used in more libraries, as they create more pretty printers for their types. Then we can disable the script embedding in all libraries all at once.

Thanks. This is indeed the kind of consistency i was hoping for. Regards, Julien

Дмитрий Архипов

4 Oct 4 Oct

10:30 a.m.

ср, 11 сент. 2024 г. в 22:46, Дмитрий Архипов :

...

My questions to the community: do you find this useful? Would you use it in your Boost or non-Boost library? If you do, the module should be moved to tools rather than stay in libs/json.

I went ahead and created a project that contains the described utilities. You can find it here: https://github.com/cppalliance/pretty_printers. Its readme currently serves as the documentation. The project contains two Python scripts and b2 and CMake modules to simplify using those scripts in a project. I want to make it a Boost tool, that is to put it inside boost/tools. So, I was advised to ask for a review on the list. I know we don't usually have reviews for tools, but I guess we should start sooner or later.

René Ferdinand Rivera Morell

1:43 p.m.

On Fri, Oct 4, 2024 at 5:31 AM Дмитрий Архипов via Boost wrote:

...

ср, 11 сент. 2024 г. в 22:46, Дмитрий Архипов :

...
My questions to the community: do you find this useful? Would you use it in your Boost or non-Boost library? If you do, the module should be moved to tools rather than stay in libs/json.

I went ahead and created a project that contains the described utilities. You can find it here: https://github.com/cppalliance/pretty_printers. Its readme currently serves as the documentation. The project contains two Python scripts and b2 and CMake modules to simplify using those scripts in a project.

I want to make it a Boost tool, that is to put it inside boost/tools. So, I was advised to ask for a review on the list. I know we don't usually have reviews for tools, but I guess we should start sooner or later.

I endorse both this tool being part of Boost. And having a review for it. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

Vinnie Falco

8 Oct 8 Oct

6:57 p.m.

On Fri, Oct 4, 2024 at 6:43 AM René Ferdinand Rivera Morell via Boost < boost@lists.boost.org> wrote:

...

I endorse both this tool being part of Boost. And having a review for it.

I endorse the tool, and I do not agree that it needs a review. Since when are we reviewing tools? These are not specifically for end-users the same way that libraries are. This tool is for supporting the libraries. As long as the tool works and is useful to some of the libraries I don't see why the review is necessary. Or to put it differently, what is the benefit of a REJECT outcome from a formal review of pretty_printers? Thanks

René Ferdinand Rivera Morell

7:58 p.m.

On Tue, Oct 8, 2024 at 1:57 PM Vinnie Falco wrote:

...

On Fri, Oct 4, 2024 at 6:43 AM René Ferdinand Rivera Morell via Boost wrote:

...
I endorse both this tool being part of Boost. And having a review for it.

I endorse the tool, and I do not agree that it needs a review. Since when are we reviewing tools? These are not specifically for end-users the same way that libraries are. This tool is for supporting the libraries. As long as the tool works and is useful to some of the libraries I don't see why the review is necessary. Or to put it differently, what is the benefit of a REJECT outcome from a formal review of pretty_printers?

Given how this tool generates code that is directly part of Boost libraries I think a review would help in getting approval that it does things in a way that authors will accept. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

Vinnie Falco

8:12 p.m.

On Tue, Oct 8, 2024 at 12:59 PM René Ferdinand Rivera Morell via Boost < boost@lists.boost.org> wrote:

...

Given how this tool generates code that is directly part of Boost libraries I think a review would help in getting approval that it does things in a way that authors will accept.

Neither approval nor acceptance from "authors" is necessary. Authors express their dislike of pretty_printers by not using the tool. Library authors have traditionally been free to style their code however they like. Robert for example is free to use C++98isms in his Serialization library, and I will defend the right to do so to the death (his not mine). As long as there is even one author who wishes to use pretty_printers in their already-accepted library, then the tool should be added to the Boost Project without any need for a formal review. As far as I can tell, most if not all of the tools in Boost were added without a formal review. However, there is some approval required since these tools affect shared resources. When I added the docca toolchain to the project, it required saxon-he (written in Java) which Peter thankfully helped me get going on the containers which build the releases. In my opinion, the current system for dealing with tools has been working well and I have not yet heard a rationale for why it should change. On a somewhat unrelated note, I am proud to announce that with Dmitry's work on creating a Python implementation of docca (in the same repository), we will soon be able to remove saxon-he from the containers. This is a nice small win, as the removal of outdated tools from Boost is a rare event. And shortly after Mr. Docs reaches version 1.0, we can remove the docca tool entirely from the project after the libraries which use it are upgraded. I'm sure there will be little to no opposition for this flavor of removal of obsolete things, in comparison to the removal of obsolete libraries (which in fairness cannot be safely removed anyway). Thanks

John Maddock

9 Oct 9 Oct

8:08 a.m.

On 08/10/2024 21:12, Vinnie Falco via Boost wrote:

...

On Tue, Oct 8, 2024 at 12:59 PM René Ferdinand Rivera Morell via Boost < boost@lists.boost.org> wrote:

...
Given how this tool generates code that is directly part of Boost libraries I think a review would help in getting approval that it does things in a way that authors will accept.

Neither approval nor acceptance from "authors" is necessary. Authors express their dislike of pretty_printers by not using the tool. Library authors have traditionally been free to style their code however they like. Robert for example is free to use C++98isms in his Serialization library, and I will defend the right to do so to the death (his not mine). As long as there is even one author who wishes to use pretty_printers in their already-accepted library, then the tool should be added to the Boost Project without any need for a formal review.

As far as I can tell, most if not all of the tools in Boost were added without a formal review.

I put auto_Index through review - I believe it helped to make it more usable. However, the standard for acceptance, and level of review may well be different given that this is not user-facing code. Just my 2c, John.

Matt Borland

8 Oct 8 Oct

9:04 p.m.

...

I endorse the tool, and I do not agree that it needs a review. Since when are we reviewing tools? These are not specifically for end-users the same way that libraries are. This tool is for supporting the libraries. As long as the tool works and is useful to some of the libraries I don't see why the review is necessary.

Since the preponderance of those who participate in reviews are library devs I think it is important to review. We all complain about bad tools, and good tools are beneficial to many. In all likelihood Dimitry didn't think of/document everything in his design so the review can be mutually beneficial.

...

Or to put it differently, what is the benefit of a REJECT outcome from a formal review of pretty_printers?

The same thing that happens to a library with a REJECT outcome Matt

Braden Ganetsky

18 Oct 18 Oct

4:24 a.m.

On Fri, Oct 4, 2024, 5:31 AM Дмитрий Архипов via Boost < boost@lists.boost.org> wrote:

...

I want to make it a Boost tool, that is to put it inside boost/tools. So, I was advised to ask for a review on the list. I know we don't usually have reviews for tools, but I guess we should start sooner or later.

I was suggested as the review manager for this proposed tool on the Slack. If I'm understanding the process correctly, the next step for me is to mention this here on the ML. I gratefully accept the suggestion, and I would like to volunteer as the review manager. As far as I understand, this would be the first time a "tool" would be put through the formal review process. Review wizards, please let me know of any parts of the duties to be modified for this particular role of review manager. Thanks, Braden

Дмитрий Архипов

21 Oct 21 Oct

5:53 p.m.

пт, 18 окт. 2024 г. в 07:24, Braden Ganetsky via Boost :

...

On Fri, Oct 4, 2024, 5:31 AM Дмитрий Архипов via Boost < boost@lists.boost.org> wrote: I was suggested as the review manager for this proposed tool on the Slack. If I'm understanding the process correctly, the next step for me is to mention this here on the ML. I gratefully accept the suggestion, and I would like to volunteer as the review manager.

Thank you Braden. I accept your proposition to manage the review.

Дмитрий Архипов

6 Nov 6 Nov

1:12 p.m.

пн, 21 окт. 2024 г. в 20:53, Дмитрий Архипов :

...

Thank you Braden. I accept your proposition to manage the review.

I recently realised that there's an apparent conflict of interest arising from the fact that Braden and I both work for the C++ Alliance. It sounds silly that I have not thought about it before I accepted Braden as the review manager, but I did. Now I am asking the Boost community, if you think that this compromises the review process. If you do, I will change the review manager.

Glen Fernandes

1:40 p.m.

On Wednesday, November 6, 2024, Дмитрий Архипов wrote:

...

пн, 21 окт. 2024 г. в 20:53, Дмитрий Архипов :

...
Thank you Braden. I accept your proposition to manage the review.

I recently realised that there's an apparent conflict of interest arising from the fact that Braden and I both work for the C++ Alliance.

Let's change the review manager, please. Thanks Dmitry. Glen

Richard

3:39 p.m.

In article you write:

...

I recently realised that there's an apparent conflict of interest arising from the fact that Braden and I both work for the C++ Alliance. It sounds silly that I have not thought about it before I accepted Braden as the review manager, but I did. Now I am asking the Boost community, if you think that this compromises the review process. If you do, I will change the review manager.

I see no reason to change. Just please call this library something else than 'pretty printer' :) -- "The Direct3D Graphics Pipeline" free book http://tinyurl.com/d3d-pipeline The Terminals Wiki http://terminals-wiki.org The Computer Graphics Museum http://computergraphicsmuseum.org Legalize Adulthood! (my blog) http://legalizeadulthood.wordpress.com

Braden Ganetsky

8 Oct 8 Oct

9:57 p.m.

On Wed, Sep 11, 2024, 2:47 PM Дмитрий Архипов via Boost < boost@lists.boost.org> wrote:

...

My questions to the community: do you find this useful? Would you use it in your Boost or non-Boost library? If you do, the module should be moved to tools rather than stay in libs/json.

Whatever the administrative procedure is, I endorse this tool. I would like to see it called something a bit more general than "Pretty Printers" though. I'd prefer something like "Debugging" or "Visualizations". Something that can encompass other related tools, other than just GDB pretty-printers, including Natvis or any other popular debugger extension that may pop up in the future. Regardless, this point isn't blocking my endorsement. Braden

...

Дмитрий Архипов

9 Oct 9 Oct

8:27 a.m.

Thank you Rene, Vinnie, and Braden for your endorsements. ср, 9 окт. 2024 г. в 00:57, Braden Ganetsky via Boost :

...

I would like to see it called something a bit more general than "Pretty Printers" though. I'd prefer something like "Debugging" or "Visualizations". Something that can encompass other related tools, other than just GDB pretty-printers, including Natvis or any other popular debugger extension that may pop up in the future.

I actually too started thinking that pretty_printers is a misleading name for the project. Julien's PR to JSON gave me the idea to rename the project to debug_printers. But now I've realised that it's not necessarily limited to pretty printers/visualizers. The embedded GDB extension module can contain other utilities like xmethods and new commands. Just the other day I was debugging some code and thought that a command to put a breakpoint at a boost::source_location could be very helpful. So, how about DebuggerUtils?

René Ferdinand Rivera Morell

3:59 p.m.

On Wed, Oct 9, 2024 at 3:28 AM Дмитрий Архипов via Boost wrote:

...

So, how about DebuggerUtils?

There's also going for the mascot name effect.. How about "Roach" ;-) -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

hermann＠stamm-wilbrandt.de

4:22 p.m.

On 2024-10-09 17:59, René Ferdinand Rivera Morell via Boost wrote:

...

On Wed, Oct 9, 2024 at 3:28 AM Дмитрий Архипов via Boost wrote:

...
So, how about DebuggerUtils?

There's also going for the mascot name effect.. How about "Roach" ;-)

That sounds too much like the 1991 xroach game ... https://github.com/interkosmos/xroach

Richard

3:47 p.m.

In article you write:

...

While doing all this I thought that most of this work can be useful to other libraries. So, I moved the reusable parts into a separate directory (https://github.com/boostorg/json/tree/develop/pretty_printers) and added both a b2 module, and a CMake module.

This is a great idea, but the name 'pretty_printers' does not convey to me what is going on. (Naming things is hard.) How about 'gdb_printers'? It's only useful for gdb. Visual Studio has it's own, different, mechanism for providing pretty output in the debugger -- VS calls them visualizers. -- Richard -- "The Direct3D Graphics Pipeline" free book http://tinyurl.com/d3d-pipeline The Terminals Wiki http://terminals-wiki.org The Computer Graphics Museum http://computergraphicsmuseum.org Legalize Adulthood! (my blog) http://legalizeadulthood.wordpress.com

Peter Dimov

3:52 p.m.

Richard wrote:

...

In article you write:

...
While doing all this I thought that most of this work can be useful to other libraries. So, I moved the reusable parts into a separate directory (https://github.com/boostorg/json/tree/develop/pretty_printers) and added both a b2 module, and a CMake module.

This is a great idea, but the name 'pretty_printers' does not convey to me what is going on. (Naming things is hard.) How about 'gdb_printers'? It's only useful for gdb.

Or "gdb_support" if in the future it would include more than just printers.

Age (days ago)

Last active (days ago)

List overview

Download

24 comments

12 participants

participants (12)

Braden Ganetsky
Glen Fernandes
hermann＠stamm-wilbrandt.de
John Maddock
Julien Blanc
Matt Borland
Peter Dimov
René Ferdinand Rivera Morell
Richard
Seth
Vinnie Falco
Дмитрий Архипов