- Boost - lists.preview.boost.org

Boost.Parser review by Andrzej
by Andrzej Krzemienski 24 Feb '24

24 Feb '24

Hi, This is another review of Boost.Parser. Usefulness If Boost.Parser is a maintained version of Boost.Spirit that is already a worth goal. If Boost.Parser is a better documented version of Boost.Spirit that is already a gain. Note that the docs for Boost.Spirit X3 are hidden, and I was not aware of its existence until the review of Boost.Parser. I hear that the selling point of this library is Unicode support "through and through". I cannot confirm that it works as advertised, nor benefit from this guarantee, as all my use cases for a parser so far only dealt with the ASCII set. Design Boost.Spirit has a great design, and because Boost.Parser adopts the same design, it must be also good. I would intuitively expect that C++17 offers sufficient number of new features that the design for a parser library would be improved over Boost.Spirits. For instance, I am surprised that we still need to use a macro. Not that I can offer a better alternative. I do not know much about the implementation difficulties. so maybe the macro is the best choice. Wouldn't template explicit specializations do the job? The fact that this library automatically adapts aggregate types as attributes in a Boost.PFR way is really cool. I am convinced that compiler error messages could be better. This can be done by using a combination of static_assert and implementation details SFINAEd away based on the same condition as in static_assert. See https://github.com/tzlaine/parser/issues/56 I am unhappy about the fact that as simple task as parsing a JSON-like array of ints `[1, 2, 3]` into a vector requires me to define a rule. Spirit introduced operator% that doesn't work for empty lists, but doe to its clever attribute conversions (at least in Spirit2), I could define: auto array = '[' > ((int_ % ',') | eps) > ']'; And it would deduce the return attribute alright. But because Boost.Parser wants to be more strict about clever attribute conversions this option is gone, and no alternative is provided other than to declare a rule, which I never needed with Boost.Spirit. See https://github.com/tzlaine/parser/issues/67 I am not comfortable that I have to use the macro BOOST_PARSER_DEFINE_RULES. I would expect that template specializations should give a similar effect. But I do not know how to make it better. But if I have to use a macro, maybe it could offer more, like checking if the rule is compatible with its definition (for earlier bug reporting). I am missing a tool that would give me a compile-time answer * can I pass this attribute to that parser when parsing this type of character? * will my rule with its parser compile for a given input character type? * will my rule with its parser compile for at least one input character type? Documentation To me, the tutorial and overview in the documentation were sufficient to understand the library. But this is because I had learned Boost.Spirit before. If this were my first exposure to "a parser implementation via expression templates", I would be overwhelmed. Even though a lot of effort went into preparing a soft introduction to the subject, the subject is so complicated that it requires a super-gentle introduction. Some things remain underdocumented, like the interface of context. See https://github.com/tzlaine/parser/issues/104 After having stared at the docs for more than a month, I still have trouble finding the table with all the parsers listed. The reference section of documentation is missing details, such as what functions throw, what they expect of their arguments, what they guarantee, what is the exception safety level. For instance, see https://github.com/tzlaine/parser/issues/103 Also, the layout of the reference section looks like it is driven by an auto-generation tool run on the header files. It is not very user friendly. for instance, trying to find the specification of function parse() in https://tzlaine.github.io/parser/doc/html/header/boost/parser/parser_hpp.ht… isn't easy: neither visually (the amount of content is overwhelming) nor with grep-like tool (a lot of things there have "parse" in the name). What is good for header organization is not necessarily good for documentation organization. I am uncomfortable with the responses from the author that the reference section is missing parts, such as concepts, because the documentation toolchain does not support them. This would look like a bad choice of the tool. On the other hand, I do not know of any satisfactory tool for good C++ documentation. My testing I spent like a week trying to compile my parser for a text-based game engine, which was previously compiled with Boost.Spirit v2. Here's a sample file to parse (in Polish): https://raw.githubusercontent.com/akrzemi1/wdungeon/master/wdungeon.plot Here's the original Spirit V2 parser: https://raw.githubusercontent.com/akrzemi1/wdungeon/master/parser.cpp The new parser in BoostParser is included below. I didn't have to use rules in the original version. I am forced to use them in Boost.Spirit. I was not interested in performance testing as my use case is to read the config file upon application startup, so I can afford to wait. I tried the library with the newest versions of MSVC, GCC and Clang. It works in all of them. I filed a number of bug reports, to which Zach has been responding extremely quickly. Thank you! I also appreciate the negative responses, as they indicate that the author has a clear and coherent vision of what he wants the library to be. I recommend accepting the library into Boost. I would also like to thank Zach for writing and sharing this library! Regards, &rzej; ``` namespace parser { using namespace plot2; const auto identifier = bp::lexeme[+(bp::digit | bp::lower | bp::upper | bp::char_('_'))]; const bp::rule<struct cond_tag, std::vector<Variable>> cond = "cond"; const bp::rule<struct set_vars_tag, std::vector<Variable>> set_vars = "set_vars"; const bp::rule<struct choice_tag, Choice> choice = "choice"; const bp::rule<struct continuation_tag, Alternative> continuation = "continuation"; const bp::rule<struct map_tag, std::vector<MapStep>> map = "map"; const bp::rule<struct label_tag, std::string> label = "label"; const bp::rule<struct text_tag, std::string> text = "text"; const bp::rule<struct transition_tag, Transition> transition = "transition"; const bp::rule<struct redirection_tag, Redirection> redirection = "redirection"; const bp::rule<struct comment_tag> comment = "comment"; const bp::rule<struct plot_tag, GamePlot> plot = "plot"; const auto cond_def = ("[" >> ((identifier >> "=" > bp::int_) % ",") > "]") | bp::attr(std::vector<Variable>{}); const auto set_vars_def = ("[" >> ((identifier >> ":=" > bp::int_) % ",") > "]") | bp::attr(std::vector<Variable>{}); auto end_tag = bp::lit("[") >> bp::char_("+!") >> bp::lit("]"); auto label_def = bp::lit("[") >> identifier >> bp::lit("]"); auto text_def = bp::lexeme[+(bp::char_ - ']' - '[')]; auto map_step = bp::lit("^") >> -bp::char_("NESW") >> -(bp::lit(".") >> (+bp::char_("STCLPRXHUVD#"))); auto map_def = bp::lit("[") >> +map_step >> bp::lit("]"); auto redirection_def = +(bp::lit("[->]") >> cond >> (-map) >> label >> set_vars); auto option = bp::lit("[#]") >> cond >> text >> (-map) >> label >> (-text) >> set_vars; auto choice_def = bp::lit("[?]") >> text >> (+option); auto continuation_def = redirection | end_tag | choice; auto transition_def = bp::lit("[##]") > label >> -map >> -text > continuation; auto comment_def = bp::lit("[//]") >> bp::omit[*(bp::char_ - ']' - '[')]; auto plot_def = +transition; BOOST_PARSER_DEFINE_RULES(cond, set_vars, choice, continuation, map, label, text, transition, redirection, comment, plot); } ```

2 3

Parsers vs Unicode
by Arno Schoedl 24 Feb '24

24 Feb '24

Since the reviews of Boost.Parser are currently on, I wanted to share an insight we had at think-cell when working with Boost.Spirit. We have standardized on it for years for all custom parsing needs. Most parsers are small, but some are larger, like Excel expressions. Of course, our input is mostly Unicode, either UTF-8 or UTF-16. Matching Unicode is complex. Comparison by code point is usually not the right thing. Instead, we must normalize, for which we even have various choices of what to accept as equal: https://en.wikipedia.org/wiki/Unicode_equivalence Case-insensitive matching is even more complex, slow, and even language-dependent. Input is often not guaranteed to be valid Unicode. For example, file names on Windows are sequences of 16-bit units, allowing unmatched surrogates, same with input from Win32 edit boxes and file content. But we realized that for almost all grammars we have, all this complexity does not matter. The reserved symbols of most grammars (JSON, XML, C++, URLs, etc.) are pure ASCII. Semantically relevant strings are ASCII as well („EXCEL.EXE“). ASCII can be correctly and quickly matched on a per-code-unit basis. Case-insensitive matching for ASCII is simple and fast. User-defined strings, such as JSON string values, may contain Unicode, but then they usually do not affect parsing decisions. The user may want Unicode validation for these strings, but this can be done by the leaf parser for these strings, rather than for the whole input. Since so much matching is against ASCII, we found it useful to have compile-time known ASCII literals in the parser library. With them, the same grammar can be used for all input encodings. When parsing user-defined strings, they will have the encoding of the input, but that’s fine. Any encoding conversion can be dealt with separately from the parser. Finally, we may want to parse more than just strings. Parsing binary files, or sequences of DNA, should be possible and efficient. Thus I recommend separating Unicode processing from the parser library. The parser library operates on an abstract stream of symbols. For Unicode text these would be code units. It provides the composite parsers such as sequences with and without backtracking, alternatives, Kleene star etc., and leaves the interpretation of the symbols entirely to the leaf parsers, which may or may not care about Unicode. We started modifying Boost.Spirit in this direction, and it serves us well. -- Dr. Arno Schödl CTO schoedl(a)think-cell.com<mailto:schoedl@think-cell.com> | +49 30 6664731-0 We are looking for C++ Developers: https://www.think-cell.com/developers think-cell Software GmbH (Web site<https://www.think-cell.com>) Leipziger Str. 51, 10117 Berlin, Germany Main phone +49 30 6664731-0 | US toll-free +1 800 891 8091 Amtsgericht Berlin-Charlottenburg HRB 180042 Directors: Christoph Hobo, Dr. Arno Schödl Please refer to our privacy policy<https://www.think-cell.com/privacy> on how we protect your personal data.

3 4

Odd build error in 1.85 pre-release - filesystem
by Tom Kent 23 Feb '24

23 Feb '24

When building for windows against the 1.85.0 nightly snapshot, I'm seeing an odd error with no info around it: - type_erasure : building - url : building - wave : building ...patience... ...patience... ...patience... ...patience... ...patience... ...patience... ...patience... [errno 22] failed to scan file '': Invalid argument...found 39744 targets... Upping the verbosity in B2, I was able to find that it appears to happen in filesystem: bind -- <object(c-scanner)@8602>boost\filesystem\detail\type_traits\disjunction.hpp: boost\filesystem\detail\type_traits\disjunction.hpp time -- <object(c-scanner)@8602>boost\filesystem\detail\type_traits\disjunction.hpp: 2024-02-22 07:18:09.000000000 +0000 make -- <object(c-scanner)@8602>boost\filesystem\detail\type_traits\disjunction.hpp make -- <object(c-scanner)@8602>boost\filesystem\detail\type_traits\disjunction.hpp time -- <object(c-scanner)@8602>boost\filesystem\detail\type_traits\disjunction.hpp: unbound made stable <object(c-scanner)@8602>boost\filesystem\detail\type_traits\disjunction.hpp make -- <object(c-scanner)@8602>boost\filesystem\detail\header.hpp make -- <object(c-scanner)@8602>boost\filesystem\detail\header.hpp bind -- [errno 22] failed to scan file '': Invalid argument <object(c-scanner)@8602>boost\filesystem\detail\header.hpp: boost\filesystem\detail\header.hpp time -- <object(c-scanner)@8602>boost\filesystem\detail\header.hpp: 2024-02-22 07:18:09.000000000 +0000 make -- <object(c-scanner)@8602>boost\filesystem\detail\header.hpp make -- <object(c-scanner)@8602>boost\filesystem\detail\header.hpp time -- <object(c-scanner)@8602>boost\filesystem\detail\header.hpp: unbound made stable <object(c-scanner)@8602>boost\filesystem\detail\header.hpp Anyone have ideas on what this can be? Why it doesn't show up more like a normal error? Thanks, Tom

3 4

2023 Transparency Report
by Louis Tatta 21 Feb '24

21 Feb '24

Hello, Louis Tatta here, CEO of The C++ Alliance, Inc, a registered 501(c)(3) non-profit organization dedicated to the support of C++. This is our 2023 transparency report regarding our charitable work for the Boost C++ Library Collection. First, some background on our operations. The C++ Alliance seeks out the highly talented and experienced C++ experts and hires them, usually full-time, so they can focus completely on their open-source projects. Our Staff Engineers maintain existing Boost libraries, write new Boost libraries, and contribute to other Boost projects. We employ IT experts full-time to maintain and improve the infrastructure which supports Boost, including the legacy web server, the Drone continuous integration servers, the scripts used to deploy GitHub Actions for Boost repositories, and the scripts which publish the Boost release. In addition, we provide on-demand technical support services for Boost authors and maintainers who have infrastructure related problems. The Boost Mailing List continues to be the premier place for high-quality discussion of the Boost libraries and tangential C++ Standards related topics. The C++ Alliance supports the use of the mailing lists going forward. We are upgrading it to the latest Mailman version, and building an optional web-based front-end allowing interaction using only a browser. We continue to operate the Official C++ Language Slack Workspace (https://cpp.al/slack) which hosts the most popular interactive discussions regarding the Boost libraries. The C++ Alliance sponsors a paid plan to provide full history, video and audio conferences, collaboration tools, and unlimited document storage for seventeen dedicated Boost library channels and one main Boost discussion channel. Our largest project for 2023 is the continuation of work on the new Boost website. The new site is a C++ social media portal with rich support for user-generated content, and a powerful back-end which has visibility into the contributions made on GitHub and the release process. This allows us to deliver an elevated user experience with the hopes of attracting fresh talent to Boost. Also,note that the graphics on the website are hand-drawn by our contract artist and reflect our level of commitment to ensure that Boost, to put it simply, looks good. In 2023 there were a total of four Boost Formal Reviews. One of the reviewed libraries is authored by a C++ Alliance Staff Engineer. Three of the reviews were managed by C++ Alliance team members. Out of a total of twenty-eight submitted reviews for the year, twelve were from C++ Alliance team members. Last year we increased our efforts to build Boost’s social media presence. There were a total of twelve messages from the @Boost_Libraries X (formerly Twitter) account, all submitted by C++ Alliance team members. Three of them contained artwork from Bob Ostrom commissioned by us. Messages containing images get three times more engagement than text-only messages. We believe that the X platform has unique strategic value for informing newer generations about the Boost value proposition. A total of $1,768,000 was spent on Boost-related contributions, broken down thusly (and rounded to the nearest thousand): Staff Engineers Compensation $1,240,000 Website Software Development $570,000 Server Hosting $62,000 C++ Slack Workspace $96,000 Conference Sponsorship $10,000 Conference Attendance $10,000 During 2023 we employed a total of 12 individuals to contribute to Boost. Here is a high-level summary of accomplishments: Sam Darwin (Chief Technical Officer) * Boost website maintenance and administration * Boost per-pull-request documentation previews * Boost Release Process development, maintenance, and administration * Boost Mailman services maintenance and upgrade * Boost GitHub Actions self-hosted runner project development * Boost Drone service administration and maintenance * New Boost website deployment, administration, and development Dmitry Arkhipov (Staff Engineer) * Boost.JSON maintenance Alan de Freitas (Staff Engineer) * Boost.URL maintenance * Mr. Docs project management (https://mrdocs.com) * GitHub actions support scripts for Boost CI (https://alandefreitas.github.io/cpp-actions/cpp-actions/index.html) Christian Mazakas (Staff Engineer) * Boost.Unordered evolution project * Boost.Buffers development (new library) * Boost.Http.Proto, Boost.Http.Io development (new libraries) * Boost.Websocket.Proto, Boost.Websocket.Io development (new libraries) Krystian Stasiowski (Staff Engineer) * Boost.StaticString maintenance * Mr. Docs Lead engineer (https://mrdocs.com/) * Clang/LLVM contributions Peter Turcan (Senior Technical Writer) * Boost User Guide development (new documentation) * Boost Contributor Guide (new documentation) * Boost Formal Reviews (new documentation) Matt Borland (Staff Engineer) * Boost.Charconv (new, accepted library) Ruben Perez * Boost.MySQL maintenance (new, accepted library) * User Guide, Contributor Guide contribution (new documentation) Mohammad Nejati (Staff Engineer) * Boost website full-text search feature * Boost.Beast maintenance Fernando Pelliccioni (Staff Engineer) * Mr. Docs development (https://mrdocs.com/) Spencer Strickland (Senior Web Developer) * New website front-end and scripting Klemens Morgenstern (Staff Engineer) * Boost.Beast maintenance * Boost.Cobalt (new, accepted library) Joaquin M Lopez Munoz (Staff Engineer) * Boost.Unordered evolution project lead * Boost Contributor Guide contributions (new documentation) Our sources of funding are secure and we look forward to continuing our charitable contributions throughout this decade and beyond. It is an honor and a pleasure to support the Boost C++ Library collection. Thank You Louis Tatta

5 4

Reminder: Boost 1.85.0 will be closing for new libraries and breaking changes THIS WEDNESDAY
by Marshall Clow 20 Feb '24

20 Feb '24

As always, look at the calendar at https://www.boost.org/development/ — Marshall

2 1

Review of Parser library begins today
by Marshall Clow 19 Feb '24

19 Feb '24

The review of Zach Laine’s proposed Boost.Parser library begins today and will end on February 28th. From the introduction page of the documentation: Boost.Parser is a parser combinator library. That is, it consists of a set of low-level primitive parsers, and operations that can be used to combine those parsers into more complicated parsers. There are primitive parsers that parse epsilon (the empty string), chars, ints, floats, etc. There are operations which combine parsers to create new parsers. For instance, the Kleene star operation takes an existing parser p and creates a new parser that matches zero or more occurrences of whatever p matches. Both callable objects and operator overloads are used for the combining operations. For instance, operator*() is used for Kleene star, and you can also write repeat(n)[p] to create a parser for exactly n repetitions of p. Boost.Parser also tries to accommodate the multiple ways that people often want to get a parse result out of their parsing code. Some parsing may best be done by returning an object that represents the result of the parse. Other parsing may best be done by filling in a preexisting data structure. Yet other parsing may best be done by parsing small sections of a large document, and reporting the results of subparsers as they are finished, via callbacks. Boost.Parser accommodates all these ways of working, and even makes it possible to do callback-based or non-callback-based parsing without rewriting any code (except by changing the top-level call from parse() to callback_parse()). All of Boost.Parser's public interfaces are sentinel- and range-friendly, just like the interfaces in std::ranges. Boost.Parser is Unicode-aware through and through. When you parse ranges of char, Boost.Parser does not assume any particular encoding — not Unicode or any other encoding. Parsing of inputs other than plain chars assumes that the input is Unicode. In the Unicode-aware code paths, all parsing is done by matching code points. This means that you can feed UTF-8 strings into Boost.Parser, both as input and within your parser, and the right sort of matching occurs. For instance, if your parser is trying to match repetitions of the char '\xcc' (which is a lead byte from a UTF-8 sequence, and so is malformed UTF-8 if not followed by an appropriate UTF-8 code unit), it will not match the start of "\xcc\x80" (UTF-8 for the code point U+0300). Boost.Parser knows that the matching must be whole-code-point, and so it interprets the char '\xcc' as the code point U+00CC. Error reporting is important to get right, and it is important to make errors easy to understand, especially for end-users. Boost.Parser produces runtime parse error messages that are very similar to the diagnostics that you get when compiling with GCC and Clang (it even supports warnings that don't fail the parse). The exact token associated with a diagnostic can be reported to the user, with the containing line quoted, and with a marker pointing right at the token. Boost.Parser takes care of this for you; your parser does not need to include any special code to make this happen. Of course, you can also replace the error handler entirely, if it doesn't fit your needs. Debugging complex parsers can be a real nightmare. Boost.Parser makes it trivial to get a trace of your entire parse, with easy-to-read (and very verbose) indications of where each part of the trace is within the parse, the state of values produced by the parse, etc. Again, you don't need to write any code to make this happen — you just pass a parameter to parse(). Github: https://github.com/tzlaine/parser Docs https://tzlaine.github.io/parser — Marshall

1 0

Re: [boost] Should boost::core::string_view be made public?
by g.peterhoff＠t-online.de 18 Feb '24

18 Feb '24

Have you come to a conclusion how this should be handled (in the future)? The problem is basically that there are different implementations for string_view. I don't know why it was done this way. Presumably it's because various boost libraries want to be independent of others, so that some things are implemented anew each time (within the library) instead of using existing libraries. This will also be due to the fact that boost is not atomistic enough, so that large libraries might have to be included/used, although only a fraction of the functionality is needed. My suggestion: - Throw out boost::utility::string_view - Make boost::core::detail::string_view public, just like all other backports in boost::core. - Check if there are other multiple implementations. If so, pull them out into separate small boost libraries. (For databases, this would be called normalization, but it must not be over-normalized). regards Gero

5 4

Potential Launch of Boost Website: Request for Final Feedback
by Louis Tatta 17 Feb '24

17 Feb '24

Louis here from the C++ Alliance. With the potential launch of the new Boost website on the horizon this week, we're making a final call for your feedback. Your insights a few months ago were incredibly helpful, and as we approach the launch, we'd value your final review. If you have a moment to look over the site again and note any issues or suggestions, it would be of great help. Feedback can be submitted on our GitHub issues page: https://github.com/cppalliance/temp-site/issues. Thank you for your ongoing support and contributions to the Boost community. Looking forward to your feedback. Best regards, Louis C++ Alliance

18 79

Re: [boost] New Website Repositories
by Glen Fernandes 16 Feb '24

16 Feb '24

On 2024-02-16, Vinnie Falco wrote: > On Fri, Feb 16, 2024 at 12:33, Glen Fernandes wrote: > > If you want to follow the tradition of source files, the easiest > > convention to follow is to state that your files, e.g. > > > >  > > We can do that, or alternatively is it possible to just plop a LICENSE > file at the root of the repository? I would prefer both (thanks). Glen

1 0

New Website Repositories
by Vinnie Falco 16 Feb '24

16 Feb '24

All: The contribution of a brand new, mostly complete website at the level of quality of what we at The C++ Alliance have developed (see https://preview.boost.org) is a rare event. We are proud to share this special moment in Boost history. As is the Boost tradition we propose to do the following: * Transfer of all website related repositories to the Boost GitHub organization (these will also be renamed to something sensible): https://github.com/cppalliance/temp-site https://github.com/cppalliance/temp-site-documentation https://github.com/cppalliance/site-docs https://github.com/cppalliance/boost-mailman ...plus any other miscellaneous necessary repositories that Sam has created for the administration of the website, mailing list, and release process. * Whatever is the appropriate license for the artwork to be used on the website, with copyright to "The C++ Alliance, Inc. (https://cppalliance.org)" With respect to the artwork licensing, I have to admit I am not an expert so we will work to find out what the proper solution is. Thanks

3 3