- Boost - lists.preview.boost.org

Interest in new parsing library?
by Zach Laine 30 Dec '23

30 Dec '23

I'm trying to gauge interest in a parsing library to replace Boost.Spirit 2/Spirit X3. I'm also looking for endorsements. The library is intended to remedy some shortcomings of Boost.Spirit*. I think these are great libraries, but Spirit 2 was written in pre-11 C++ (I think; certainly its dependencies were). Most-to-all of the downsides stem from that -- long compile times, inscrutable compilation failures, etc. (Boost.Parser compile times are quite low.) I'm calling my proposal Boost.Parser, and it follows many of the conventions of Boost.Spirit 2 and X3, such as the operators used for overloading, the names of many parsers and directives, etc. It requires C++17 or later. >From the introduction in the online docs: """ Boost.Parser is a parser combinator library. That is, it consists of a set of low-level primitive parsers, and operations that can be used to combine those parsers into more complicated parsers. There are primitive parsers that parse epsilon (the empty string), chars, ints, floats, etc. There are operations which combine parsers to create new parsers. For instance, the Kleene star operation takes an existing parser p and creates a new parser that matches zero or more occurrences of whatever p matches. Both callable objects and operator overloads are used for the combining operations. For instance, operator*() is used for Kleene star, and you can also write repeat(n)[p] to create a parser for exactly n repetitions of p. Boost.Parser also tries to accommodate the multiple ways that people often want to get a parse result out of their parsing code. Some parsing may best be done by returning an object that represents the result of the parse. Other parsing may best be done by filling in a preexisting data structure. Yet other parsing may best be done by parsing small sections of a large document, and reporting the results of subparsers as they are finished, via callbacks. Boost.Parser accommodates all these ways of working, and even makes it possible to do callback-based or non-callback-based parsing without rewriting any code (except by changing the top-level call from parse() to callback_parse()). All of Boost.Parser's public interfaces are sentinel- and range-friendly, just like the interfaces in std::ranges. Boost.Parser is Unicode-aware through and through. When you parse ranges of char, Boost.Parser does not assume any particular encoding — not Unicode or any other encoding. Parsing of inputs other than plain chars assumes that the input is Unicode. In the Unicode-aware code paths, all parsing is done by matching code points. This means that you can feed UTF-8 strings into Boost.Parser, both as input and within your parser, and the right sort of matching occurs. For instance, if your parser is trying to match repetitions of the char '\xcc' (which is a lead byte from a UTF-8 sequence, and so is malformed UTF-8 if not followed by an appropriate UTF-8 code unit), it will not match the start of "\xcc\x80" (UTF-8 for the code point U+0300). Boost.Parser knows that the matching must be whole-code-point, and so it interprets the char '\xcc' as the code point U+00CC. Error reporting is important to get right, and it is important to make errors easy to understand, especially for end-users. Boost.Parser produces runtime parse error messages that are very similar to the diagnostics that you get when compiling with GCC and Clang (it even supports warnings that don't fail the parse). The exact token associated with a diagnostic can be reported to the user, with the containing line quoted, and with a marker pointing right at the token. Boost.Parser takes care of this for you; your parser does not need to include any special code to make this happen. Of course, you can also replace the error handler entirely, if it doesn't fit your needs. Debugging complex parsers can be a real nightmare. Boost.Parser makes it trivial to get a trace of your entire parse, with easy-to-read (and very verbose) indications of where each part of the trace is within the parse, the state of values produced by the parse, etc. Again, you don't need to write any code to make this happen — you just pass a parameter to parse(). Dependencies are still a nightmare in C++, so Boost.Parser can be used as a purely standalone library, independent of Boost. """ Boost.Parser aims to be a superset of Boost.Spriit* in most ways. Major things missing from the set of features in Spirit 2 + Spirit X3 are: - A separate lexer. - Binary parsers (meaning for parsing bits, not binary numbers written as text; the latter is fully supported). I've been in touch with Joel de Guzman, Hartmut Kaiser, and Michael Caisse, to make sure I was not toe-stomping, for those who are concerned about that. They gave this new library their blessing. One feature comes entirely from them: Boost.Parser is usable in a Boost-free environment -- as a standalone library -- at the user's option. They said that was the #1 request from users, which surprised me a bit. The Github page is here: https://github.com/tzlaine/parser The online docs are here: https://tzlaine.github.io/parser To see an extended example, here's a JSON parser that passes all the published JSON tests, including most of the optional ones, in only about 300 lines of code, go here: https://tzlaine.github.io/parser/doc/html/boost_parser__proposed_/extended_… Finally, for those wanting to know how this lib differs from Boost.Spirit* without digging through the docs, here is the doc page that explains Boost.Parser's relationship to Boost.Spirit*: """ Boost.Spirit is a library that is already in Boost, and it has been around for a long time. However, it does not suit user needs in some ways. Spirit 2 suffers from very long compile times. Spirit 2 has error reporting that requires a lot of user intervention to work. Spirit 2 requires user intervention, including a (long) recompile, to enable parse tracing. Spirit X3 has rules that do not compose well — the attributes produced by a rule can change depending on the context in which you use the rule. Spirit X3 is missing many of the convenient interfaces to parsers that Spirit 2 had. For instance, you cannot add parameters to a parser. All versions of Spirit have Unicode support, but it is quite difficult to get working. I wanted a library that does not suffer from any of the above limitations. It should be noted that while Spirit X3 only has a couple of flaws in the list above, the one related to rules is a deal-breaker. The ability to write rules, test them in isolation, and then re-use them throughout a complex parser is essential. Though no version of Boost.Spirit (Spirit 2 or Spirit X3) suffers from all those limitations, there also does not exist any one version that avoids all of them. Boost.Parser does so. However, there are a lot of great ideas in Boost.Spirit that have been retained in Boost.Parser. Both libraries: - use the same operator overloads to combine parsers; - use approximately the same set of directives to influence the parse (e.g. lexeme[]); - provide loosely-coupled rules that are separately compilable (at least for Spirit X3); and - are built around a flexible parse context object that has state added to and removed from it during the parse (again, comparing to Spirit X3). """ Zach

7 15

Re: [boost] Interest in new parsing library?
by Zach Laine 29 Dec '23

29 Dec '23

On Fri, Dec 29, 2023 at 12:40 PM Zach Laine <whatwasthataddress(a)gmail.com> wrote: > > On Fri, Dec 29, 2023 at 3:53 AM Andrzej Krzemienski <akrzemi1(a)gmail.com> wrote: > > > > > > > > czw., 28 gru 2023 o 22:05 Zach Laine via Boost <boost(a)lists.boost.org> napisał(a): > >> > >> I'm trying to gauge interest in a parsing library to replace > >> Boost.Spirit 2/Spirit X3. I'm also looking for endorsements. > >> > [snip] > > > > Hi Zach, > > Thank you for writing and sharing this library. I intend to test it on my mini-language early next year. > > That sounds useful. Please let me know how it goes. > > > For now, let me dig a bit about the high-level differences between Boost.Parser and Boost.SpiritX3. > > > > Your introduction mentions "a separate lexer" as a feature that Boost.Spirit is missing. > > How does that square with the entire section for Spirit.Lex in Boost.Spirit docs? > > No, I meant the other way around. *Boost.Parser* is missing that. > Sorry for the confusion. > > > "Boost.Parser aims to be a superset of Boost.Spirit". But Boost.Spirit is also a generator. > > Quite right! I forgot about that; this is another thing that > Boost.Parser is missing vs. Spirit. > > > You mention that "Spirit X3 has rules that do not compose well". I personally never experienced this. Is there an example somewhere that would illustrate this problem? > > The issue for me is that the attribute types generated by X3 parsers > and rules are loose. This is by design. So, say you have a parser > that parses strings, producing a range of char: > > auto p = /* ... */; > > If you then use p to parse into some std::string variable in the > middle of the parse, everything works as you'd expect. But if you use > p to parse into a different std::vector<char> variable in the middle > of the parse, that also works. This is not by itself a problem. > However, I had a problem with this semantic, because there was no way > to lock down the attribute type to a specific, concrete type. So if > at some point in the parse I had: > > auto p_p = p >> p; > > That may be used to generate a std::variant<std::string, > std::vector<char>>, a std::variant<std::vector<char>, std::string>>, > etc. These types are not interconvertible. So, If you made a mistake > somewhere in writing a semantic action or something, you get an error > stating that std::variant<std::string, std::vector<char>> is not > assignable to std::variant<std::vector<char>, std::string>> or > whatever, and figuring out where the error actually lies is very > difficult. If you have a Boost.Parser rule, it has a concrete > attribute type A, and so it always produces As. I find this much, > much easier to diagnose and debug when I make a mistake. > > > What is the recommendation of Boost.Spirit authors to the programmers that need to do parsing? Is Boost.Parser simply the newer and improved version, or do they have disjoint sets of use cases? > > I think Joel answered that above. > > > Personally, skimming through the docs, I find the feature of producing custom error and warning messages very attractive. This is what I was always missing from the parsing libraries. > > > > Thanks again for your effort. > > Thanks for taking a look! Also, I should mention that this is the > easiest library I have ever written, given its complexity. That is > true due to all the very nice design work that came before it, from > Joel, Hartmut, and others. > > Zach I somehow sent only this one reply to Andrzej only; echoing here. Sorry for the borked formatting. Zach

1 0

About weird behavior with boost.asio 1.63 on linux 2.6.15
by Scott Zhang 25 Dec '23

25 Dec '23

Dear All: I have been using boost over 10+ years. The asio library work quite well along with document. Today I get a problem: My development environment: gcc 4.7.3 OABI Linux 2.6.15 boost 1.63 The code looks like below: void myTask(const char *text) { sleep(5); /*for(;;) { //sleep(1); }*/ cout<<"text is "<<text<<endl; // cout<<"out"<<endl; } boost::asio::io_service ioService; ioService.post(boost::bind(myTask,"hello")); ioService.run(); I get 2 problems: 1. If I run program as above, the mytask get executed correctly, but as soon as it returns, the program get "illegal instruction", and program dies. If I call myTask directly, then everything is ok. So what's wrong with io_service.post 2. If I run ioservice within thread using threadpool.create_thread( boost::bind(&boost::asio::io_service::run, &ioService) Then after myTask executed, I get "Killed", using strace says program get SIGKILL. As the test code is so simple, I can't tell where goes wrong. I also noticed, if I don't call io_service.post, everything looks ok. And the threads, in linux ps, shows as many processes. I reread the document of asio 1.63, it says had beed tested against linux 2.4 and 2.6 kernel. so it should be ok. Please help. Thanks.

3 5

The master branch is now open for all changes
by Marshall Clow 14 Dec '23

14 Dec '23

Thanks to everyone who participated in the release! — Marshall

1 0

[release] Boost 1.84.0
by Marshall Clow 14 Dec '23

14 Dec '23

Release 1.84.0 of the Boost C++ Libraries is now available. These open-source libraries work well with the C++ Standard Library, and are usable across a broad spectrum of applications. The Boost license encourages both commercial and non-commercial use. This release contain two new libraries, numerous enhancements and bug fixes for existing libraries. For details, including download links, see <http://www.boost.org/users/news/version_1.84.0> You can also download directly from: <https://boostorg.jfrog.io/artifactory/main/release/1.84.0/source> To install this release on your system, see <http://www.boost.org/doc/libs/release/more/getting_started/index.html> Thanks to everyone who participated in this release. -- The Boost Release Team Marshall Clow, Glen Fernandes

1 0

[Fiber] aligned_storage is deprecated
by Niall Douglas 12 Dec '23

12 Dec '23

On Boost 1.81 with clang 18 in C++ 23 mode: ``` [build] /usr/include/boost/fiber/future/detail/shared_state.hpp:160:19: warning: 'aligned_storage<4, 4>' is deprecated [-Wdeprecated-declarations] [build] 160 | typename std::aligned_storage< sizeof( R), alignof( R) >::type storage_{}; [build] | ^ [build] /usr/include/boost/fiber/future/promise.hpp:28:22: note: in instantiation of template class 'boost::fibers::detail::shared_state<int>' requested here [build] 28 | typedef typename shared_state< R >::ptr_type ptr_type; ``` It could have been fixed upstream since, I haven't checked. Niall

1 0

[release] Boost 1.84.0 Release Candidate 1 is available
by Marshall Clow 12 Dec '23

12 Dec '23

The first release candidates for the 1.84.0 release are now available at: <https://boostorg.jfrog.io/artifactory/main/release/1.84.0/source/> The SHA256 checksums are as follows: cc4b893acf645c9d4b698e9a0f08ca8846aa5d6c68275c14c3e7949c24109454 boost_1_84_0_rc1.tar.bz2 cc77eb8ed25da4d596b25e77e4dbb6c5afaac9cddd00dc9ca947b6b268cc76a4 boost_1_84_0_rc1.zip 81a4d10075731966477276c47324ecd9cac02da49899d36c48eba66e40014f25 boost_1_84_0_rc1.7z a5800f405508f5df8114558ca9855d2640a2de8f0445f051fa1c7c3383045724 boost_1_84_0_rc1.tar.gz As always, the release managers would appreciate it if you download the candidate of your choice and give building it a try. Please report both success and failure, and anything else that is noteworthy. -- The Release managers

4 4

[scope] review
by Julien Blanc 07 Dec '23

07 Dec '23

Hi, This is my review of the proposed boost scope library. * Does this library bring real benefit to C++ developers for real world use-case? Yes. Scope guards are a well-known and widely used feature, nearly every code-base has some implementation of it. The same goes for raii resource wrappers. * Do you have an application for this library? I would consider using it for new projects. It does not, however, offer significant value at the time which would justify migrating from our homebrew scope guard. Worse, it misses some guarantees that makes it less fitted (but arguably it is more generic than what we use now). * Does the API match with current best practices? The API is copied from a TS proposal, for some parts. Most of my griefs against the API also applies to the TS. While I was not fond of it at the start, i think there's a clear value in differentiating scope_success and scope_fail. The first one is allowed to throw, the second is not. This, however, is not a strong requirement, but a general advice. It thus raises the concern about why not simply giving a boolean parameter to the scope_guard handler function, and letting it act accordingly. I suggest making the noexcept-ness of scope_fail executor a strong requirement. scope_success / scope_fail are not default-constructible, although they have a valid empty state. While there's a rationale behind this (it is a bit uncommon to create a scope guard that does nothing), it is a limitation for the sake of nothing. Initializing an empty scope_exit allows scope escaping (useful when initializing the scope guard in the body of an if statement). There will always be the possiblity to do it with an optional, so it just seems that forbidding default construction has no value. To check whether a scope_xxx is active or not, one has to call the active() method. To check whether a unique_resource has a value, one has to call the allocated() method. These are additions compared to the TS. While they may be useful, the names are inconsistent, both are new, also inconsistent with what other nullable types (optional, unique_ptr, variant...) uses. While this is common practice accross the c++ standard, I would not called this *best*. This leads to problems for generic code, and proposals such as this one: https://herbsutter.com/2022/09/25/something-i-implemented-today-is-void/ . In both case, i think at least an operator bool() should be provided, because it's a more widespread and common way to check for emptiness / valid state. scope_success / fails / exit takes an additional parameter (extension over TS), which is the execution condition. scope_success executes when the condition returns false. This is unintuitive, and something i will usually get wrong. While i get the reasoning behind, i think it tells more in favor of a scope_conditional<SuccessHandler, FailureHandler> interface. I'm not sure there's a lot of use-cases for a success_only or fail_only scope guard, especially since you can release in case of success, to avoid the cost of the failure check. About this customizable failure detector, i'm not found of the idea. The default, detection of a current exception, is a sane one. And it allows us to require FailureHandler to be noexcept. That's no longer the case if you allow customizing the failure detector. I'm not sure it's worth the tradeoff, since the failure detection could always be done inside the body of a standard unconditional scope_exit. Finally, i don't get the point of set_active. Sure, it's cheap to implement, but do we really want to encourage dynamically enabling scope guards ? Disabling a scope guard makes a lot of sense, that's what you do to disable a rollback scope_guard upon successful operation. Dynamically enabling it looks more like being hostile to the reader to me. In all cases, it's possible to get this behaviour with a boolean inside the scope guard callback, so this use case is already addressed. There's no need for it to be a first-class citizen. I like the idea of the lighter, unconditional, scope_final, which is missing in the TS. * Is the scope of the library well-defined? unique_resource and scope_exit are quite different beasts. They only have in common being RAII. But that also include unique_lock, unique_ptr, and nobody thought about making them part of the same library. I think it is a mistake to mix them. unique_resource takes three template parameters, the third one being a predicate telling if the resource is valid. I understand the wish to avoid storing an extra information, when there's a possibility to do without by storing an invalid value. But to me, it looks like a more widespread problem already solved, for example by https://github.com/akrzemi1/markable . It would be far better IMHO to get markable in boost, and have unique_resource specialized for markable<X> than replicating most of its functionality into unique_resource. * Is the documentation helpful and clear? The documentation is clear enough, although i could not find some information that i think should be emphasized more (ie, what the library guarantees in terms of pre / post conditions, noexceptness, etc). I don't like the way the reference section subitems are links to the file headers descriptions, and not links to class descriptions. I'd prefer something like: Reference * template< > class scope_exit * template< > class scope_final * ... instead of the headers, for which i usually don't care (i just need to know which header i need to include, and this information is provided in the class description). There are a few examples in the documentation that can be improved. There's always room for improvement, the overall quality is good. * Did you try to use it? What problems or surprises did you encounter? How much time did you spent? I made some small experiments with it. I spent around 6-8 hours reviewing the library and writing this review. Most of the time was spent reading the documentation and checking the behaviour. No problems and no surprises so far. * What is your evaluation of the implementation? The library is big (~2,5k lines of code), given what it offers. Most of this comes from the fact that the library offers a lot of customization points, maybe too much. Others have pointed that it reimplements some standard features, for the sake of not including some headers. I don't like this choice, but I don't know what's boost policy in this regard is. In all cases, a reimplementation with a name is better than nothing. In particular, there are some static_cast<T&&> which would advantagely be replaced by a detail::forward<T>, the latter being far more readable than the former. It's a shame that compact_storage is not provided as a facility: i guess there are several similar implementation across boost. But that's not the library's fault. Outside that, the organisation is clean, and the code largely readable, given that it needs to be compatible with C++11. * What is you final evaluation of the library? My current recommandation is to CONDITIONNALY ACCEPT the library. I think the following should be adressed, but i'd note that several of these comments also applies to the TS: - scope_success and scope_fail should be unified into a single call, taking two callbacks. This will make a slightly bigger object, but if you plan on detecting exceptions then you don't mind the extra cost anyway. And it will allow to evaluate the condition only once, so that may prove a gain in several situations. - scope_success and scope_fail should not take an execution condition. Exception detection should be enough for everyone, or else you should resort to scope_exit. - the failure handler (scope_fail) should be enforced to be noexcept - scope_exit should be default constructible (released by default, equivalent to moved-from state). That does not hold for scope_final, which is not releasable. - the active / released name fiasco should be adressed. At least provide operator bool(). - set_active should be removed. This is encouraging bad practice. - unique_resource should be moved outside the library. I consider that it belongs to a separate library, and i think it has some flaws in its design that should be addressed. Finally, many thanks to Andrey for proposing this library, and for quickly replying to the questions during the review. Regards, Julien

3 5

[release] Master Branch is closed for ALL changes
by Marshall Clow 07 Dec '23

07 Dec '23

Please do not commit anything to the master branch. Other branches are always fine. The 1.84.0 release is scheduled for next Wednesday, 13-December. — Marshall

1 0

[scope] Scope review starts on November 26th
by Дмитрий Архипов 07 Dec '23

07 Dec '23

Dear Boost community. The peer review of the proposed Boost.Scope will start on 26th of November and continue until December 5th. Boost.Scope is a small library implementing utilities defined in <experimental/scope> from C++ Extensions for Library Fundamentals v3 with a few extensions. Namely, the library contains: * A number of scope guards for various use cases. * A unique resource wrapper that automatically frees the resource on destruction. * Utilities for wrapping POSIX-like file descriptors in the unique resource wrapper. You can find the source code of the library at https://github.com/Lastique/scope and read the documentation at https://lastique.github.io/scope/libs/scope/doc/html/index.html. The library is header-only and thus it is fairly easy to try it out. In addition, the library can be used with Conan and vcpkg (see its README for instructions on how to do that). Finally, there's a single header version suitable for Compiler Explorer (https://raw.githubusercontent.com/Lastique/scope/single-header/include/boos…) As the library is not domain-specific, everyone is very welcome to contribute a review either by sending it to the Boost mailing list, or me personally. In your review please state whether you recommend to reject or accept the library into Boost, and whether you suggest any conditions for acceptance. Thanks in advance for your time and effort! Dmitry Arkhipov, Staff Engineer at The C++ Alliance.

16 83