August 2022 - Boost - lists.preview.boost.org

Library proposal - indexed_array
by Julien Blanc 12 Oct '23

12 Oct '23

Hi, I've written and released a (rather small) library which allows defining arrays with arbitrary index mechanisms (hence the name, indexed_array). This especially allows indexing with scoped enums, use non zero-starting indexes, or use sparse indexes. This is not a view, but an array in the sense that it owns the data, and provides the same guarantees and functionalities as std::array, such as triviality propagation, being able to reside in read-only program sections, etc. It also supports multidimensional indexing, and provides an indexed_span template class to access lower dimension content (which can be used outside indexed_array). Finally, it provides an opt-in safer way to initialize the array content, which is especially useful when using complex and/or changing indexing schemes (like based on externally generated files). The library is available here: https://github.com/Julien-Blanc-tgcm/indexed_array And the doc is here: https://julien-blanc-tgcm.github.io/indexed_array/ It requires C++17 and mp11, and has first-class support for describe-d enums. If there is some interest, i'd like to propose it for inclusion in boost. So i'm looking for people who would endorse it. Thanks and best regards, Julien

8 24

Boost.Text is looking for a (mini-)review manager
by Zach Laine 07 Dec '22

07 Dec '22

I've made all the recommended changes to Boost.Text that came up in the review. Glen, the previous review manager, recommended a mini review of Boost.Text once I'd made those changes. If you'd like to manage the mini-review, please let me know. For everyone, including potential reviewers and review managers, here are the major changes: * All the specific review feedback issues have been addressed. * The string layer is now gone. * The text layer got a major re-work. It is now a set of template-based adaptors (basic_text<>, vs. a text typedef. etc.), which lets everyone get what they want. You can change the underlying storage type (including whatever allocator you prefer), select UTF-8 or UTF-16, and change the normalization form. Glen recommended I drop this layer, but the library is way less useful without this stuff, and these changes address all the concerns raised about the text layer in the review. The updated docs can be found here: https://tzlaine.github.io/text/doc/html/index.html Oh yeah, and I added concept constraints to the whole library when you build in C++20 mode. Zach

17 46

[Units] Incorrect definitions for torque, newton_meter
by Albert Dvornik 04 Sep '22

04 Sep '22

#include <iostream> #include "boost/units/systems/si.hpp" #include "boost/units/systems/si/io.hpp" using boost::units::quantity; using boost::units::si::torque; using boost::units::si::newton_meter; using boost::units::si::newton; using boost::units::si::meter; using boost::units::si::radian; int main() { // works fine: quantity<torque, double> tau0 = 0.0 * newton_meter; // SURPRISE: compilation fails because newton_meter is not the // same as newton*meter! // quantity<torque, double> tau1 = 0.0 * newton * meter; // Compilation succeeds even though the units on the RHS do not // represent torque but torsional stiffness! quantity<torque, double> tau2 = 0.0 * newton * meter / radian; // newton * meter should give us a quantity with units of Newton- // meters, i.e. m^2 kg s^-2. What this actually prints out is "0 J", // which is reasonable since dimensionally J is the same as N m. std::cout << boost::units::to_string(0.0 * newton * meter) << std::endl; // One might hope that newton_meter would be the same thing as // newton * meter. Instead, the result is "0 m^2 kg s^-2 rad^-1" std::cout << boost::units::to_string(0.0 * newton_meter) << std::endl; // The "torque" unit is also messed up in the same way. std::cout << boost::units::to_string(torque()) << std::endl; }

5 8

additional math-functions
by Gero Peterhoff 04 Sep '22

04 Sep '22

Hello, I would like to add several missing math functions in boost or specialize/overload existing ones for more types. These should be constexpr if possible. My problems with the concrete implementations are: - Which C++ standard to use C++11/14/17/20/23 ? - Should these be implemented in ccmath and math ? a) ccmath requires C++17 (eg "if constexpr"), so it is not possible to simply call the ccmath version in math, since math requires C++11. b) On the other hand, the math functions are already included in the standard (since C++11), so it wouldn't be necessary to continue to provide them (especially since they are not constexpr). Where and how exactly and with which C++ standard can I add the functions ? thx Gero

4 10

Password_protected .csv file
by Daggumati, Rajesh 31 Aug '22

31 Aug '22

Hi Team, We can create an .csv file using std::ofstream() function. But my project requirement demands to create a .csv file with password protected. Ofstream has open() function to create a file but how to create a file with password protected? Please help me on this. Regards, Rajesh D Confidential - Company Proprietary Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.

4 4

Boost warning philosophy
by Scott Johnson 30 Aug '22

30 Aug '22

Hello, We are migrating a legacy codebase (embedded system though a PC architecture) from Centos 7.x, using various boost 1.5x components and GCC 7.3, to Ubuntu 22.04 and GCC 11.2 (the compiler that ships with this version of Ubuntu). As our application needs things that don't come with the "stock" Boost version that Canonical packages for 22.04, specifically Python 2.x support, I have downloaded and configured Boost 1.80 for our application to use. Unfortunately, our code development standard is -Wall -Werror (turn on all warnings and treat them as errors), with exceptions to -Wall only allowed grudgingly. And this version of Boost seems to break things, with all sorts of warnings being thrown that we simply don't see on the older version, distro, and toolchain. One of the biggest offenders seems to be things like #if _MSC_VER when this preprocessor symbol is not defined--GCC likes to warn that the symbol is being "defaulted" to zero in this context. (Defining it as 0 causes all heck to break loose, as there is also plenty of #ifdef _MSC_VER in the code). I've also seen warnings for overloaded virtual functions being hidden (a warning I would prefer not to disable), null pointer dereferences (ditto), and local variables being shadowed (ditto) from various Boost components. This is on production builds with -O2 optimization level. What is, if anything, the Boost policy towards compiler diagnostics of this sort? Is there a set of warnings we should be prepared to ignore if we use Boost with a more modern (and pickier) compiler? Should we perhaps consider another version of GCC? An older, more mature version of Boost (we don't need any specific features in the current one)? Thanks, Scott -- engineer_scotty (no, not that one) If life gives you lemons, drink Hefeweizen

3 2

[filesystem] becoming worse than the standard?
by Vinnie Falco 29 Aug '22

29 Aug '22

I see this in the commit log for boost.filesystem: Deprecated path construction/assignment/appending from container types. Users are advised to use string types and iterators instead of containers to construct/assign/append to paths. In v4, the support for containers is removed. This can't be right now, can it? Because std::filesystem supports path construction from anything meeting the requirements of Source which is kind of range-like. Boost.URL and its downstream libraries HTTP-Proto, HTTP-IO, Websocket-Proto, and Websocket-IO are depending on the container-based interface to achieve reasonably decent syntax and semantics despite the limitations of the path API (which are the fault of the standard and not Boost). If std::filesystem::path supports container-based parameters to assign() and append() then why is boost::filesystem::path removing it? Thanks

11 45

Re: [boost] [url] Andrzej's notes
by Vinnie Falco 25 Aug '22

25 Aug '22

On Mon, Aug 22, 2022 at 11:48 PM Andrzej Krzemienski <akrzemi1(a)gmail.com> wrote: > Like in this example from the docs: > https://www.boost.org/doc/libs/1_80_0/libs/beast/doc/html/beast/quick_start… > > http::request<http::string_body> req{http::verb::get, target, version}; > req.set(http::field::host, host); > req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING); > > At no point do I have to see or provide a full URL. "target" is the URL there, it came from the command line. It is a relative-ref. >> * modifiers which take un-encoded inputs have a wide contract: all >> input strings are valid >> - however the url might need to reallocate memory to encode the result > > Does set_port() taking a string_view fall into that category? > Unlike other parts, port has special requirements on the string > contents that cannot be satisfied by pct-encoding. Yeah, we have a problem here. It sounds like we need to design an extra set functions. Maybe: url_base& url_base::set_port( string_view ); // throws result<void> url_base::try_set_port( string_view ); // returns result what do you think about that? > If you used result<>, you would lose the ability to chain the setters. Yeah... well, I think I'm OK with that. > As an alternative, you could say that function set_port() has a precondition: > the input string has to represent a number, the caller is responsible for that, > and set_port() performs no validation, and therefore throws nothing. But I > guess that would violate one of the design goals of the library: "securely, > including the case where the inputs come from untrusted sources". My original thinking was that untrusted sources would only be presented to parse functions. But our dialog has convinced me that we should treat the parameters to modification functions as untrusted as well. True, in some situations we will be performing unavoidable, needless re-validation. But I think it is the right tradeoff, as offering the stronger invariant has more value for users. Besides, there are workarounds for assembling a URL which trade back performance in exchange for weaker invariants. > Just to clarify what I mean: https://godbolt.org/z/qY3Y76fd9 > > urls::string_view s = "https://path?id=42&id=43"; > urls::url r = urls::parse_uri( s ).value(); > > Should the above input string not cause the invariant to be broken? (because param 'id' appears twice.) No, because the query is just a string. The interpretation of the query as "params" (an ampersand delimited list of name=value pairs) is an HTTP thing (and it has also spread to some other domains). When the query is used this way, duplicates are allowed. So the invariant is preserved in this case. It is always a valid URL. It might not be valid for custom schemes though. I could invent the "boost" scheme which requires that keys used in query parameters are unique. But there is no way for Boost.URL to enforce this (see my previous message regarding "mailto"). Users are provided with the tools to further specialize the generic URLs into custom schemes in a way that can preserve scheme-specific invariants. Thanks

5 11

Re: [boost] Boost.URL review
by Zach Laine 24 Aug '22

24 Aug '22

On Mon, Aug 22, 2022 at 9:41 AM Zach Laine <whatwasthataddress(a)gmail.com> wrote: > > On Sun, Aug 21, 2022 at 6:47 PM Vinnie Falco <vinnie.falco(a)gmail.com> wrote: > > > > On Sun, Aug 21, 2022 at 4:03 PM Zach Laine <whatwasthataddress(a)gmail.com> wrote: > > > ... > > > > It wouldn't be a proper review without at least one person forgetting > > to use "Reply All" :) So here it is, from Zach. I have quoted his > > message: > > > > > This makes sense. The larger issue I'm raising is that a user is > > > going to have a harder time remembering this, if the names don't > > > match. A way to mitigate that is to make this naming rationale > > > explicit, so that user > > > expectations will adjust. I didn't get that this was the pattern from > > > the documentation, though I think you did say it. A call-out > > > specifically noting this would porbably be enough. > > > > Yeah we can do that. > > > > > Hm. By lazy flat_map, do you mean that the positions are computed one > > > step at a time during iteration? is that why they have > > > forward_iterators? > > > > Yes. operator++ and operator-- intelligently find the next param or > > segment using a non-validating parse algorithm (since we know that the > > underlying character buffer has already been validated). > > > > > If so, please document that. It might make a > > > pretty big difference whether I decided to keep using a params object > > > that I need to update a lot, or copy everything over to a proper > > > flat_map or vector to do my work. > > > > Yes I agree, but I thought we did document that. Here, no? > > > > <https://master.url.cpp.al/url/ref/boost__urls__params_encoded_view.html#url…> > > > > We could probably add something here too: > > > > <https://master.url.cpp.al/url/ref/boost__urls__params_encoded_view/iterator…> > > Right, and I understood how those worked. However, I had no idea that > the owning containers were based on the same lazy approach (unless I > misunderstand what you said in this thread...). > > > > This works for an immutable container. It is *very surprising* for a > > > mutable one. The rest of my notes about the containers are predicated > > > on a false assumption that mutability implied non-laziness in the > > > container itself. I think all of these container would be better off > > > referred to as views, and made immutable. All you really need to > > > support is begin/end, plus maybe key-lookup. I suggest you drop the > > > mutability entirely. With mutability, these containers are trying to > > > do two things at once. > > > > Well, that's another way to do it and it certainly would remove > > complexity and simplify the names. Right now we have 8 containers: > > > > segments > > segments_view > > segments_encoded > > segments_encoded_view > > params > > params_view > > params_encoded > > params_encoded_view > > > > All of these containers reference the underlying character buffer of > > the URL (although you can also construct the views from separately > > parsed string views but that's basically the same thing in terms of > > the ownership model being reference-like). > > > > If we remove mutability as you suggest, then we can cut the number of > > containers down to 4: > > > > params_view > > segments_view > > params_encoded_view > > segments_encoded_view > > > > But this leaves us with a problem - how do you modify individual path > > segments and query params? > > Easy -- you just copy them into a flat_map, vector, or whatever. The > 90% use case is just to look at the result of the parse. Also making > the parsing library responsible for mutations you do to the data later > seems unnecessary. If you provide parsing -- including the lazy views > -- and then writing of URLs, and the writing can be done via a generic > container, that's all your library needs, IMO. To be clear: > > Parsing -> lazy_containers -> user mutations outside of Boost.URL -> > writing URLs > > That last step only needs to understand the different kinds of data it > might write, abstractly. That abstraction might be a params_view, a > flat_map, or a std::vector<std::pair<strd::string, std::string>>. > > > How do you do position-based insert and > > erase of elements using an iterator? These features seem kind of > > useful, I'm not sure I want to lose them. > > You won't lose them, because the STL exists. > > > > > Did you mean that it "doesn't" have proper error reporting? Where do > > > > you see this lack of reporting (if that's what you meant)? > > > > > > By "proper" I mean that if the parse fails it should provide an > > > end-user-friendly diagnostic, and optionally quote the line and put a > > > caret under the position that the parse failed (a la the Clang > > > diagnostics). Your non-end-user error reporting is fine. > > > > Oh... hmm.. well, each rule does leave the iterator out-param at the > > spot where an error occurred. But I'm not quite sure how to emit the > > diagnostic - surely you aren't suggesting we write to std::cout or > > std::cerr? Are we using curses or something? Or maybe just plain ASCII > > art? How exactly would this look? > > The diagnostics go to some optional output channel, which might be a > stream or file or logger. If you don't set the output, the diagnostic > is never generated. You set this output in the call to parse*(), as > an optional param -- usually a std::function<void(std::string const > &)>. > > > Thanks to Peter's work on boost::system::error_code, when a parse > > error occurs the error is augmented with file name, line, and function > > name information. And I have worked on adding visualizers so these are > > nicely presented in the debugger. So at least we have that. > > That's really nice for programmers, but it leaves the programmer with > a lot of work to do to present the error to the user. > > Zach Forgot to reply-all *again*. Zach

4 11

boost.url is accpted into boost
by Klemens Morgenstern 24 Aug '22

24 Aug '22

Boost.URL is ACCEPTED into boost without any conditions. The review recommendations were 6 ACCEPT, 1 CONDITIONAL ACCEPT and 1 REJECT. Many points were raised as part of the review, based on which I recommend doing the following: - reviewing which types/functions in the lower-level API are exposed and which ones are not (Ruben Perez' condition) - reviewing pct_encoded_view & encoded_path (as mentioned by Дмитрий) - adding more verbose docs without relying on knowledge of RFC3986 on the user side (mentioned by Richard) - adding more simple examples (recommended by Jon Kalb) - consider adding IRI support in the future (recommended by Rainer) Alan & Vinnie have been very responsive and read all the reviews, which makes it unnecessary to summarize everything. There is only one real point of controversy, namely the only reason for rejection written by Phil Endecott: > More fundamentally, I'm not convinced that this allocation-avoiding > view-rich API style is best for a general purpose vocabulary type > like URL. URL should be a "Keep It Simple" type, accessible to > C++ beginners, with straightforward lifetime semantics. Compare with > std::filesystem::path, which does not try to optimise by actually > being a view. > > Because of these concerns about the basic API design, I believe that > the library should be REJECTED. This argument has however been directly addressed in other reviews, with Seth considering it as an appropriate choice: > Like some of the earlier reviewers, I also have some concerns over introducing > non-owning vocabulary type. I do appreciate the design choice though, because I > think it ultimately fits the Boost ecosystem best - favoring > performance/versatility over ease of use. While Zach Laine considers this as the correct choice: > Overall, I quite like the design. Lazy, lazy, lazy. The fact that > only views into the underlying sequence are returned from the parse is > great, and the fact that there are both owning (url::url and > fixed-capcity owning url::static_url) and view versions of the URL > representations is exactly what I would need as a user. For my decision, the first factor is that more people approved of this particular design choice than disapproved. Secondly it seems to me that the C++ ecosystem is moving towards more view-support. The potential pitfalls can be addressed by coding guidelines (e.g. don't use auto) and do not need to be addressed by constraining libraries. Thus in finding a result, I did not consider Phil's argument to outweigh the other reviews which were considering it an acceptable choice. Other issues were the documentation, the confusion about encoded and decoded data and IRI support. Neither did lead to a rejection, so I'll leave it up to Alan's & Vinnie's discretion to review & address all the raised issues and to engage in further discussion on the mailing lists. Thank you Dmitry, Jon, Phil, Rainer, Richard, Ruben, Seth and Zach for your reviews and Alan & Vinnie for providing yet another boost library.

2 1

Fwd: Review of Boost.url
by Klemens Morgenstern 24 Aug '22

24 Aug '22

Forwarded from an email I received directly. ---------- Forwarded message --------- From: Jon Kalb Date: Mon, Aug 22, 2022 at 11:39 AM Subject: Review of Boost.url To: *- Does this library bring real benefit to C++ developers for real world use-case?* Boost currently contains at least three parsers (Regex, Spirit, and Xpressive) and most application seem to make do with std::string as a URL container implementation. That said, URL parsing is a non-trivial undertaking if done right and need to be done with in a surprising number of applications (because the number of applications that simply store and/or process URLs far out numbers the number of applications that directly use URLs). Having to re-invent the wheel of URL parsing in each of these apps results in both wasted time and poorly implemented solutions. URLs are both general enough and broadly used enough that they a library such as this will be a vocabulary library. *- Do you have an application for this library?* No. * - Does the API match with current best practices?* It appears so. The designed guidelines for the library are the correct ones. The issues that would cause a library such as this to not become widely used would be: - Language version dependency is too recent. - Module dependencies - Slow build times - Relies on dynamic allocation - Relies on exceptions - Not header-only - Not suitable for minimal devices/environments This is pretty much the design requirements for the library. In particular the approaches for minimizing and even avoiding dynamic allocations and functioning with or without exceptions will remove most what would be common stumbling blocks for this library. * - Is the documentation helpful and clear?* The documentation does appear to be thorough with a bit of a gentle introduction. I’d like to see more examples of getting, setting, and removing parameters and a better explanation of how escaping is handled. I suspect there is some magic there that I’d like to understand better. The “Help Card” is a brilliant design but the text in the URL syntax at the top renders improperly in my browser. * - Did you try to use it? What problems or surprises did you encounter?* I only did a read-through of the documentation, but it was thorough enough that I found (and submitted) a few minor corrections needed in the docs. * - What is your evaluation of the implementation?* I didn’t examine the implementation. *Some additional thoughts/questions for the author:* - I read Ruben’s review and tended to agree with and appreciate his thoughts. I did however disagree about exposing the low level parsing tools. If they were not a complete general implementation due to the fact that the library only needed some use cases, so they didn’t represent a completely implemented set of functionality, then I’d be skeptical and wonder if they were well tested. But it seems to me that the tools involved represent a valid and valuable contribution to the community. - I wonder if an operator<() should be defined so that urls could be used as keys out-of-the-box. - I wonder if more scheme constants should be provided. Probably not for the entire official list, but ‘ldap’, ‘mailto’, ‘news’, and ‘telnet’ might be useful enough to include. - This is out of my area of expertise, but I wondered if there might be platforms where instead of using “char” the library should be built on “signed char” or “unsigned char.” I think the URL library should be added to the Boost Libraries. I think will be valuable to the community and the level of professionalism in its design will reflect well on Boost. Thanks. Jon

2 1

boost.url review period has ended
by Klemens Morgenstern 24 Aug '22

24 Aug '22

Based on the amount of reviews I don't think we need an extension to make a clear & well-founded decision, which I hope to be able to announce soon. I would encourage those who didn't find the time to write a review to engage in discussion with Alan & Vinnie who have been very responsive to criticism. Thanks to everyone who participated in the review and discussion.

1 0

[url] Andrzej's notes
by Andrzej Krzemienski 23 Aug '22

23 Aug '22

Hi Everyone, It is somewhat unfortunate that the review happens during the vacation period, therefore I will not be able to devote the amount of time that the library deserves for the review. There is a procedure for extending the review period for a couple of days, but even if it were engaged, I doubt it would have bought me sufficient time. So let me instead offer some loose thoughts. Am I familiar with the problem domain, or do I have a use for this library? My understanding of an URL is that it is a text that I type into my browser (or other tools) that starts with either "http" or "https". I use URLs indirectly by using Boost.Beast, but Boost.Beast offers a higher-level interface for providing parts of URL, and even if this library is accepted, I would use the higher-level interface of Boost.Beast. This is not to claim that there are no uses for the proposed library. It is just my experience. Obviously, Boost.URL fits into the trend of providing many types with strong invariants, which I am very fond of. The fact that you have to set host, port and path separately avoids many bugs: you just cannot create an invalid URL. There is one thing that I do not understand though. I read that: 1. The library is tailored for environments that do not use exceptions 2. url guarantees that it maintains its invariant of always storing a string that represents a valid url. So given the following code executed on a platform with exceptions disabled, what happens? void test(url my_url) { my_url.set_port("twenty-one"); } is it going to kill the program? If so, is this considered a satisfactory solution? I use exceptions in my job, so this is a rather theoretical issue for me, but if I were using urls in no-exceptions environment, I think I would expect an interface like: error_code ec; my_url.set_port("twenty-one", ec); Or something similar. Next, the interface of parse_uri(). This library offers three types for storing (and owing) the characters representing a url: * url * static_url * an unnamed type returned by url_view::persist(). This is good, as in C++ the way the data is stored and laid out is in fact part of the contract. For the same reason we have multiple string types that differ by how memory is allocated, if it is stack allocated, how big the SSO is, what is guaranteed for swap. As a consequence we have a practical engineering issue -- not confined to URLs -- if there is N types storing the same logical data in N different ways, and I have a function returning this logical data, I do not want to offer N functions with N names. I may add a template, or I may use a universal type that can cheaply convert to the N types. How do I do that if I do not want to put templates in the interface of my library? The choice made by Boost.URL may not be optimal, but it is certainly a good starting point in the pursuit of the solution to this problem. I see url_view not as a view but as a universal type for cheaply constructing different containers storing URLs. Of course, this approach is incompatible with the "almost always auto" philosophy, which some people now promote to simply "always auto". My personal opinion is that this AAA philosophy is harmful from the outset and it begs for problems and bugs. If someone adapts the AAA one must accept the responsibility for the problems like the one with dangling references. Next, classes url_base and url_view_base are part of the library's public API. Why? You can keep them as bases but not advertise them in the library documentation. Do you expect any use cases for them? Next. the design in url classes is that it is a mixture of the generic URL as defined in the RFC, and the HTTP-speciffic stuff, such as paths and parameters. It is never the case that I need both of this worlds at the same time: I either need a generic URL that has a password but doesn't have path or params, or I need a HTTP-like URL, where I do not need a password. So if we think about strong types with strong invariants, I never get as strong a type as I would need. For instance, I would never want to pass the password in URL when using Boost.Beast, so having member set_password would only be a potential for bugs. Am I suggesting anything? The only use case that I have for this library is HTTP-like URLS. I would appreciate a http::url class that is tailored for http or even REST protocols. Regarding the docs, while I agree with others that some examples of use cases, such as percent encoding, would be desired, I also note that the documentation stands up to the high standards of of Boost libraries: not only could I figure out the architecture and how things work, but I was able to learn a lot about the problem domain, the URLs, URIs, IRIs, design trade-offs. I really appreciate the effort the authors have put in making the documentation this good. I have two recommendations, though. One, the picture that associates IQs with facial expressions: I strongly recommend that it is removed. It does not bring any technical value, but it seems to be sending a hostile message. Two, the colouring of different parts of URL syntax, as in https://master.url.cpp.al/url/containers.html#url.containers.generic_syntax: it help to quickly grasp things, but it also bring some confusion. For instance, I cannot tell if the double-slash is part of authority, or if the colon is part of the scheme. The colors suggest that they are, but the member functions tell otherwise. I have spent maybe like 30 hours studying the docs, superficially studying the implementation and tried to find if the two match, played with the toy examples in godbolt (gcc 12.1): https://godbolt.org/z/Tbsqrd77M Found a number of bugs and reported them in GitHub. The authors were very responsive and very quickly addressed practically all of them. Every implementation of every Boost library is difficult to read, in my opinion. Boost.URL matches the Boost standard in this respect. I could understand what the functions do, and what they do wrong. I found the solution in function url_view::persist() very innovative: you have one allocation instead of three. I didn't have time to play with path and param views. They seem fine on the surface, but I have a feeling that they would be clumsy to use in practice. I observed that params allow duplicate keys? Is it legal? I am hesitant to give a recommendation for acceptance. I would rather have a longer discussion than a fast decision. I also want to see how the authors respond to these notes. Finally, I want to thank the authors for writing and sharing this library. It is clear that a lot of effort and thought has been invested in it. Regards, &rzej;

6 9

boost.url review
by Christopher Bläsius 22 Aug '22

22 Aug '22

Hi, I'm not a boost developer, but as someone who has to deal with URLs (for calling or implementing Web-APIs) at his job frequently, I want to give my two cents as a most likely user of this library. I hope this is OK (I won't give a recommendation to accept or reject though). I spent about 2.5h integrating the library into our program and tried some simple parsing and constructing of urls. I used MSVC2022 which worked without problems. > - Does this library bring real benefit to C++ developers for real > world use-case? Yes definitely. I believe probably every programmer has written a (mostly crappy) URL-parser before, and I would be more than happy to replace our parser with this library. > - Do you have an application for this library? Yes. Besides parsing incoming HTTP-Requests, I would use it to programmatically construct urls to call external Web-APIs. One of the more common scenarios in our applications is the construction of urls after the user provided some login credentials and often hosts/port combinations. Also a prefix path is often needed (for example to differentiate between a demo or beta service and the real production api). I feel like the current API design lacks a bit in this regard. To construct a new url i did the following: auto url = boost::urls::url{}; url.set_scheme("https") .set_host("special-api.com") .set_port("443") .set_path("/prefix/api/v1/"); url.segments().push_back("applications"); url.segments().push_back(applicationId); url.segments().push_back("devices"); url.params().append("since", "2022-01-01"); The url::set_... methods seem to follow a fluent api design which is nice, but unfortunately they return url_base&. So I can't construct a url inside a function call like: httpClientGet(url{}.set_host(host).set_path("some/path")) Being able to add segments is great. Personally I'm a big fan of overloading the /-operator to append paths like in the filesystem library, because I think this results in more readable code. The example of above could become: httpClientGet(url{host} / "some/path") I really like that the library offers different parse_ functions, so an user can choose the appropriate one depending on the context. But it is a shame that there are no semantic types to go along with them. As mentioned above we often have the case that we have to construct a url by combining a prefix path with the actual path of the resource. As this prefix path has to be absolute and the resource path relative I want to use 2 distinct types for them. It seems to me, the resolve() function has a similar problem. The documentation states that the base url has to satisfy the absolute-URI grammar. Wouldn't it be great to have a distinct type for this? BTW why is there only an authority_view and not an owning container? Seems inconsistent to me. Its nice that there are functions to manually do percentage encoding, but I would prefer (additional) simpler functions which just return a std::string. It would be nice if the library also provided the default port numbers for the schemes. I know the documentation states that this is not part of the URL-protocol, but I gonna need that information if I use the library. > - Is the documentation helpful and clear? Mostly yes. Reading the "Quick Look" page is enough to get started with using the library which I appreciate. I also like the help card, which would be greatly improved by adding links to the reference pages. I skipped / skimmed the Customization and Concept pages. I would recommend not to start your example page with a 700 lines implementation of magnet links, that's super intimidating. Something I missed was a simple example showing that the parse functions also accept unknown schemes like opc.tcp:// for example. Thanks a lot for all your work on providing useful high quality libraries for everyone! Best Regards, Chris

3 2

URL Review
by Phil Endecott 22 Aug '22

22 Aug '22

My review of the proposed Boost.URL library follows. Unfortunately I have been away for most of the review period and I have not been able to spend as much time on this as I would have liked; in particular I have not tried to run any code. But I have looked at the source and documentation. In previous email discussions a few months ago I complained about the dangerous use of non-owning view types in the library interface, and I see that these issues are still present. Specifically, quoting the "Quick Look" docs: result<url_view> r = parse_uri( s ); url u = parse_uri( s ).value(); That's fine for C++03, but since C++11 we have auto, and this is where it becomes dangerous: auto r = parse_uri( s ); There's no indication here that r is a non-owning view and that the caller is responsible for keeping s alive for the lifetime of r. Such errors could be difficult to detect; the code may appear to work correctly much of the time if the storage for s is not re-used. auto u = parse_uri( s ).value(); That's even worse, because we're calling a method called value() but what it returns is not actually a value, it's still a view. A safer API design would, at a minimum, give the parsing function a name with "view" in it. The default, shorter-named function should return a value type. More fundamentally, I'm not convinced that this allocation-avoiding view-rich API style is best for a general purpose vocabulary type like URL. URL should be a "Keep It Simple" type, accessible to C++ beginners, with straightforward lifetime semantics. Compare with std::filesystem::path, which does not try to optimise by actually being a view. Because of these concerns about the basic API design, I believe that the library should be REJECTED. Following are some comments about other more minor issues that I have noticed: I see that there is a method called persist(): std::shared_ptr< url_view const > persist() const; This function returns a read-only copy of the URL, with shared lifetime. The returned value owns (persists) the underlying string. The algorithm used to create the value minimizes the number of individual memory allocations, making it more efficient than when using direct standard library functions. Hmm, so url_view doesn't own the underlying storage, except when it does as a result of this mysterious method. What's the rationale for this? (Have I missed some documentation?) What is sizeof(url) or sizeof(url_view)? I think it must be about 160 bytes, since you store offsets to each of the components. I'm not convinced that this is the best approach. I would be inclined to determine the offsets each time they are needed by the accessor methods. In the constructor, parse to validate the syntax but don't store any offsets. I believe that in the case of the path segments and the query parameters, you are not storing offsets (because the number is variable, and that would need allocations); why not adopt that approach for the other offsets? Making the objects smaller would probably have measurable performance benefits. In your non-view url object, you seem to allocate storage using new char[]. Why not use a std::string? One benefit of a std::string is SBO. Another is that you could provide a move constructor from a string. I find the names of the encoding/decoding views confusing. I would expect a "decoded_view" to refer to underlying encoded data and return equivalent decoded data when it is read from, but you seen to call that an "encoded_view". In your query parameter API, you expose something that looks like a standard associative container except that you call the key and value "key" and "value" respectively, rather than "first" and "second". While I think many of us would agree that those are better names, it does make it more difficult to use this container with generic code than it would be otherwise. I see that you also have a bool called "has_value" in the value_type. I think this is to handle this: ?a=1 // has_value == true && value == "1" ?a= // has_value == true && value == "" ?a // has_value == false Is that correct? Again having this third member in the container's value_type makes generic code more difficult. I would be inclined to use a std::optional for the value instead. In the past I have had situations where it was necessary to specify the case for hex letters in %-encoding; please consider adding this to pct_encode_opts. The Magnet Link example needs a paragraph at the start to say what a Magnet Link is. Regards, Phil.

5 6

boost.url review
by Klemens Morgenstern 22 Aug '22

22 Aug '22

Hi all, the formal boost review of the boost.url library starts today, the 13th of August and will last until and including the 22nd of this month. The library has been developed by Vinnie Falco and Alan de Freitas. The master branch is frozen during the review and can be found here: https://github.com/CPPAlliance/url The current documentation can be found here: https://master.url.cpp.al/ Boost.URL is a portable C++ library which provides containers and algorithms which model a "URL" and understands the various grammars related to URLs and provides for validating and parsing of strings, manipulation of URL strings, and algorithms operating on URLs such as normalization and resolution. Please explicitly state that you either *accept* or *reject* the inclusion of this library into boost. Also please indicate the time & effort spent on the evaluation and give the reasons for your decision. Some questions to consider: - Does this library bring real benefit to C++ developers for real world use-case? - Do you have an application for this library? - Does the API match with current best practices? - Is the documentation helpful and clear? - Did you try to use it? What problems or surprises did you encounter? - What is your evaluation of the implementation? More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Reviews can also be submitted privately to me, and I will not disclose who sent them. I might however cite passages from those reviews in my conclusion. Thanks to Vinnie & Alan for providing us with a new library & thanks for all the reviews in advance. Klemens

9 20

Boost.URL review planned for 13-22 August 2022
by Mateusz Loskot 22 Aug '22

22 Aug '22

Folks, Klemens Morgenstern kindly volunteered [1] as the Review Manager (thanks Klemens!) for the Boost.URL [2], so we have planned the review shortly after the upcoming release of the Boost 1.80. Check the Review Schedule [3] for details! [1] https://lists.boost.org/Archives/boost/2022/06/253225.php [2] https://lists.boost.org/Archives/boost/2022/04/252814.php [3] https://www.boost.org/community/review_schedule.html Best regards, -- The Boost Review Wizards Mateusz Loskot

3 4

boost.url review ends today
by Klemens Morgenstern 22 Aug '22

22 Aug '22

Today is the last day to submit your review for boost.url. Thanks in advance!

1 0

bug - boost::math::ccmath::copysign not work
by Gero Peterhoff 22 Aug '22

22 Aug '22

Hi, I just see that boost::math::ccmath::copysign doesn't work. https://godbolt.org/z/9af1ME9fT cu Gero

5 11

Re: [boost] Boost.URL review
by Vinnie Falco 21 Aug '22

21 Aug '22

On Sun, Aug 21, 2022 at 4:03 PM Zach Laine <whatwasthataddress(a)gmail.com> wrote: > ... It wouldn't be a proper review without at least one person forgetting to use "Reply All" :) So here it is, from Zach. I have quoted his message: > This makes sense. The larger issue I'm raising is that a user is > going to have a harder time remembering this, if the names don't > match. A way to mitigate that is to make this naming rationale > explicit, so that user > expectations will adjust. I didn't get that this was the pattern from > the documentation, though I think you did say it. A call-out > specifically noting this would porbably be enough. Yeah we can do that. > Hm. By lazy flat_map, do you mean that the positions are computed one > step at a time during iteration? is that why they have > forward_iterators? Yes. operator++ and operator-- intelligently find the next param or segment using a non-validating parse algorithm (since we know that the underlying character buffer has already been validated). > If so, please document that. It might make a > pretty big difference whether I decided to keep using a params object > that I need to update a lot, or copy everything over to a proper > flat_map or vector to do my work. Yes I agree, but I thought we did document that. Here, no? <https://master.url.cpp.al/url/ref/boost__urls__params_encoded_view.html#url…> We could probably add something here too: <https://master.url.cpp.al/url/ref/boost__urls__params_encoded_view/iterator…> > This works for an immutable container. It is *very surprising* for a > mutable one. The rest of my notes about the containers are predicated > on a false assumption that mutability implied non-laziness in the > container itself. I think all of these container would be better off > referred to as views, and made immutable. All you really need to > support is begin/end, plus maybe key-lookup. I suggest you drop the > mutability entirely. With mutability, these containers are trying to > do two things at once. Well, that's another way to do it and it certainly would remove complexity and simplify the names. Right now we have 8 containers: segments segments_view segments_encoded segments_encoded_view params params_view params_encoded params_encoded_view All of these containers reference the underlying character buffer of the URL (although you can also construct the views from separately parsed string views but that's basically the same thing in terms of the ownership model being reference-like). If we remove mutability as you suggest, then we can cut the number of containers down to 4: params_view segments_view params_encoded_view segments_encoded_view But this leaves us with a problem - how do you modify individual path segments and query params? How do you do position-based insert and erase of elements using an iterator? These features seem kind of useful, I'm not sure I want to lose them. > > > "Remove" is not really a term used in the STL. I think your > > > "remove*()" container members should be "erase*()" instead. > > > > I'm not sure where you're seeing this? There are no "remove" functions > > in these containers. > > params::remove_value() is one. I think(?) there are others. Oh. Well... remove_value() only removes the value part of the param. It does not erase the entire param. There's no such thing as insert_value. The javadoc is wrong unfortunately (sorry about that). We do use the term erase() when the operation removes the entire element. > > Did you mean that it "doesn't" have proper error reporting? Where do > > you see this lack of reporting (if that's what you meant)? > > By "proper" I mean that if the parse fails it should provide an > end-user-friendly diagnostic, and optionally quote the line and put a > caret under the position that the parse failed (a la the Clang > diagnostics). Your non-end-user error reporting is fine. Oh... hmm.. well, each rule does leave the iterator out-param at the spot where an error occurred. But I'm not quite sure how to emit the diagnostic - surely you aren't suggesting we write to std::cout or std::cerr? Are we using curses or something? Or maybe just plain ASCII art? How exactly would this look? Thanks to Peter's work on boost::system::error_code, when a parse error occurs the error is augmented with file name, line, and function name information. And I have worked on adding visualizers so these are nicely presented in the debugger. So at least we have that. Thanks

1 0

boost.url review
by Richard Hodges 21 Aug '22

21 Aug '22

My review of Boost.URL Before answering the questions presented by the review manager, I thought I’d share a few words about what Boost represents to me and why I take an interest in it. Many moons ago, Boost was pretty much the only way to use the C++ programming language in any kind of useful, cross-platform way. Today, the C++ standard has evolved and grown considerably. Many of the original drivers to use Boost are seemingly “solved” in the language itself. Notwithstanding that some of the design decisions taken by the Committee over the years have resulted in an anaemic half-copy of Boost features being standardised. Because of this (in my view) failure of the committee decision process in the face of reality, I still use many of the fundamental Boost utilities in place of their standard counterparts. Therefore, to my mind, Boost is still a necessary component of any non-trivial program I write in C++. Boost also has a collection of utility libraries, some of which have waned in relevance over time. For example, Boost.Format is now pretty much obsolete in the face of fmtlib (not std::format, which is another example of a half-hearted attempt to standardise a solution). When I program, I do so almost exclusively in C++. I don’t really want the bother of having to reach for other languages or to stay current in their dependency management solutions of the day. What I really want is one massive library that I can pull in which does everything I need, or at least the common things. Common things for me are: - Communications, including HTTP, WebSockets, databases and messages buses. - Console IO - Logging - Graphics & Human Interfaces - Sound For me, anything that takes us a step closer to having one library that shrink-wraps all that into one `find_package` call in CMake is a win. Now I will admit that once upon a time, I believed that all this stuff should be in the standard library. However since then, having worked as a library maintainer myself, I have come to realise that this is an error. Giving compiler maintainers the burden of maintaining humongous libraries will simply discourage innovation in compiler technology and result in incomplete implementations and porting to new computing platforms. Boost’s cross-platform compatibility is unparalleled. Therefore I am strongly of the view that the place for C++ library innovation is Boost, and the place for standardising the language changes that are demonstrated as necessary by Boost innovation lies with the Committee and compiler developers. For me, any new library that increases Boost’s real-world utility, is fully cross platform and reduces my workload when writing application programs is very welcome. For for the review: > Does this library bring real benefit to C++ developers for real world use-case? Yes. URL parsing can be fluked with regex, split and a few hand-rolled utilities. But why give developers the burden of hand-rolling this boilerplate nonsense? Make simple things simple! > Do you have an application for this library? Absolutely. I am happy to retrofit Boost.URL into existing code and would certainly use it in all future code. > Does the API match with current best practices? I have no idea what “best practices” are in API design. The API allows me to interrogate, build and update internet URLs. I may have made different naming choices and if I had written it I would probably have made more use of free functions, but I’m not going to argue about style. Utility is all that matters to me. > Is the documentation helpful and clear? Much like the Boost.ASIO documentation, it is 100% correct and tersely describes the exact functionality of the API. Use of the documentation requires at least a cursory understanding of the terms in RFC3986. Twenty years ago, it would probably have been reasonable to require users to read this RFC prior to using the library. Today, attention spans are shorter and the internet is a commodity. I would prefer to see a little more narrative and descriptions of real-world uses of some of the methods, along with code examples. > Did you try to use it? Yes, I retrofitted Boost.URL into a program that I used as the basis of a blog post - a program to demonstrate following redirect responses when connecting a WebSocket. I have also integrated Boost.URL into a new program which connects to an exchange in the past week. Doing so saved probably 200 lines of hand-rolled code. > What problems or surprises did you encounter? The library lacked a specific facility to extract just the HTTP TARGET field from a URL. After reporting this to the author with a supporting use case and hand-rolled work-arwound, this has now been added. > What is your evaluation of the implementation? It is clear that a great deal of care has gone into making the implementation robust and efficient. > How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I retrofitted Boost.URL into an existing program and wrote a new program that uses the library. I spent some time discussing what I felt were missing features with the author, who has been responsive. My required features subsequently appeared. > Are you knowledgeable about the problem domain? I’ve been writing internet-facing servers and clients in C++ for a decade. In all that time I have never read RFC3986. I suspect like 99% of developers, I have simply inferred the rules of URL encoding through use. So perhaps a time-served journeyman rather than domain expert. Recommendation: ACCEPT The day I can write a CML that has only one dependency, Boost, will be a happy day. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

3 2

Boost.URL review
by Zach Laine 21 Aug '22

21 Aug '22

- Does this library bring real benefit to C++ developers for real world use-case? Yes. - Do you have an application for this library? No. - Does the API match with current best practices? See details below. - Is the documentation helpful and clear? In Overview: - s/will be easy to read/is easy to read (no need to use the future tense) - s/1.78/1.81 (can't use 1.78 if URL depends on TOT, which is stated elsewhere in the review announcement) - The bullet list is an awkward mix of verb- and noun-phrases (e.g. "Require only C++11" and "Fast compilation, few templates"). I suggest all noun-phrases, like "Requires only C++11"). - s/1.80/1.81 -- same reason as above. - I think that flow in tutorial-style documentation is important. In the Overview, you introduce what the library is at a high level, then jump into some build- and compiler-support weeds, only to then follow with the Quick Look section. IMO, you should move the bits after the bullet list to a section called Compilers and Build Considerations or similar, and place it much later in the docs. In Containers: The text at the top makes the relative/absolute distinction sound important, but the rest of the page does not define or show examples of the difference between the two. I suggest single quotes around "/" in "The reason is any decoded character / could make", since on first reading I thought it was part of the documentation text, not in code-font. Maybe do it like you do for the mention of double slashes a few lines later? When the ambiguity of the string "https://www.example.com/get-customer.php?name=John%26Doe" is mentioned, the docs say that if the query string represents parameters, the quoted "&" is not ambiguous. Then it shows an example of getting all the parameters from the string "https://www.example.com/get-customer.php?id=409&name=Joe&individual". This begs the question, what happens when there's a "%26" in the string, and you use .params()? Please show this as part of the example, or at least explain it. Examples: It would be nice if these were on separate pages. Even nicer would be some context, besides just a bunch of code. I had to Google "magnet link", and I still don't really know what it is. A brief explanation would be helpful. API Docs: The utl::params API docs refer to it as a random-access view, but it is neither random-access, nor a view. - Did you try to use it? What problems or surprises did you encounter? I did not. - What is your evaluation of the implementation? I did not look too much at the implementation. but what I did look at was somewhat hard to follow. I went looking for where a result<> is filled in during the parse, and it took me a solid 45 minutes to find it. Design: Overall, I quite like the design. Lazy, lazy, lazy. The fact that only views into the underlying sequence are returned from the parse is great, and the fact that there are both owning (url::url and fixed-capcity owning url::static_url) and view versions of the URL representations is exactly what I would need as a user. When I call parse_uri(), I get a result of type result<url_view>? Why isn't the function spelled parse_url(), or the return type spelled result<uri_view>? This seems needlessly inconsistent, given the assertion in the docs that URL is fine to use in place of URI. The assign_to() API seems awkward to me (as in std::string str; u.query().assign_to(str);). This would be better as a conversion function with a name, like string(), to_string(), or as_string(). Assignment is fundamental, and the compiler knows what to do with it (how to optimize it away, for one thing). Assignment through an out-param is awkward at best, and is actually more likely to lead to more pessimal code, since RVO cannot be applied. The append_to() API is really very nice. Most libraries don't support appending as a first-order operation, but it is essential for string perf in many use patterns. However, the containers are frankly a mess. What are they, exactly, at a high level of abstraction? For instance, is params a singly-linked list? It has a forward iterator, so it kinda looks like one. Is it an associative container? It has key/value pairs (where, sure the value might be empty, but it still exists at the level of the container), so it looks like one of those too. Is it array-like? The keys are not sorted, and the order corresponds to the order in a URL, so it sure looks array-like. This is important to users who want to use your containers in generic code, including the standard algorithms (which often have a pessimal implementation specific to forward_iteretors). Also, your containers have about 10% of the API of any of the kinds of containers listed above, meaning generic code will have to special-case use of your container if it's missing some of the SequenceContainer or AssociativeContainer API that the generic code relies on. I recommend that you implement params in terms of a std::vector<std::pair<string_type, string_type>> (using whatever string_type you're already using), and then add the API that is URL-specific on top of it, and have two members: std::vector<std::pair<string_type, string_type>> & storage(); std::vector<std::pair<string_type, string_type>> const & storage() const; Then make the iterators RA. Similar changes seem appropriate for the other containers as well. "Remove" is not really a term used in the STL. I think your "remove*()" container members should be "erase*()" instead. The replace() API is nice. I only wish it were range-capable (that is, I wish there were an overload taking two iterators to erase, and two iterators to insert). Why did you define your own find_if() and find_if_not(), instead of using the std ones? And why the hell does CharSet have these two as members? That's crazy! I find the creation of a mini-parsing library and the expectation that users will use this mini-library to write their own parsers to be somewhat problematic. No one is going to use your mini-library for serious parsing work (except perhaps for modifications to the way URL does its parsing). It does have proper error reporting, for one thing. There are other equally obvious limitations. If the parsing were an implementation detail, or if you were only providing the parsing lib for extensions to your existing parsing, what you've provided would be enough. However, you also claim that you can use URL as part of a larger parser, and I don't think that's realistic. I vote to ACCEPT. I think I've raised legitimate concerns about the design here, but none of them rises to the level of a blocker. Zach

2 1

Boost.url review reminder
by Klemens Morgenstern 21 Aug '22

21 Aug '22

Hi all, this is a reminder that the boost.url review is currently going on, and we only got one review so far. Any review is appreciated. The Review ends on the 22nd, i.e. next Monday. Thanks in advance, Klemens

3 2

Is there an analogue of `boost::reference_wrapper`, that can also work with rvalue references?
by Денис Михайлов 20 Aug '22

20 Aug '22

I find an wrapper like `boost::reference_wrapper`, but with more abilities, like this: ``` int value = 0; auto rref = boost::ref(std::move(value)); auto prref = boost::ref(int{}); f(rref); // pass rref into f by ordinary reference f(std::move(rref)); // move rref into f f(std::move(prref)); // move prref into f ``` Is there such functionality on the Boost side? If not, is it allowed to discuss the possibility of contributing it?

1 0

Is there a way to get ctype facet with boost::locale::generator?
by Gleb Igumnov 19 Aug '22

19 Aug '22

Hello. I have a problem. I need to get std::locale object for my locale (ru_RU.UTF-8) on a systems that don't have one. I decided to take a look at boost::locale::generator, as it has ICU as a backend and there must be full locale info in it. But to my surprise generator do not generate only locale facet that I really need - ctype one. For example following code will go "oops" branch: #include <iostream> #include <boost/locale.hpp> #include <boost/locale/localization_backend.hpp> using namespace std; int main() { using namespace boost::locale; localization_backend_manager my = localization_backend_manager::global(); my.select("icu"); // I know that icu is default backend but I select it explicitly to be sure. boost::locale::generator gen(my); std::locale loc( gen.generate( "ru_RU.UTF-8" ) ); if( !std::isalpha( L'ф', loc ) ) std::cout<< "Oops\n"; else std::cout << "Ok\n"; return 0; } But if I will do std::locale loc( gen.generate( std::locale( "en_US.UTF-8" ), "ru_RU.UTF-8" ) ); It will work, as it will have ctype facet from en_US.UTF-8 locale, but that is not a solution as I may not have en_US.UTF-8 locale either, and on some systems, (for example iOS) I won't have any UTF-8 locale available for std::locale constructor. So I'm curious, why generator won't generate ctype locale? Is there any reason behind it? Or maybe I'm doing something wrong? Is there any way to generate locale with proper ctype facet from ICU using boost::locale::generator? -- -- With respect Gleb "Crazy Sage" Igumnov mailto:crazy.sage@gmail.com

1 0