July 2017 - Boost - lists.preview.boost.org

My Beast Review
by Phil Endecott 12 Jul '17

12 Jul '17

Dear All, This review is not very thorough - I've only looked at the docs and some small parts of the code, and read the discussion on the list. I've not tried to use the library. But I think I've seen enough to express an opinion. I've never used ASIO. I've studied it in the past, and concluded that it offers portability to platforms that I don't use but doesn't work well with other platforms that I do use. It also seems more complicated than I would have liked, perhaps in order to achieve some optimised level of performance that I don't need (or perhaps for no good reason). I have however quite often used HTTP (hasn't everyone?) and I think it's an embarrassment that std::c++ can't download something from the 'net with one or two lines of code in the way that almost every other language can. So, my first reaction is that it's a disappointment to see that a proposed HTTP library is tied to ASIO. On further investigation, however, I think this is no great loss to me as the functionality that this library provides is so limited in scope. Parsing HTTP headers, for example, is not much work. Even my own hacked-together-in-an-afternoon HTTP classes do more than this - to mention just one thing, parsing and formatting of dates in the correct format. Everyone who uses this library will have to implement this sort of thing themselves on top of it. (And many of them will implement it wrongly.) Vinnie says that he hopes that more functionality will be layered on top of this work. The danger is that the person who comes along wanting to implement the other 75% will not like Vinnie's starting point, and re-implement it all in their own style. (I know I would.) Unless, of course, Vinnie is going to offer this "other 75%" himself eventually. *That would be great*. If that is the plan, my review is "this is a fine start but please remove the ASIO dependency and come back when it's finished". But I think Vinnie has said that the library is to be reviewed "as is". In that case, my review is "just add this to ASIO and call it boost.asio.http and boost.asio.websocket, no need for a review" (assuming that the ASIO developer is happy with that). Regarding what I have seen of the implementation: - Security must be one of the most important properties of an HTTP library, and I'd really hope to find that such a library had been implemented by someone with significant experience in this area. I never cease to be amazed by the ingenuity of attackers. - I fear that there may be much premature optimisation in the parser code that I've looked at. For example: static bool is_digit(char c) { return static_cast<unsigned char>(c-'0') < 10; } Is that right? Say c='!' (ASCII 33). '0' is ASCII 48. So c-'0' is -15. But you cast that to an unsigned char, so -15 = 241, and 241 > 10, so the function returns false. So it is correct - but (1) it needs a comment to explain it, (2) it's the sort of thing that makes me worry from a security viewpoint, and (3) is it actually an improvement on the more obvious c >= '0' && c <= '9'? Well I've done a quick test and I get the same assembler for both - so why bother? (Except on close inspection actually I don't get exactly the same assembler. To get the same assembler I have to replace the constant 10 in your code with 10U, which changes the final comparison from signed to unsigned. I don't think that matters, but it again makes me wonder whether this security-critical code is ideal.) But hey, why even do c >= '0' && c <= '9' when we have std::isdigit()? But it would be much slower, right? Wrong, it's actually one instruction shorter; somehow it manages to avoid one of the zero-extension instructions. So this is not a premature optimisation, it's a premature pesimisation. (My test code is at the end of this message.) Now, is_digit is just about the simplest thing in the parser; Vinnie has also implemented his own code for parsing numbers, and then we come to parse_field. Please have a look at this (around line 666 of basic_parser.hpp). Note that it calls find_fast() near the top of that file, which uses some Intel SIMD instruction to, I think, skip forward over the header field name, and then over the field value to the CRLF. Not that there are any comments in find_fast to explain what it does, for those of us who don't know what _mm_loadu_si128 and _mm_cmpestri do. Has this code actually been benchmarked against a much simpler and more obviously-correct version, and found to have a worthwhile performance benefit? (I believe that some of the discussion has used this "optimised parser" as a justification for not making the code more generic, i.e. to work with any iterator pairs. I believe that to be flawed, since (a) I don't think this optimisation is useful, and (b) even if it were, the code could use it for any contiguous iterators, and fall back to "unoptimised" code in other cases. I think that having the "unoptimised" code present would be useful for documenting what the parser is trying to do, in any case.) Could it be that the reason why the scope of the library seems so limited is that Vinnie has spent his time fiddling with this optimisation, rather than adding useful features? My opinion: having looked at the implementation, reject. Test for is_digit: #include <cctype> bool is_digit_1(char c) { return static_cast<unsigned char>(c-'0') < 10U; } bool is_digit_2(char c) { return c >= '0' && c <= '9'; } bool is_digit_3(char c) { return std::isdigit(c); } Aarch64 assembler output from g++ 6.3.0 -O3, with the uninteresting bits removed: _Z10is_digit_1c: uxtb w0, w0 sub w0, w0, #48 uxtb w0, w0 cmp w0, 9 cset w0, ls ret _Z10is_digit_2c: uxtb w0, w0 sub w0, w0, #48 uxtb w0, w0 cmp w0, 9 cset w0, ls ret _Z10is_digit_3c: uxtb w0, w0 sub w0, w0, #48 cmp w0, 9 cset w0, ls ret Regards, Phil.

6 7

Outcome v2 is feature complete
by Niall Douglas 11 Jul '17

11 Jul '17

I am pleased to announce that Outcome v2, rewritten to fulfil the Boost peer review feedback from two months ago, has reached feature completeness. Outcome provides a STL-quality `result<T, EC>` and an `outcome<T, EC, E>` implementation of success|failure value transports, and is expected to pass a second Boost peer review and be submitted for C++ standardisation as the lowest level of an overall framework as proposed by: * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0650r0.pdf C++ Monadic interface * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0323r2.pdf A proposal to add a utility class to represent expected object (Revision 4) The idea is that result/outcome are the extremely lightweight struct-like success|failure value transports suitable for usage in public APIs, whereas expected is the much richer variant-like monadic success|failure value transports, and everything participates together under P0650R0. A detailed changelog over v1 can be found at https://github.com/ned14/outcome/blob/master/Readme.md. The main changes are the removal of lots of stuff which was in v1, and we now replicate `std::variant<...>` constructor API. v2 is an “absolute bare minimum” implementation with no frills. Outcome v2 is passing the modified v1 test suite handily, and v1 code should port to v2 easily enough. But as an all-new codebase it lacks: * Maturity * Polish * Use experience * Documentation (a class synopsis can be found at https://ned14.github.io/outcome/synopsis/) My hope is to return Outcome v2 for Boost peer review by October. If you are using an internal Expected-like object, or Outcome v1, I’d hugely appreciate it if you tried porting your code to using v2 and giving me feedback on how it went for you. Minimum compilers needed are GCC 6 or clang 4. Help with documentation and testing is also hugely appreciated. My thanks in advance! Github: https://github.com/ned14/outcome Docs (highly incomplete): https://ned14.github.io/outcome/ Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

5 19

icl and C++17
by Duarte Nunes 11 Jul '17

11 Jul '17

Hi, Boost icl doesn't compile using GCC 7.1 with -std=gnu++17. There's a pull request solving the issue[1], but icl seems non-maintained. It would be great if someone could upstream that change so that projects depending on icl can get upgraded to C++17. Thanks, Duarte [1] https://github.com/boostorg/icl/pull/9

1 0

Re: [boost] [Boost-users] Boost 1.65.0 Beta 1 Release Candidate 2
by Marshall Clow 11 Jul '17

11 Jul '17

On Tue, Jul 11, 2017 at 4:06 AM, Raffi Enficiaud via Boost-users < boost-users(a)lists.boost.org> wrote: > Le 11.07.17 à 02:14, Marshall Clow via Boost-users a écrit : > > The second release candidates for the 1.65.0 beta release are now >> available at: >> >> >> https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/ >> boost_1_65_0_beta1_rc2.7z >> >> https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/ >> boost_1_65_0_beta1_rc2.tar.bz2 >> >> https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/ >> boost_1_65_0_beta1_rc2.tar.gz >> >> https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/ >> boost_1_65_0_beta1_rc2.zip >> >> The SHA256 checksums are as follows: >> >> d54d29568c4e4933cbfe40a292b6cb84a7f68121e34f6a83f7d0f7c652b6b2d8 >> boost_1_65_0_beta1_rc2.7z >> 092676557513e992347d30dd0cd058fb8af2a401e5da4f4413f5b577852d726b >> boost_1_65_0_beta1_rc2.tar.bz2 >> 31fa3c807e2205e18f7d5c74f77f540677f7b0937af1e37f595c137e1e87bf84 >> boost_1_65_0_beta1_rc2.tar.gz >> 83c8d8b76fa9591572226a1a49fd191d76db3011b52b907cbbc8d37d46213bd2 >> boost_1_65_0_beta1_rc2.zip >> >> >> This RC contains a single fix to the stacktrace library. >> >> As always, the release managers would appreciate it if you download >> the candidate of your choice and give building it a try. Please report >> both success and failure, and anything else that is noteworthy. >> >> Thanks! >> >> -- The release managers >> > > Hi, > > I have this error during the build on OSX. Built with: > ./b2 cxxflags="-std=c++11 -stdlib=libc++" linkflags="-stdlib=libc++" > --without-log --layout=versioned install > > > ...on 14200th target... > common.mkdir bin.v2/libs/fiber/build/darwin-4.2.1/release/threading-multi > common.mkdir bin.v2/libs/fiber/build/darwin-4.2.1/release/threading- > multi/numa > darwin.compile.c++ bin.v2/libs/fiber/build/darwin-4.2.1/release/threading- > multi/numa/pin_thread.o > libs/fiber/src/numa/pin_thread.cpp:23:10: warning: pin_thread() not > supported [-W#pragma-messages] > # pragma message "pin_thread() not supported" > ^ > 1 warning generated. > darwin.compile.c++ bin.v2/libs/fiber/build/darwin-4.2.1/release/threading- > multi/numa/topology.o > libs/fiber/src/numa/topology.cpp:23:10: warning: topology() not supported > [-W#pragma-messages] > # pragma message "topology() not supported" > Given these two messages, I suspect that is deliberate. I see that too in my tests. -- Marshall

1 0

Fwd: [beast] (In)formal review
by Michael Caisse 11 Jul '17

11 Jul '17

Forwarding to the list: Received 1:19 AM my time... still July 10th in Hawaii ... and I'm still up reading Beast reviews so it seems good enough for me. Thanks for the review! -------- Forwarded Message -------- Subject: [boost][beast] (In)formal review From: Rüdiger Berlich ------------------------------------------- Dear Michael, dear Vinnie, having read the very lively discussion during the review, I am not sure I can add much to the technical side. Please feel free to either count this voice or leave it out, as you see fit, particularly as this mail didn’t come in time for the formal review. Hence I am sending this as PM. In my own applications, I limit myself to Boost as an external dependency. Having Websocket-functionality as part of Boost (the „pure“ HTML-side is of less significance to me) would be a god-send and would save me countless hours, allowing me to focus more on what I’m good at. Currently I just build directly on ASIO. I feel that Beast fills a very big niche and, having followed the development of the library for a while now, I am also confident that it is sufficiently advanced to be included into Boost. Thus, from a mere user’s perspective, I’d like to strongly vote in favor of ACCEPT. ============== - What is your evaluation of the design? I have worked with ASIO for a while and appreciate that Beast follows the same principles. - What is your evaluation of the implementation? I have browsed though some of the code and feel that it is well structured and certainly easier to read than some of the other Boost-code I have come across - What is your evaluation of the documentation? Beast does not intend to be a high-level library, and for the intended audience, the documentation is well-designed and complete. I would hope that, if and when Beast becomes part of Boost, there’ll be more tutorial-like information similar to what Boris Schaeling does in his books (e.g. here: https://theboostcpplibraries.com — could well become part of that). - What is your evaluation of the potential usefulness of the library? Immense. C++ is lacking a LOT of functionality on the library side, and this is one of the essential building blocks needed by people intending to create the higher-level libraries. I believe that we will see many new library proposals on top of Beast. - Did you try to use the library? With which compiler(s)? Did you have any problems? I have followed the development of Beast for a number of months and have „played“ with it in different contexts. I am in the process of integrating it into my own (client-server type application). In comparison to other libraries, it seems to be much more vivid and fast-paced. I have used it on MacOS and Linux (—> clang and g++). - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? No in-depth study, but in summary there is certainly more than a day of involvement. - Are you knowledgeable about the problem domain? Having done quite a bit of „do it yourself“ on top of ASIO for my own application, I appreciate what Beast can do for me (on the Websocket side) and am in dire need of this functionality. As a library-USER, I actually do not intend to be an expert for the implementation side and would be very thankful if it became available as part of Boost. I’d like to thank Vinnie for the huge amount of work that went into Beast and the Boost-community for the superb collection of libraries that have kept C++ alive. Kind Regards, Beet P.S.: I do not like the name „Beast“, but do not think it is important. Certainly this is up to the author to decide :-)

1 0

Encoding address-model in library names
by Peter Dimov 11 Jul '17

11 Jul '17

I have prepared the pull requests necessary to encode the address-model (32 or 64) in the library names, which allows placing the 32/64 libraries into the same stage/install directory, and building with address-model=32,64 in one go. These are https://github.com/boostorg/boost/pull/147 and https://github.com/boostorg/config/pull/159 This being a Serious Change, the prudent thing to do is to wait out 1.65, then proceed. On the other hand, experience shows that this kind of change is only tested when a release goes out anyway; delaying it for four more months would not help much. Therefore, I'm calling for opinions; would people want this to go into 1.65? Into 1.66? Not at all?

16 50

Boost 1.65.0 Beta 1 Release Candidate 2
by Marshall Clow 11 Jul '17

11 Jul '17

The second release candidates for the 1.65.0 beta release are now available at: https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/boost_1_65_0_beta… https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/boost_1_65_0_beta… https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/boost_1_65_0_beta… https://dl.bintray.com/boostorg/beta/1.65.0.beta.1/source/boost_1_65_0_beta… The SHA256 checksums are as follows: d54d29568c4e4933cbfe40a292b6cb84a7f68121e34f6a83f7d0f7c652b6b2d8 boost_1_65_0_beta1_rc2.7z 092676557513e992347d30dd0cd058fb8af2a401e5da4f4413f5b577852d726b boost_1_65_0_beta1_rc2.tar.bz2 31fa3c807e2205e18f7d5c74f77f540677f7b0937af1e37f595c137e1e87bf84 boost_1_65_0_beta1_rc2.tar.gz 83c8d8b76fa9591572226a1a49fd191d76db3011b52b907cbbc8d37d46213bd2 boost_1_65_0_beta1_rc2.zip This RC contains a single fix to the stacktrace library. As always, the release managers would appreciate it if you download the candidate of your choice and give building it a try. Please report both success and failure, and anything else that is noteworthy. Thanks! -- The release managers

1 0

[mpl] [bug] Bug on python script
by Alessandro Marzialetti 10 Jul '17

10 Jul '17

Hello, I cannot open a ticket on the website due to an error OperationalError: could not extend file "base/19263/19289": No space left on device HINT: Check free disk space. Anyway, I found a bug in the library. The file is mpl/preprocessed/boost_mpl_preprocess.py a script that creates the headers required to have more than 50 types on a template. Basically the problem is on the string substitutions, with some values the output is wrong. I made a pull request that fixes the problem making a temporary substitution of the values https://github.com/boostorg/mpl/pull/29

2 1

[review][beast] Five-days Remaining
by Michael Caisse 10 Jul '17

10 Jul '17

Hi all - We are half-way through the review and have some great involvement and exchanges and a few reviews already. I hope if you haven't written a review yet that you still will. Please provide whatever effort you can afford toward the review process. Boost depends on community involvement. You might have noticed that I have solicited input from you regardless of your experience with the domain. I personally think it is a mistake to have input only from domain experts. True innovation often comes from "outsiders". If you have no experience with Asio or with HTTP, how approachable is this library? Were you able to understand the documents? Could you use the examples? Were you able to build something of your own? Domain experts -- you know the dark corners of the domain. The hard parts that have caused you hours of lost sleep and the need to hack around an existing library or to write your own. How does Beast address your concerns and the problems you have experienced in the past? Is it flexible enough to be put into your tool chest? So far we have reviews from: * Marcel Ebmer * Peter Dimov * Artyom Beilis * Zach Lain * Nik Bougalis * Klemens Morgenstern * Glen Fernandes If I missed a review from you, please let me know! Happy Coding! michael -- Michael Caisse Ciere Consulting ciere.com

6 13

[beast] Review
by Jens Weller 09 Jul '17

09 Jul '17

This is my review of the beast libary. First, I find the name quite fitting, but also we clearly need a "cage" library to make the internals of beast more useful for the endusers. I spend a few days on and off reviewing examples, and documentation. I used the library as an example for a fuzzing experiment on the weekend. What is your evaluation of the design? The design is very boost like, the library fits well into boost. One thing the library shares with asio is the buffer concept, which I never really liked. Buffer feels to me to abstract, to far away from the usual type vocabulary one is used to from standard C++. But once you're used to the concept, its ok. So no big deal. The dependence on Asio was already mentioned during the review and I am not sure about this. Surely a nice goal to have, but also not an easy one. But replacing asio with an abstraction that allows for pluggin in other socket implementations would be very powerful. Its nice that the library is header only, but over asio you get the dependency to boost::system for error_code. I'd love to be able to exchange boost::error_code dependency to the std::error_code one. As the beauty of header only really shines when you don't need to link to other non header only libraries... What is your evaluation of the implementation? Fuzzing. I spend this weekend some time to fuzz beast with libFuzzer. The basic_parser and the websocket::stream were fuzzed. A bug (buffer overflow) in basic_parser was found, and is already fixed. What is your evaluation of the documentation? Boost typical. Its good, helps with getting the library to know. What is your evaluation of the potential usefulness of the library? This library could be the foundation of many things to come. The internet is the basic building block for so much today, that beast is a nice improvement upon asio in providing the vocabulary to use boost in the internet age. As I already mentioned, I think that other libraries are needed in order to make this library really useful for most end users. But the scope the author aims at, beast fits very well. Did you try to use the library? With what compiler? Did you have any problems? Fuzzing was based on clang 5.0, but I also used the library with GCC 5.4 to generate example data for the fuzzing. I plan to use the library to write a small http server and clients for my own usage. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? My time for this review is limited, but as the author mentioned on this list, that the library hasn't been fuzzed, I aimed for doing this instead of implementing an http client/server. I still had to write code in order to reach the parts in beast to be fuzzed, this gave a more detailed tour of beast, then one would need for simple example exploring. Are you knowledgeable about the problem domain? A little, I am mostly a high level user of various HTTP libraries, but not at the low level beast covers. I found it interesting to get a glance at the details. And finally, every review should answer this question: Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. I think that this library should be ACCEPTED into boost. It is an important improvement on asio, in order to offer a more detailed networking stack in boost. The library has the exact right scope to do this, widening the scope into providing client/server implementations would not be good. I see this in another library. thanks, Jens Weller

3 4