- Boost - lists.preview.boost.org

Another Boost.Decimal Review
by Jeff Garland 23 Jan '25

23 Jan '25

Thanks to Matt Borland and Chris Kormanyos for this good work. I vote to ACCEPT the library. I trust the authors to improve documentation, api, and performance as a matter of continued development. Let me say a few things about some discussions in the review threads up-front. *Standardization:* I'm an advocate for it and would be willing to help the authors with the process. Rationale: 1) this standardizes existing practice (aka IEEE 754), something like it is supported in many languages (specifically C23), and there's significant use cases for this in many domains even if it's slightly slower. 2) While a standalone library can go far, often a compiler/library can more easily optimize fundamental things like numeric types. 3) I spoke with Dietmar about the review and standardization -- he's still an advocate for having this capability, but has higher priorities at the moment. *N2849*: It's a decent design for 2009 -- less so in 2025. Specifically, w.r.t. conversion we should only perform automatic conversions from types (see below). The library already takes care of from_chars, format and some other more modern things N2849 wouldn't address. *C23 and _Decimal** Should this library consider being a layer over C23 types where that's possible? GCC supports them at least in a limited fashion. https://gcc.gnu.org/onlinedocs/gcc/Decimal-Float.html > Does this library bring real benefit to C++ developers for real world use-case? > Do you have an application for this library? Yes and yes. > Does the API match with current best practices? I have some quibbles here. *Construction*: template <typename T, typename T2>constexpr decimal32(T coeff, T2 exp, bool sign = false) noexcept; Is the sign flag helpful here? It seems simpler to just handle a negative coefficient -- like it already does. When I look at it there's already cognitive dissidence in my brain about if the sign flag *overrides* or *inverts* the coefficient sign. The N2849 api doesn't have this flag. *Fast Types:* I read through the list conversation about the _fast types. My suggestion is to drop them. Simplifies everything for users and implementers. The movement in c++ over the last decade(s) has been to precisely say the size of the type to ensure correctness and not 'let the platform decide' (aka uint32_t, int64_t, etc). In my comment on documentation you'll notice the c++20 extended floating point types *only provide* fixed sizes. Also N2849 and C23 don't support _fast types for decimal. *Conversions:* I'd like to see the library *pull back* on convertibility. long double ld{1.23456789234567}; decimal32 d(ld); // what happens -- round, truncate, doesn't compile? The best answer is doesn't compile. There's been significant work in the standard now to define floating point rank and limit conversions with the extended floating point types. Specifically: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1467r9.html#rank There's also other work on type traits and value preserving conversions in the SIMD capabilities. Also, std::chrono basically already does this -- any value destroying conversion has to be explicit. At a minimum the answer needs to be better documented - aka does what 754-2008 says? For other examples of things I'd remove and make a free function cast/round -- aka explicit user decision: class decimal64 { ... explicit constexpr operator std::int8_t() const noexcept; This is highly likely to not be value preserving so shouldn't be possible without a decision on how to handle overflow. class decimal64 { ... explicit constexpr operator decimal32() const noexcept; *Literals* constexpr auto operator "" _DF(const char* str) -> decimal32constexpr auto operator "" _df(const char* str) -> decimal32 Why not _d32, _d64, _d128? _df makes me think fast, not 32. > Is the documentation helpful and clear? Not bad, but as usual improvements can be made. As a user I'm going to need to select one of these types -- I'd love to see a table like this one at cppreference for named floating point types: *Type name* Defined in header <stdfloat> <https://en.cppreference.com/w/cpp/header/stdfloat> Literal suffix Predefined macro C language type Type properties bits of storage bits of precision bits of exponent max exponent std::float16_t f16 or F16 __STDCPP_FLOAT16_T__ _Float16 16 11 5 15 std::float32_t f32 or F32 __STDCPP_FLOAT32_T__ _Float32 32 24 8 127 std::float64_t f64 or F64 __STDCPP_FLOAT64_T__ _Float64 64 53 11 1023 std::float128_t f128 or F128 __STDCPP_FLOAT128_T__ _Float128 128 113 15 16383 std::bfloat16_t bf16 or BF16 __STDCPP_BFLOAT16_T__ (N/A) 16 8 8 127 Not sure how it renders in email so source: https://en.cppreference.com/w/cpp/types/floating-point Not looking for exact replication, but something like this. In the *using the library* section I'd like to see some: - examples of using it it math - examples of the behavior of NAN/inf values - conversions from other types like float expanded Where is the design decisions section? Some of the things I'm asking about should be covered in the docs. > Did you try to use it? What problems or surprises did you encounter? I didn't have time to play with it unfortunately -- godbolt would be helpful. That's all for now. Jeff

4 5

Re: [boost] Decimal: Formal review begins
by Glen Fernandes 23 Jan '25

23 Jan '25

I mentioned this to Matt, but I would like to see benchmarks comparing boost::decimal64_fast to a Decimal64 backed by Intel's DFP library, which is what I know to be in actual use today: https://www.intel.com/content/www/us/en/developer/articles/tool/intel-decim… Not that the benchmarks comparing to GCC's Decimal64 (which is based on the libbid which ships with libgcc) aren't useful, I personally don't know anyone using that today. Benchmarks should ideally also include Intel's compiler, because at least one of the relevant parties who motivated me to suggest the Decimal64 library (to Vinnie as a potential project) do use the Intel C++ compiler (and their Fortran compiler) for areas where they perform better. (The non-fast versions don't matter to me. I don't know anyone who would want to use them). Glen

12 36

Decimal Library: Last call for reviews?
by John Maddock 22 Jan '25

22 Jan '25

Formally, today is the last day of the Decimal Library review, so far there has been a lot of discussion, but only three full reviews with outcomes. Are there any more reviews incoming? Thanks, John.

1 0

My review of Decimal
by Ivan Matek 21 Jan '25

21 Jan '25

Hi everybody, PDF size exceeds ML limit so here is google drive link https://drive.google.com/file/d/1kH9WkeTDxN7FFqM_g7hV_-e3kJPlTGBr/view

6 16

Reminder to submit Boost.Decimal reviews
by John Maddock 20 Jan '25

20 Jan '25

Just a gentle reminder that the Decimal library review is currently ongoing, there's been plenty of discussion, but only one full review so far.... so if you're thinking of submitting a review, now would be the time ;) Thanks all, John.

1 0

Fw: Boost accepted in Google Summer of Code 2021
by Christopher Kormanyos 17 Jan '25

17 Jan '25

----- Forwarded Message ----- From: "boost-users-owner(a)lists.boost.org" <boost-users-owner(a)lists.boost.org>To: "e_float(a)yahoo.com" <e_float(a)yahoo.com>Sent: Wednesday, March 10, 2021 at 07:51:24 PM GMT+1Subject: Re: [boost] Boost accepted in Google Summer of Code 2021 To post messages to the Boost Users mailing list, you will need to subscribe this e-mail address via the Boost mailing list web page at: http://lists.boost.org/mailman/listinfo.cgi/boost-users You will need to take this step even if you intend to post only through GMane or some other news gateway. In this case, you may wish to edit your subscription options to turn off mail delivery. >> We're in again! > Thank you, David! Hi Matt (@mborland) This was, as far as i recall, our last successful timegetting into and finishing some GSoC 2021 projects. We had that whole Corona thing going on and it wasa tough time for everyone. As far as applying as an org, maybe David hassome old material. - Chris_________________________________________Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

2 2

Adding logical implication to boost
by Ivan Matek 09 Jan '25

09 Jan '25

Hi everyone, there has been a recent proposal to add logical implication to C++ as an operator. github tracking issue <https://github.com/cplusplus/papers/issues/1641> P2971R2 <https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2971r2.pdf> For a video description you can see this talk <https://youtu.be/bgyY3x8y4PE?feature=shared&t=1470> by Walter(it is only few minutes, not entire talk). While I agree with concerns mentioned on the issue, i.e. using a *precious *token => on rarely used functionality I do think that implementing a simple function helper is reasonable. Poll in issue shows that this is also likely standardization future. POLL: We agree with adding a function object logical_implication (tentative name, akin logical_and, `logical_or’). SF F N A SA 9 12 3 0 0 As for if it is worthwhile to add this simple logic to boost instead of people just writing their own I will just say I would prefer to have something named in boost instead of people reinventing same thing many times in their projects. I have used it few times in my career, so quite rarely, but again that means I have used it much more often than 80+% of algorithms in <algorithm> header. As for location I presume best place would be Boost.Algorithm although you could say that Boost.Core can be a candidate because it is a basic operation and would benefit from shorter namespace. I would be still inclined against this since I have seen core/util/common become huge in many codebases I have worked in. I have tried to check if this already exists in Boost; all I found was an example in Boost.Hana.

6 7

Forthcoming Decimal Library review (15th Jan)
by John Maddock 08 Jan '25

08 Jan '25

Just a gentle reminder that the review of the decimal number library by Matt Borland and Chris Kormanyos will start NEXT WEDNESDAY 15th Jan. The library is an implementation of IEEE 754 Decimal Floating Point Numbers. Documentation: https://cppalliance.org/decimal/decimal.html Code: https://github.com/cppalliance/decimal/ Motivation: Current C++ floating point types store the significand (often incorrectly referred to as the mantissa) as binary digits. Famously this leads to representation errors: https://0.30000000000000004.com <https://0.30000000000000004.com>. Decimal floating point numbers avoid this issue by storing the significand in base-10 (decimal). The other major difference between binary and decimal floating point types is that the latter allows for multiple representations of the same number. For example 1e5 could also be stored as 0.1e6, 0.01e7, so on and so forth. These are referred to as cohorts which binary does not have as there is only one way to represent each number in binary floating point. Use Cases: The use case for Decimal Floating Point numbers is where rounding errors are significantly impactful such as finance. In applications where integer or fixed-point arithmetic are used to combat this issue Decimal Floating Point numbers can provide a significantly greater range of values. For example, while a fixed-point representation that allocates 8 decimal digits and 2 decimal places can represent the numbers 123456.78, 8765.43, 123.00, and so on, a floating-point representation with 8 decimal digits could also represent 1.2345678, 1234567.8, 0.000012345678, 12345678000000000, and so on. I believe Matt has some minor doc updates to push between now and next week, but other than that the library is good to review (but hold your fire until we start next week unless you're away etc). Best, John.

1 0

Invalid sha256 for boost_1_87_0.tar.bz2 archive
by Samuel DEBIONNE 07 Jan '25

07 Jan '25

Dear all, In the conda forge CI (that builds the boost packages), I noticed that some jobs are failing due to a difference between the advertised and computed sha256 checksum. The json file at https://archives.boost.io/release/1.87.0/source/boost_1_87_0.tar.bz2.json has af57be25cb4c4f4b413ed692fe378affb4352ea50fbe294a11ef548f4d527d89 While sha256sum of https://boostorg.jfrog.io/artifactory/main/release/1.87.0/source/boost_1_87… gives 79e6d3f986444e5a80afbeccdaf2d1c1cf964baa8d766d20859d653a16c39848 Has the archive been updated and not the sidecar json file? Thank you, Samuel

2 2

Your Boost Talk at CppCon?
by Vinnie Falco 02 Jan '25

02 Jan '25

Hi everyone. I think it would be nice to have several talks about Boost for the 2025 CppCon program. We need volunteers who are able to attend and give a talk. If you are interested please email me at vinnie.falco(a)gmail.com to indicate your intent and propose a subject for the talk. There are some resources available to help volunteers get their abstract prepared for submission, to improve the quality of the proposal, ensuring a greater likelihood of the talk being accepted. Getting a Boost talk accepted has been hit or miss in recent years and I would like to change that. If we can get enough volunteers involved much earlier in the process (say, this month) then we have time to assemble a strong "Boost track" with high quality abstracts. This can incentivize the conference to make Boost talks an enduring feature of its programs going forward. If you have questions or comments about Boost talks at CppCon or other conferences, a discussion here is also welcomed. Thanks!

1 0

Re: [boost] announcing boost.async
by microcai 01 Jan '25

01 Jan '25

Klemens Morgenstern via Boost <boost(a)lists.boost.org> 于2023年3月17日周五 23:22写道： > Hi, > > I've been working on a C++20 coroutine library, called `async`, > which is meant to be for C++ what asyncio is for python or generic > async/await is for javascript. > It's meant to be a framework for single threaded coroutine applications. > > It provides coroutine types, such as promises, tasks and generators to > generators are useless for real word coroutine usage. > model async operations, > channels for inter-coroutine communication and functions > like select, gather & join to wait or combine multiple awaitables. > > It's asio-based, i.e. uses asio::io_context as its event-loop and can > co_await any asio operation. > > As this is a very complex domain, I really would be happy to have some > users. > This will help me to improve documentation & come up with more > example ideas. > > Docs: https://klemens.dev/async > Repo: https://github.com/klemens-morgenstern/async > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost >

2 1

[boost.context] questions about Lazy Float Point State Preservation
by microcai 01 Jan '25

01 Jan '25

Does jum_fcontext do lazy state preservation on FPU? most of the time, coroutine is executing async IO code rather than cpu intensive code. Mostly, IO code does not use floating point. but no preserving FPU state is error prone. Does fcontext employ "lazy state preservation on fpu registers"?

1 0

Adding optional<T&> find like method to associative containers in boost.
by Ivan Matek 28 Dec '24

28 Dec '24

Hi everybody, I presume somebody thought of this before but for some reason it never happened, but just in case it was never discussed I wonder if people here believe adding an nicer version of find(with different name obviously) to associative containers in boost would be beneficial? Since boost::optional allows references this would work. I presume naming is what most people will have strong feeling about. I personally prefer ofind/oat since I love short names, Barry suggested different names in his blog <https://brevzin.github.io/c++/2023/05/23/map-api/>, e.g. try_at. boost::optional did not get all the updates C++23 std::optional got, e.g. transform that make this usage compose nicely, but those could be added to boost::optional.

13 80

Formal review request for Hash2
by Peter Dimov 23 Dec '24

23 Dec '24

I'd like to request a formal review for the Hash2 library https://github.com/pdimov/hash2 https://pdimov.github.io/hash2/ (by Christian Mazakas and myself.) The library implements an extensible framework for implementing hashing algorithms that can support user-defined types. Its structure is largely based on the paper "Types don’t know #" by Howard Hinnant, Vinnie Falco and John Bytheway. https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3980.html We're looking for a review manager. Comments and questions are welcome; you don't need to wait for the review.

14 75

[hash2] Review Report
by Matt Borland 21 Dec '24

21 Dec '24

Dear all, Again, thank you to those who participated in the review of `Hash2` by Peter Dimov and Christian Mazakas from 7 to 15 December 2024. Bottom line up front: The library is ACCEPTED. By and large, the discussion around the library was positive, with reviewers appreciating that it implements the "Types don't know #" proposal. I think some discussion was perhaps too narrowly focused on providing hash algorithms, as that is not the over-arching goal of the library. Below is a summary of the main discussion points and recommendations from both reviewers and me. Discussion and Recommendations: 1) Support for `std::span`. Based on the discussion, it seems like the use of `std::span` has moved from theoretical to being quite popular. Adding an API for span would in principle be OK behind the requisite feature test macros, but it will add to the conceptual requirements which I'm not convinced is beneficial. Unless Peter and Christian intend to move to C++20, and they don't, I think it would be unreasonable at this time to require spans behind macros in the implementation to allow warning-free builds with Clang's `-Wunsafe-buffer-usage` and whatever memory safety profiles turn out to be. If this library was targeting C++20, or intended to be certified by a standards organization, this would be an entirely different conversation. 2) C++ language standard requirement. As I kind of expected, there was limited support for keeping the library at C++11. The authors should feel free to use C++14 and drop support for any non-conforming compilers before initial release. 3) Documentation. Peter Turcan provided a great deal of feedback on the documentation which I expect the authors to handle. There were also quite a few calls to re-arrange the documentation to put working examples and pitfalls discussed during review first, and details of concepts later. The single vs. multi-page documentation debate will never have a clear winner, so I defer to doc writer's preference on that one. The authors have a pretty good sense of the edits they need to make, so I won't belabor the point. 4) Security. Secure construction was included in the conditional acceptance criteria of two reviews - for example, default construction and allowing too-short keys in SipHash were considered harmful. There were also concerns that the algorithms provided would be considered cryptographically secure. Since the library does not intend to be a cryptographic provider (as that is not the real point of the library), I think an annotation at the top of the docs stating as much should be added. 5) Relationship to `ContainerHash`. Several reviewers noted that since there is no explicit Hash1, the relationship with `ContainerHash` should be explained. Peter currently maintains `ContainerHash`, so I think it would be useful to add similar documentation there as well. This could help avoid the situation we have now where `MP11` fully replaces the functionality of `MPL`, but that's not obvious to the casual outside observer. 6) The concept of HashAlgorithm. There was much discussion around the concept of HashAlgorithm and how it deviates from existing practice. If Peter and Christian don't want to be the sole providers of hash algorithms for their library, they need to provide: - A way to use existing implementations and/or examples in the documentation (this was part of the acceptance criteria for all but one conditional acceptance). This can be accomplished by adding Guidance and an example of an adapter. - An FAQ section addressing questions like "Why does the library impose additional requirements such as result extension on hash algorithms which go beyond the cryptographic specifications in corresponding RFCs?" Boost.Multiprecision already implements this approach to allow unknown arithmetic backends to be used. Final Verdict: I ACCEPT the library. The library was clearly well designed, and the authors provided clear justification for implementation decisions when questioned (e.g., `tag_invoke` for functions as customization points). Peter and Christian are both responsive enough that I expect they will address the above list without needing the forcing function of a conditional accept followed by a review in the future to ensure completeness. In fact Christian already showed me a draft of an OpenSSL adaptor. If this library was coming from an outsider, my opinion would be different. For transparency the final tally of reviews received was ACCEPT: 9, REJECT: 3, CONDITIONAL ACCEPT: 5. Thanks again to all that participated and congratulations to Peter and Christian. Happy Holidays, Matt

3 2

Query on std::variant support
by Murali Kishore 20 Dec '24

20 Dec '24

Hi, Want to check is std::variant allocating memory from shared memory using boost interprocess allocators. EX: std::variant<int, float> v, w; // Define types A and B that we will store in the variant struct A { int x; A(int val) : x(val) {} }; struct B { std::string str; B(const std::string& s) : str(s) {} }; // Define the variant type using MyVariant = std::variant<A, B>; guide me for the above examples to create custom allocators. -- Regards, Murali Kishore

2 1

Re: [boost] Interest for C++20 modules support of boost officially
by Ruben Perez 19 Dec '24

19 Dec '24

On Thu, 19 Dec 2024, 15:44 John Maddock, <jz.maddock(a)googlemail.com> wrote: > > > If I understood the CMake developers and Chuanqi correctly (please let > > me know if I didn't), you do install a library with the generated > > object files, but you do not install the BMIs, as they're like > > pre-compiled headers. > > > > Let me explain how I understood it with two concrete examples. Take > > Boost.Url as an example of a compiled library, and Boost.Regex as an > > example of a header-only library. > > > > 1. Boost.Url today installs: > > * libboost_url.a, containing function definitions. > > * Headers, containing declarations. > > 2. Boost.Url in the module world would install: > > * libboost_url.a, containing the same function definitions as above. > > * A boost_url.cppm file, akin to what headers are today. > > CMake would build a BMI from this file when a user needs to > > import boost.url. > > In practice, this cppm file will likely be implemented in terms of > the > > header files that we have today. > > 3. Boost.Regex today installs: > > * Headers, containing declarations and definitions. > > 4. Boost.Regex in the module world would install: > > * A libboost_regex.a object, containing module initializers [1]. > > For instance, if you have a singleton defining a Boost.System > > error category, > > initialization code for the singleton would be emitted here. > > From the docs, it seems this object gets created even if no > > initialization code is required. > > * A boost_regex.cppm file, containing the bulk of the library, akin to > > what we provide using headers today. > > > > My intention is doing some testing to see how much trouble would this > > new libboost_regex.a library cause. > > I'm not quite convinced it's that simple, at it's most basic, consider > an inline function that is part of the module interface, if I compile > the module's object library for release, and then do a debug build of > the bmi and my project what happens? Even if clang makes that example > work out in msvc land where almost every option creates binary > compatibility issues, I see users compiling the object library from > source on a per-project basis. > I'd expect the inline function to be part of the BMI (which gets built by the user), and not part of the object file, thus not creating trouble. Of course, I could be wrong, so I'll try it. Regards, Ruben.

1 0

Boost 1.87.0 Release Report
by Vinnie Falco 17 Dec '24

17 Dec '24

We are proud to publish the first Boost Release Report, for 1.87.0. This report is designed to showcase and credit the work of the many volunteers who contribute to the libraries and other things in Boost, published as an official release three times a year. https://downloads.cppalliance.org/reports/boost_1_87_0_release_report.pdf Thanks

4 8

[hash2] Review of Hash2
by Antony Polukhin 16 Dec '24

16 Dec '24

> - What is your evaluation of the design? The design is good, but there is a major issue with security/safety in default constructors of the hash algorithms. ## HashDOS issue The issue is related to HashDOS attacks. The default seed must not be 0. Because with 0 the attacker could easily find strings of the same size that result in the same trailing bits of the hash result. As a result on those strings all the operations with unordered containers degrade to O(N) for each string, giving O(N*N) complexity for a set of strings. The issue affects any parsing where the result is stored into an unordered container: HTTP headers, HTTP cookies, JSON keys, XML... Our in-house experiments show that a single laptop can easily put down a few hundreds of servers just by sending the same precomputed HTTP request. The servers start to use 100% CPU and do not respond to other requests. The problem is so critical, that for some time we write HasDOS tests to make sure that our data structures are not affected: https://github.com/userver-framework/userver/blob/develop/universal/src/htt… With some math knowledge the attack becomes even more simple see https://orlp.net/blog/breaking-hash-functions/ Quick fix for the HashDOS is to generate a random number at the program start use it to initialize all the hashes on construction. Is still leaves a theoretical possibility of attack if there's only one instance of the application or the attacker generates strings set for each instance and sends them in one go. We were not patient enough to generate the exploit, which does not actually mean that it is not possible. A more mature fix is to generate a random seed for each request or container. We were using that approach for quite some time https://github.com/userver-framework/userver/blob/d09b427b42fc80dcd66b8161d… . We also checked some other popular frameworks for the HasDOS issue. Too many of them were affected (we reported to the authors, but not all of them reacted to the problem). All of the above leads to the following conclusion: developers are not aware of that critical issue or do not take the issue seriously. To not provoke vulnerabilities the hashing must be safe by default. ## tag_invoke is dreadful tag_invoke is over complicated. It may make sense if there is a need to specialize the behavior not just for the user defined type, but also to specialize behavior of the user defined type depending on the hashing algorithm and/or flavor. Such functionality does not seem to be required here. So instead of ``` template<class Hash, class Flavor> friend void tag_invoke( boost::hash2::hash_append_tag const&, Hash& h, Flavor const& f, X const& v ) { boost::hash2::hash_append(h, f, v.a); boost::hash2::hash_append(h, f, v.b); } ``` how about using a simpler approach: ``` template<class Visitor> friend constexpr void hash_visit(Visitor visitor, X const& v) { visitor(v.a); visitor(v.b); } ``` where Visitor could be ``` [&h, f](const auto& value) { boost::hash2::hash_append(h, f, value); } ``` ## std::hash and hash_value I'd appreciate functionality to reuse boost::hash2 functionality with std::hash and hash_value. It would be ideal to have something like: ``` template <class T> requires boost::hash2::supports_v<T> struct std::hash { std::size_t operator()(const T& x) const { boost::hash2::default_hasher h{boost::hash2::on_program_start_seed}; boost::hash2::hash_append(h, {}, x); return boost::hash2::get_integral_result<std::size_t>( h.result() ); } } template <class T> requires boost::hash2::supports_v<T> std::size_t hash_value(const T& x) { // put into Boost.Hash return std::hash<T>{}(x); } ``` So that the user just needs to provide boost::hash2 customization for its type, and after that it works with the standard library hashing and old boost hash. > - What is your evaluation of the implementation? Very good. > - What is your evaluation of the documentation? Good. > - What is your evaluation of the potential usefulness of the library? Do you already use it in industry? Not used it. The library is very useful because it researches the constexpr hashing, proposes hash combining and attempts to solve the other std::hash issues. > - Did you try to use the library? With which compiler(s)? Did you have any problems? Not used it. > - How much effort did you put into your evaluation? A quick reading of sources and docs. > - Are you knowledgeable about the problem domain? Yes, but not an expert. > Ensure to explicitly include with your review: ACCEPT, REJECT, or CONDITIONAL ACCEPT (with acceptance conditions). CONDITIONAL ACCEPT. The behavior of the default constructor must be secure. `tag_invoke` seems to be an overkill.

4 5

[hash2] Formal Review Begins
by Matt Borland 16 Dec '24

16 Dec '24

Dear all, The review of Hash2 by Peter Dimov and Christian Mazakas begins today Saturday, December 7th through Sunday December 15th, 2024. Code: https://github.com/pdimov/hash2 Docs: https://pdimov.github.io/hash2/ Please provide feedback on the following general topics: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? Do you already use it in industry? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? Ensure to explicitly include with your review: ACCEPT, REJECT, or CONDITIONAL ACCEPT (with acceptance conditions). There's already been a flurry of conversation so I thank you all for that. If you have any questions during this process feel free to reach out to me. Additionally, if you would like to submit your review privately, which I will anonymize for the review report, you may send it directly to me at: matt(a)mattborland.com. Matt Borland Review Manager and C++Alliance Staff Engineer

20 94

[hash2] Review
by Julien Blanc 13 Dec '24

13 Dec '24

his is my review of the proposed boost::hash2 library # What is your evaluation of the design? The base idea, separating the hash algorithm from the actual hashing of objects, is a brilliant one. The library consist on two parts: * a collection of hashing algorithm, some of them of crypto quality * a c++ objects hashing facility Given this design, i would not have expected from the library to provide a thorough collection of hash algorithm. But it's a good think to get them. I would have expected, however, to see some guidance about how to integrate an external hashing library. Actually it is unfortunate that this part is missing, because it seems a lot more harder than it should be. Several design decisions could be improved in my opinion: * the requirements on the HashAlgorithm concept are too strict, and have no rationale in the documentation * copy constructibility and copy assignment are unusual across current hash implementations, as it has been noted by other reviewers. I can understand a rationale for copy construct, copy assignment looks very weird to me. The only rationale i can think of is resetting the internal state to a defaulted state, which could be a specific operation. * creation with seed (either as uint64 or as an array) as well as default construction is required. It seems odd. The documentation, for example, strongly advice agains using siphash without a seed. I would prefer that it is a compile time error to use siphash without a seed / with an incorrectly sized seed. I've no problem adding a key/seed to algorithms that do not mandate them, though. But, since the library itself does not create the HashAlgorithm themselve (there is one notable exception to that that), it should not put additional requirements beyond what it uses. The HashAlgorithm concept could be: struct HashAlgorithm { using result_type = /*integral or array-like*/; static constexpr int block_size = /*...*/; // optional void update( void const* data, std::size_t n ); result_type result(); }; Interestingly enough, most of object hashing works if you provide it with such an algorithm (not copyable and not default constructible). I failed to see where default construction is used outside tests, for example. * the usage of void const* + size in interface is controversial, as several people would prefer a span<>. There are good reasons for not using span, and this is not something that should be an issue, as this should only impact HashAlgorithm writers. It is in my opinion an unfortunate design choice that this internal interface is leaked to the end user. The functionality is already provided by hash_append_range, which could be the advertised way of hashing bytes via that library (ie, when mixing hashing of object and hashing of bytes, eg when implementing custom hashing for a type). * constexpr use mandates usage of unsigned char const*. So it could be the primary supported interface over void const*, to avoid interface duplication. * the requirement on result extension is also uncommon. While i understand the rationale, it should be noted that it is likely to cause some language interoperability issues, as it is not necessarily specified by the algorithm. Implementations in other languages are unlikely to provide them, making the feature a niche one, unless the underlying algorithm specification provides it. * most of the functionality is provided by the hash_append function, and the hash_append_xxx helpers. This is a neat design. * The Flavor template argument allows to customize hashing of objects, to get platform-independants hashes. This is a good idea, but it comes with some problems: * the flavor is repeated over each hash_append call. I don't think it is a realistic use case that the flavor changes over time. If i were to integrate the library in our code base, the first thing i'll do is write a wrapper to get rid of that flavor argument at every call site. * the default flavor uses std::uint64_t for size. It seems odd. If i'm not asking for a specific flavor, i'm not asking the hash to be consistent accross different systems. So std::size_t should be used there instead. This looked to me like a pay for what you don't use situation, so i ran a quick near worst-case benchmark on our arm platform (hashing a million times a std::vector of the three integers 1-2-3), to find out that it makes a the code using size_t runs 15% faster. # What is your evaluation of the implementation? The implementation looks good. Code is clean, very readable. A minor remark in hash_append_unordered_range : m should be declared as std::size_t (or std::uint64_t). The current typename std::iterator_traits<It>::difference_type m = 0; is wrong. I'm not at ease with hash_append_unordered_range. The fact that it reduces the hash to 64 bits, simply making a sum of the element hashes, and then hashing that, does not look right. If the goal is simply to avoid unintended collisions, it's probably fine. If the goal is to preserve against an attacker mindfully trying to provoke collisions, it may do the trick,although intuitively xoring the whole results would be better at that job than summing truncated results. If it's to preserve cryptographic properties of the hash function, then it looks wrong. Another remark is about size serialization. The library relies on std::tuple_size to identify container-like structs, but not include the size in the hash (or is it because of is_contiguously_hashable?). This leads to unintuitive results, such as: std::array<int, 4> a{1,2,3,4}; std::span<int, 4> s(a); Hash h1,h2,h3,h4; hash_append(h1, {}, a); hash_append_range(h2, {}, begin(a), end(a)); assert(h1.result() == h2.result()); // TRUE hash_append(h3, {}, s); hash_append_range(h4, {}, begin(s), end(s)); assert(h3.result() == h4.result()); // FAILS I'm not sure there's a general solution to this issue, though. But since std::array models a container, not including its size in the digest may not be the wisest choice (the doc talks about plain arrays, not about std::array, so i don't know if it's a bug or a feature) # What is your evaluation of the documentation? The order of the documentation is odd. I would expect it to first explain me how i use the library with the provided algorithm to hash c++ data or bytes, but it first goes into deep details about how for example how result() can be called multiple times, or what requirements are set on custom hash algorithms. A description of the scope of the library would be welcomed. It both provides hashing for use by containers, and cryptographic quality hashing. These two seems at first unrelated, and the library lies in a not clearly defined in-between. It has been explain during the review, and the arguments given are convincing, and should find their place in the documentation. The motivation for machine architecture independant hashes is lacking. Usually, the way to achieve this is to use a domain specific canonical form, because endianness or word size is unlikely to be the only variable at stake here. But overall, the documentation has all the relevant information. A tutorial / quick start would be a nice addition. I dislike the "single page style" documentation, and hope the tooling can be improved in this regard. # What is your evaluation of the potential usefulness of the library? Do you already use it in industry? I have a need for an easy to use sha1, that i can easily compile for embedded targets. Having that in boost actually lowers the number of external dependancies, so yes this is definitely useful. I currently don't have a need for the object hashing facility, but would definitely use the library it if the needs arise. # Did you try to use the library? With which compiler(s)? Did you have any problems? I tried with several flavors of gcc, cross-compiling for bare-metal cortex-m0 targets or embedded arm32 linux, as well as standard debian x64. Everything worked. # How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I did not review the algorithm implementations. I spent around 15 hours on studying the library doc and internals. # Are you knowledgeable about the problem domain? No specific knowledge of the domain. # What is your final recommendation I would recommend to CONDITIONALLY ACCEPT the library into boost, the conditions being: * demonstrate the usage of an existing external hash library into the boost::hash2 framework * remove default construction and too short key construction for algorithms where this is actually harmful (such as siphash). I don't have an issue with providing constructors with keys for algorithms that do not mandate it. But boost code should encourage correct usage of the algorithms it propose, not allow bad usage in the name of genericity. The following improvements should imho be done, but i don't base my conditional acceptance on them. Actually some of them can be done on top of the proposed library: * advertise hash_append_range as the recommended way to add untyped data to the hash algorithm (or define an additional hash_append_bytes to fulfill this role). * adds a tutorial / introduction in the doc, move the whole hashing byte sequences item after the hashing c++ objects item (if previous point is adressed, part of it would have to be included in the hashing c++ objects item). * makes copy construction requirement on hash algorithm a requirement only if hash_append_unordered_range is used. Hashing unordeder ranges is an open problem, some creative other solutions may be found, and i see no reasons why they would involve duplicating the algorithm state. Actually, if the hash is default constructible, then it is not needed. You need either one of default or copy constructible, not both. * use std::size_t instead of std::uint64_t for the default_flavor, as this is faster on some platforms. * provide wrappers that avoid repeating the flavor over and over. It is much more convenient for the user to write once using MyHash = hash2::FlavoredHash<hash2::default_flavor> and then MyHash::append(X) than repeating over and over the parameter. Finally, i want to thank Peter for developing and proposing this library for boost, and Matt for managing the review. Regards, Julien

3 3

Ruben's Hash2 review
by Ruben Perez 13 Dec '24

13 Dec '24

Hi all, This is my review of the proposed Boost.Hash2. Thanks Peter and Chris for submitting the library, and Matt for managing the review. - What is your evaluation of the design? Very good. This is how hashing C++ objects should work, as it properly separates concerns between types and hash functions. I think usability could be improved if the template class hash shown as an example was included as part of the library and introduced much earlier in the documentation. In general, the HashAlgorithm concept makes sense. I'm not convinced about the HashAlgorithm::result() interface, for two reasons: * It makes integration of third-party hashes more difficult, as already noted by other reviewers. * The naming suggests an accessor, rather than a modifier. I'd consider splitting it into two functions, maybe finalize_block() and result(). But I can also see the benefits of the approach taken. I miss some higher-level features when working with untyped byte sequences. While this doesn't seem the main value proposition of the library, it's listed as a valid use case. See below for more info. These are nice-to-haves rather than requirements, though. I was initially surprised to find out that the library was header-only, as I didn't know that compile-time hashing was something worth it. I was initially worried about compile-times. I built one of the examples with compile-time profiling (-ftime-trace) and the library is pretty lightweight, so I have no problems with the header-only design. Have you considered somehow supporting hashing for objects describable via Boost.Pfr? I had it requested by some users in MySQL even when I had built-in Boost.Describe support, so it may get requested here, too. - What is your evaluation of the implementation? The parts where I've peaked in are good. Some comments would have helped me, but in general I've been able to follow. The test suite looks pretty comprehensive, too. Some fuzz tests for the cryptographic algorithms would be appreciated (I can help if required). As other reviewers have commented, the algorithm for hashing unordered ranges can use some improvements to make it more secure. I'm no security expert and can't judge how important this is, but I'm pretty sure the author will solve this promptly. The implementation of cryptographic functions is currently slower than solutions like OpenSSL [4], but the author has manifested the possibility of closing this gap. It would be nice having HashAlgorithm modelled as a C++20 concept, although the required macro machinery required to perform the checking only if concepts are supported (like BOOST_ASIO_COMPLETION_TOKEN_FOR does) may be too much effort for the benefit it brings. Regarding Boost integration, a build.jam should likely be added, so the library integrates in the new modular Boost infrastructure (although I've been able to use the library fine without it). This can be done after acceptance. As a matter of curiosity: which compilers do not support __builtin_is_constant_evaluated? (as they will always get the slow memcpy implementation, even at runtime). - What is your evaluation of the documentation? The documentation is good, and explains rationales pretty well, but the order could be improved: * Most users will likely want to use the existing algorithms with their unordered containers, so this should be explained first. * The next block of users will likely want to use the existing cryptographic hash algorithms to untyped byte sequences, so this should be next. * The rest can be left as is. I miss the motivation for the flavor construct in the docs. If I understood correctly, it's only relevant if you plan to somehow serialize your hashes. Maybe an example could help. The library is also suitable for use cases that combine the C++ object hashing and untyped byte sequence hashing parts, as explained by the author here: https://lists.boost.org/Archives/boost/2024/12/258765.php. Such cases are the rationale behind having cryptographic functions within the same umbrella as non-cryptographic ones. I'd suggest adding these to the docs. In general, I dislike single-page docs (which unfortunately seem pretty popular in Boost), as they're difficult to search. I also miss linking (e.g. hash_append should contain a link to its description in the reference). It would also be beneficial to explain the library's relationship with boost::hash / Boost.ContainerHash. In general, making it more approachable for non-expert users would be good. - What is your evaluation of the potential usefulness of the library? Do you already use it in industry? Robust, extensible C++ object hashing is very useful. My immediate use case is related to hashing untyped byte sequences, though. MySQL/MariaDB authentication involve a challenge-response protocol that involves hashing the user-supplied password with a server-supplied challenge, using either SHA1 (sigh) or SHA256 [1]. I'm currently using OpenSSL, as I use it for TLS, too. This library would allow me to make the OpenSSL dependency optional (which I've been requested). I'm also planning on writing a Postgres protocol library. It will likely need to support SCRAM-SHA256 authentication, similar to what I've described. This library could be handy for this. - Did you try to use the library? With which compiler(s)? Did you have any problems? I've re-written the password hashing functionality in Boost.MySQL to use the proposed library [2]. Everything has been very smooth. The test suite passes for all the compilers I support [3], which is really good. I'd have liked some utilities on top of HashAlgorithm to make hashing byte sequences safer and easier. These utilities would *not* imply any change in the HashAlgorithm interface: * Adding a contiguous range of bytes to an algorithm currently supports only a pointer + size pair. This is the right choice for HashAlgorithm, but having a helper function on top of it would be great, as proposed by other reviewers. This would make my code go from: // password is a string_view hash2::sha1_160 hasher1; hasher1.update(password.data(), password.size()); auto password_sha1 = hasher1.result(); To: hash2::sha1_160 hasher1; hash2::hash_add_bytes(hasher1, password); auto password_sha1 = hasher1.result(); * Adding a function that computes the hash of an entire byte sequence, as a range, could be useful. OpenSSL has this as SHA1() and SHA256(). This would make my code go from: hash2::sha1_160 hasher1; hasher1.update(password.data(), password.size()); auto password_sha1 = hasher1.result(); To: auto password_sha1 = hash2::compute_hash(hash2::sha1_160(), password); These are just nice to have and trivially implemented in user-land, if required. The code is already good as is today. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I have dedicated around 8 hours to study the documentation, build the library and examples, integrate it into Boost.MySQL, update my CIs to run my tests with the library, and write this review. - Are you knowledgeable about the problem domain? I am a mere user of hash algorithms, as I usually write networking code. I don't know about their internals. I haven't authored associative containers, either. - Affiliation disclosure I am currently affiliated with the C++ Alliance. - Final decision My recommendation is to CONDITIONALLY ACCEPT the library into Boost. My only acceptance condition is adding the hash template class developed in the examples to the library. The benefit of having the library in Boost is big and overcomes any concerns I might have. Many thanks to both Peter and Matt. Kind regards, Ruben. [1] https://dev.mysql.com/doc/dev/mysql-server/8.4.3/page_caching_sha2_authenti… [2] https://github.com/boostorg/mysql/pull/389 [3] https://github.com/boostorg/mysql/blob/c438f26731e36c2db6457705ec5dbb9f7657… [4] https://lists.boost.org/Archives/boost/2024/12/258687.php

4 4

Re: [boost] [hash2] My Review
by Vinnie Falco 12 Dec '24

12 Dec '24

On Sat, Dec 7, 2024 at 5:47 AM Matt Borland via Boost <boost(a)lists.boost.org> wrote: > The review of Hash2 by Peter Dimov and Christian Mazakas begins today > Saturday, December 7th through Sunday December 15th, 2024. > > Code: https://github.com/pdimov/hash2 > Docs: https://pdimov.github.io/hash2/ Disclaimer: I am affiliated with The C++ Alliance and I am also a co-author of N3980 (the paper referenced by the library). This is my review of Hash2. Spoiler alert, my suggestion is to: ACCEPT While most of my review is dedicated to discussing some concerns I have about some aspects of the library, it is clear that Boost as a collection is better with Hash2 than without it. Also, Peter is going to do what Peter is going to do so why fight it. The uncontroversial parts are stated first, and after lies my assessment of some questionable design choices. > - What is your evaluation of the implementation? > I looked over some of the implementation to better understand how certain things work. The tests pass on CI and the test code looks comprehensive. Peter is not in the habit of producing anything less than high-quality implementations and this library is no exception. I don't know which parts are from Peter and which are from Christian and the quality appears high throughout. > - What is your evaluation of the documentation? > I think the documentation could be better in places. The named requirements exposition is too informal for my taste and talks about things that are not strictly relevant, such as zero-padding. The library alludes to cryptographic usage yet glosses over some details which surfaced during mailing list discussion (more on that below). It would be nice if there was some exposition up front explaining which groups of users the library is for, and how each group uses the library. > - What is your evaluation of the potential usefulness > of the library? Do you already use it in industry? > Not only is the library useful, but the C++ Standard is poorer for not having a modern hash solution. Competing approaches were developed by Google and N3980, then promptly abandoned. > - Did you try to use the library? With which compiler(s)? Did > you have any problems? > I tried to build a Visual Studio Solution from the CMakeLists.txt after cloning hash2 into my superproject, with this command line: cmake -G "Visual Studio 17 2022" -A x64 -B bin64 -DCMAKE_TOOLCHAIN_FILE="C:/vcpkg/scripts/buildsystems/vcpkg.cmake" -DVCPKG_CHAINLOAD_TOOLCHAIN_FILE="C:/users/vinnie/src/toolchains/msvc.cmake" It started downloading things from the internet (FetchContent I guess) and then failed. I was not able to generate a solution. I asked Peter about this and he said it was not supported. It would be nice if it was. > - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? > I focused primarily on the ways in which the library differs from the original paper N3980. I looked at the relevant documentation, interfaces, and implementation, and engaged the author on the mailing list. - Are you knowledgeable about the problem domain? Fairly knowledgeable. - What is your evaluation of the design? This is where I have concerns. The audience of users for Hash2 can be divided into roughly these groups: 1. Authors of types who just want to make objects of their type hashable without having to learn about hash algorithms cryptography, and users who want higher quality results when working with unordered containers. Hash2 is basically perfect for these two groups of users (which often overlap) although the documentation could be improved to speak more directly to them. 2. Users who wish to calculate a cryptographic digest for objects of Hash2-enabled types, or for untyped binary data. Hash2 is also almost perfect for these use cases, and as before the documentation could be more explicit about these cases. 3. Users who wish to adapt existing code to meet the HashAlgorithm requirements from Hash2. This is where I believe the library has some design problems. An unfortunate consequence of being right almost all the time, is that it becomes more unlikely to recognize when you are wrong. I will assume that the reader is at least somewhat familiar with N3980 and some of the arguments I made about the HashAlgorithm named requirements on the mailing list. Hash2 attempts to improve the quality of life for users by mandating additional requirements for the HashAlgorithm concept which may be difficult or impossible to meet for some existing libraries. For example, it requires that every HashAlgorithm support: * Seeded construction from std::uint64_t * Keyed construction from binary data * Copy construction * Digest extension from repeated calls to result() * Interleaving of calls to update() and result() * ...Future stuff that the author alludes to These issues above are already discussed extensively on the mailing list and I can't be bothered to spend hours putting it all together into a beautiful post the way that some of the other reviewers have done (Andrey Semashev and Samuel Neves come to mind). The burdensome additional named requirements of Hash2's HashAlgorithm ensure that no casual users will be authoring them. Or worse, they will get them wrong and the results will not have the same guarantees as the underlying algorithms. Right now there are approximately 1,649,127 implementations of cryptographic and non-cryptographic hash algorithms. I think it might be more prudent for Hash2 to offer a system of traits for identifying which algorithms are capable of what, such as CopyConstruction, digest extension, and so on, and give the user tools which only work in safe ways given the traits of a hash algorithm. This would make it easier to write HashAlgorithms. And as a consequence we might get many more of them rather than fewer. Realistically I don't expect the library to change, and a proliferation of user-created HashAlgorithms is probably conditional on the Boost library collection being much more popular than it currently is. This is partly why I have decided to just accept the library. And I could be wrong, and Peter's unwavering insistence that users get HashAlgorithms which all offer the same quality of service might turn out to be better. Time will tell. Thanks

2 1

[hash2] Difference from paper: HashAlgorithm interface vs. function object
by Andrey Semashev 12 Dec '24

12 Dec '24

Hi, The HashAlgorithm interface defined in Boost.Hash2 is notably different from the one described in N3980 (https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3980.html) Specifically, the paper defines HashAlgorithm as a function object, whose operator() is equivalent to Boost.Hash2's HashAlgorithm::update method. The function object also provides a conversion operator to result_type, which is equivalent to Boost.Hash2's HashAlgorithm::result method. Is this discrepancy intentional and are there plans (on the paper authors' or the library authors' side) to eliminate it? Personally, I don't think a function object is the right interface for a hash algorithm. Update and finalize are two distinct operations provided by hash algorithms and both should be performed explicitly by the user. I don't think a conversion operator is the right name for finalize. Conversion operators are normally used when one type can meaningfully "impersonate" another, and this is not the case here. The function object is also misusing result_type, as its operator() doesn't return it. The paper discusses that the fact that a hash algorithm is a function object can be used to wrap it in std::function, which can be useful with pimpl idiom and for the purpose of type erasure. I don't think this will actually work, at least not as simple as the paper describes it, because std::function won't allow to extract the computed hash digest (i.e. you won't be able to call the type conversion operator on the wrapped function object). I think, the user will have to write some wrapper classes anyway to retain access to the wrapped hash algorithm. However, this is still an interesting use case. Does Boost.Hash2 intend to provide utilities to address it? For example, Boost.Hash2 could provide a hash-algorithm-agnostic interface for feeding an algorithm with input data, and a wrapper for an external algorithm object that would implement this interface. Something along these lines: class polymorphic_hash_algorithm { public: polymorphic_hash_algorithm( polymorphic_hash_algorithm const&) = delete; polymorphic_hash_algorithm& operator=( polymorphic_hash_algorithm const&) = delete; virtual void update(const void* data, size_t size) = 0; protected: polymorphic_hash_algorithm() = default; ~polymorphic_hash_algorithm() = default; }; template< typename Hash > class polymorphic_hash_algorithm_wrapper : public polymorphic_hash_algorithm { public: using hash_type = Hash; using result_type = typename hash_type::result_type; static constexpr size_t block_size = hash_type::block_size; private: Hash& h; public: explicit polymorphic_hash_algorithm_wrapper(hash_type& h) noexcept : h(h) {} void update(const void* data, size_t size) override { h.update(data, size); } result_type result() { return h.result(); } }; This way, polymorphic_hash_algorithm could be used to abstract away the hash algorithm type while still be compatible with hash_append and friends. As a side note, this would be another use case where the hash algorithm passed to hash_append is non-copyable. And while copyability could be added, most likely it wouldn't be free.

6 11

The Master branch is now open for changes
by Marshall Clow 12 Dec '24

12 Dec '24

Thanks to everyone for their patience. The next release cycle will begin in late February. — The release managers.

1 0