September 2020 - Boost - lists.preview.boost.org

[LEAF] Review Results
by Michael Caisse 09 Sep '20

09 Sep '20

I want to thank Emil for his patience for the publishing of the review results as life got in my way. Thank you Emil for submitting LEAF to be reviewed for inclusion into Boost. As an already-Boost-author, you knew what you were getting into and continued into the process. I also want to thank you for once again pushing our thoughts on error handling. -------- Result -------- This is the part everybody scrolls to the bottom to read, so let’s just put it at the top. Boost.LEAF is ACCEPTED into Boost. After the following modifications are made to the library, the repository can be transitioned to Boost for inclusion into the next release (with the release cycle timing constraints): Specify the TLS requirement sooner in the documentation. Optional usage of Boost.Config to perform all of the Boost Configy things. (further discussion on this below) Macro naming and general Boostification clean-up. -------- Reviews -------- There was a good amount of discussion about the library on both the dev and user mail lists. I want to personally thank those who got involved in the review. You make the process work! Result summary: Accept: 7 Reject: 1 No-decision: 3 The no-decision reviews provided valuable input and discussion threads. While it would be my preference to have the review authors provide a final verdict, I was grateful for the time and energy they spent with the library and formulating thoughts into emails for discussion. Some comments from reviews: “I find function preload() really great and novel. … This really makes the error reporting code brief, clean and bug-free.” “I like LEAF very much and there's a lot offered here…” “I like it.” “...it allows bridging multiple error handling strategies and usage patterns in an efficient way.” About the docs: “They look good and manage to explain hard concepts in a very approachable fashion.” “The documentation is great -- in terms of styles it's probably the best I've seen for a C++ library.” “Clear, friendly and complete.” -------------- Some Thoughts -------------- While there was overwhelming consensus that LEAF should be included into Boost, the process is not democratic. I have spent considerable time understanding the concerns raised in each of the reviews and in particular the singular rejection. Emil took time responding to each of the reviews. The exchanges were fruitful and both “conditional accept” and “reject” reviewers changed their verdicts to accept along the way. Most of the concerns were amicably resolved. I am comfortable with the design choices and implementation made by the author for the remaining items. One reviewer brought up the re-implementation of Boost.Config type facilities. This of course is a complex issue for a library with zero dependencies. The ability to simply drop the Boost.LEAF directory into a project and run is a definite plus for a specific user segment. Being able to modify a single Boost.Config repository to support WackyCompiler14 targeting LEG Core and get a bunch of libraries for the ride is also a plus. Emil and I have discussed these concerns and the best choice seems to be defaulting to zero dependencies and the option to pull in Boost.Config to provide configuration type facilities if needed. Accepting a library into Boost is as much about the library as it is bringing the author into a group. While Emil is already part of the author crowd, it was clear to me through his careful, informative, and patient responses that the addition of Boost.LEAF brings professional maturity that strengthens the community. A user with more than a year of experience with the library voted acceptance based on the “qualities of the library itself and the responsiveness and openness for feedback of its author.” Throughout the review period, Emil integrated feedback to both the documentation and code in another branch. It was wonderful to see the community provide input and the author respond so quickly to produce a better final result. Thank you Emil for your vision of better error handling and striving to bring us along! -- Michael Caisse Ciere Consulting ciere.com

3 2

[log] logfile not created where expected
by Barend Gehrels 08 Sep '20

08 Sep '20

hi, I encountered a bug, most probably, in Boost.Log It is reproduceable in the tutorial_logging.cpp : https://www.boost.org/doc/libs/1_74_0/libs/log/example/doc/tutorial_logging… The short code should create a file named "sample.log". But it does not, instead it creates a file called "00000.log", with the expected lines logged in that file. If I compile and run exactly the same tutorial in Boost 1.70, the file sample.log is indeed created with the same content. The problem is already there in Boost 1.73 What is the problem? Is there a workaround for it? Thanks, regards, Barend

3 2

Re: [boost] Review Manager needed for Boost.Lambda2
by Mateusz Loskot 08 Sep '20

08 Sep '20

Bjorn, thank you for reminding me about it. I've been offline for several weeks and I might have missed it happened already (lots of communication moved to Slack too). Mateusz Loskot, mateusz(a)loskot.net (Sent from mobile, may suffer from top-posting) On Tue, 8 Sep 2020, 20:39 Bjorn Reese, <breese(a)stofanet.dk> wrote: > On 2020-09-08 19:39, Mateusz Loskot via Boost wrote: > > On Tue, 8 Sep 2020 at 17:02, Peter Dimov via Boost > > <boost(a)lists.boost.org> wrote: > >> > >> I turned this into a mini-library, https://github.com/pdimov/lambda2, > and > >> I'd like to submit that to Boost. > >> > >> I'm looking for a review manager. > > > > FYI, added to the schedule. > > Strictly speaking, he needs an endorsement first. >

2 1

buffer_size behavior in boost::iostreams::filtering_istream
by Justin McManus 07 Sep '20

07 Sep '20

I have some code that works as intended, but it requires setting a buffer_size parameter to zero on a std::ifstream pushed onto a filtering chain, and I'd like to understand why, to ensure I'm not introducing a bug or a hack. I have essentially the following code: -------------------------------------------------------------------------------------------------------- std::ifstream m_jf("json_filename", std::ios_base::in | std::ios_base::binary); std::locale utf8_locale("en_US.UTF-8"); m_jf.imbue(utf8_locale); boost::iostreams::filtering_istream m_inbuf; m_inbuf.push(boost::iostreams::bzip2_decompressor()); m_inbuf.push(m_jf); std::string m_line; while (std::getline(m_inbuf, m_line)) { // Process the current line from the JSON file } -------------------------------------------------------------------------------------------------------- What I find is that the std::getline call will fail before the code has reached the EOF. It will always fail at the same line in a given JSON file, but it will fail on different lines in different JSON files. It's perfectly reproducible. However, if I change lines 4 and 5 to m_inbuf.push(boost::iostreams::bzip2_decompressor(), *0*); m_inbuf.push(m_jf, *0*); then the problem goes away. My question is, Why does setting the buffer_size parameter to zero solve the issue? What does this do, exactly? I saw the suggestion to set the buffer size this way from an old post in 2009, and it appears to work, but I'd like a deeper understanding of what's happening under the hood. If the buffer size is set to zero, what does the underlying implementation do, and how might this influence whether std::getline fails before the EOF? Thanks very much, Justin -- Justin McManus, Ph.D. Principal Scientist Lead Computational Biologist and Statistical Geneticist Kallyope, Inc. 430 East 29th Street, Suite 1050 New York, NY 10016

3 4

type detection macros + multiprecision + io
by Gero Peterhoff 07 Sep '20

07 Sep '20

Hello Developer, I noticed that some type detection macros and boost::float128_type or numeric_limits do not work reliably: - BOOST_HAS_FLOAT128 with clang and icc do not work, gcc is ok - BOOST_HAS_CHAR32/16_T, BOOST_NO_CHAR32/16_T are not defined, but either BOOST_HAS_TYPE or BOOST_NO_TYPE should be defined; see later - with clang there is no boost::float128_type and an error when including boost/multiprecision/float128.hpp (quadmath.h) - with clang and icc the numeric_limits<float128_t> are not available (gcc ok) see https://godbolt.org/z/95PfTG In general it would make sense: - convert the detection macros to a uniform naming scheme: always BOOST_HAS_TYPE_T (with _T at the end) or BOOST_HAS_TYPE (without _T), but not mixed/both - in multiprecision and math: 1) float: renaming typenames to eg floatBits_mbt (m: multiprecision, b: binary); floatBits_mdt (d: decimal) 2) integer: renaming typenames to e.g. u/intBits_mt; u/intBits_mct (c: checked) 3) based on this, renaming boost::float/int128_type to boost::float/int128_t so that it is always clear and there is no confusion. You can do that for all types, e.g. some DSP/microcontrollers do not support u/int8/64_t etc. I did that quickly for gcc (has_type.hpp) - of course that has to be in the corresponding boost/config/compiler/* and platform/* headers are implemented. Building on this, feature test macros can then be made available (features.hpp). Can you please implement the io-operators and string-conversions for u/int128 (in e.g. boost/math/cstdint/cstdint_iostream.hpp)? thx and regards Gero

5 4

[sync] Documentation ?
by Edward Diener 06 Sep '20

06 Sep '20

There is a 'sync' library in the Boost build tree as a submodule, but I see no documentation for it in Boost 1.74 itself. Is this currently a Boost library, or what is it ?

2 1

Cryptography for Boost. Boost.Crypto3.
by Mikhail Komarov 04 Sep '20

04 Sep '20

Hello! Some time ago I promised to show something about cryptography library architecture and implementation for Boost (https://lists.boost.org/Archives/boost//2020/02/248205.php <https://lists.boost.org/Archives/boost//2020/02/248205.php>), so here I am. I’m actually glad my two innocent letters triggered such a conversation. That is what I wanted to see! Please, read this letter very carefully, because some explaining needs to be done before you go to the code. By the way. The code: https://github.com/NilFoundation/boost-crypto3 <https://github.com/NilFoundation/boost-crypto3>. Warning. This letter contains mentioning of my non-commercial organizations I develop Boost.Crypto3 with. Don’t consider that as advertising. It is not supposed to be an advertisement. So, what cryptography for Boost (and probably STL some time after) should look like? 1. It should definitely be simple to use and implemented with some generic programming spirit. That is exactly what was done. Remember std::transform? Typical usage technique for all of the modules of a suite looks as follows. Classic iterator pair with output iterator: using namespace boost::crypto3; std::string input = “Weird German 2 byte thing: Ã.“, out; encode<codec::base64>(input.begin(), input.end(), std::inserter(out, out.end())); Classic iterator pair with output value return: using namespace boost::crypto3; std::string input = “00112233445566778899aabbccddeeff”; std::string key = “000102030405060708090a0b0c0d0e0f”; std::string out = encrypt<block::rijndael<128, 128>>(input.begin(), input.end(), key.begin(), key.end()); C++20 Ranges (or Boost.Range) usage: using namespace boost::crypto3; std::vector<std::uint8_t> input = {0x27, 0x0f, 0xb1, 0x89, 0x82, 0x80, 0x0d, 0xa6, 0x40}; std::string out = encode<codec::base32>(input); // or encode<codec::base32>(input, out); What about generic encoding/encryption/signing/hashing/%name_your_action% interface for classes? You can definitely do that in case you define implicit conversion of your class to integral type. I’m thinking of introducing Inspectable concept which will require for the developer to have a function in his class which would return every class member which developer thinks needs to be encrypted/hashes/encoded/etc, converted to the particular algorithm’s format. (e.g. satisfying Integral concept for ciphers/codes/hashes). 2. It should contain only time-proven schemes implemented with well-known and proven techniques. Yes. Boost.Crypto3 (https://github.com/NilFoundation/boost-crypto3 <https://github.com/NilFoundation/boost-crypto3>) is derived from =nil; Crypto3 C++ Cryptography Suite (https://github.com/NilFoundation/crypto3 <https://github.com/NilFoundation/crypto3>), which my non-commercial foundation designed and developed for it’s projects. A little bit of information on how exactly it was derived from the original suite you can get from the readme.md in both or repositories. As far as you can see in the original repository, the suite contains some very novel schemes (e.g. Verifiable Delay Functions: https://github.com/nilfoundaton/crypto3-vdf.git <https://github.com/nilfoundaton/crypto3-vdf.git>) and it is going to contain more (threshold cryptography, zero-knowlege cryptography, cryptographic accumulators etc.). But Boost-ified version has to be a solid one. No novel schemes. Only proven ones with proven implementation techniques. Which ones? This is what Boost community is up to decide. So, the question for the community. What schemes should be included? For now we only present small set of block ciphers, hashes and codecs listed in here https://crypto3.nil.foundation/projects/crypto3/db/d97/group__codec.html <https://crypto3.nil.foundation/projects/crypto3/db/d97/group__codec.html>, https://crypto3.nil.foundation/projects/crypto3/db/d97/group__hashes.html <https://crypto3.nil.foundation/projects/crypto3/db/d97/group__hashes.html>, https://crypto3.nil.foundation/projects/crypto3/d2/dba/group__block.html <https://crypto3.nil.foundation/projects/crypto3/d2/dba/group__block.html>. (Or you can find the list in the include directories of particular repositories: https://github.com/nilfoundation/crypto3-block.git <https://github.com/nilfoundation/crypto3-block.git>, https://github.com/nilfoundation/crypto3-codec <https://github.com/nilfoundation/crypto3-codec.git>.git <https://github.com/nilfoundation/crypto3-codec.git>, https://github.com/nilfoundation/crypto3-hash <https://github.com/nilfoundation/crypto3-hash.git>.git <https://github.com/nilfoundation/crypto3-hash.git>). Peter (Dimov), I’m aware of your abandoned hash library implementation (https://github.com/pdimov/hash2 <https://github.com/pdimov/hash2>) based on Vinnie’s paper. I would really like to see it’s contents and concepts (probably) adapted to hash module of Boost.Crypto3. Would be a great synergy. 3. It should have a concept-based architecture. Concept-based architecture supposes every reused type trait to be formalized, and standardized. This is what exactly has been done. In particular, concepts for contents of Boost.Crypto3 are listed in sufficient documentation chapters: https://crypto3.nil.foundation/projects/crypto3/df/d5d/block_ciphers_concep… <https://crypto3.nil.foundation/projects/crypto3/df/d5d/block_ciphers_concep…>, https://crypto3.nil.foundation/projects/crypto3/d7/da8/codec_concepts.html <https://crypto3.nil.foundation/projects/crypto3/d7/da8/codec_concepts.html>, https://crypto3.nil.foundation/projects/crypto3/d1/d1f/hashes_concepts.html <https://crypto3.nil.foundation/projects/crypto3/d1/d1f/hashes_concepts.html>. Long story short. We standardize stateless policies containing all the params as constexpr ones used for functions (interface-functions like encrypt/decrypt) and actual algorithms. We also figured out that most of cryptographic algorithms are about accumulation and only then processing. So we used Accumulator concept from Boost.Accumulator to accumulate (surprise surprise) data in a format, which is required by particular algorithm to be passed into. You can get the idea of how I’ve merged the Accumulator concept in from architecture and implementation notes (for Block Ciphers) in here: https://crypto3.nil.foundation/projects/crypto3/db/dee/block_ciphers_impl.h… <https://crypto3.nil.foundation/projects/crypto3/db/dee/block_ciphers_impl.h…>. Pretty drafty as well, but better than nothing. Yes, Hash concept should be made consent with currently existing Hash concept. This is in progress. Vinnie (Falco), I remember, you’ve mentioned some more generic concepts like OneWayFunction (in here: https://lists.boost.org/Archives/boost//2020/02/248228.php <https://lists.boost.org/Archives/boost//2020/02/248228.php>). I agree, we should do that, but we simply had no time to put this fine idea to our architecture. We would really appreciate any help with that. Cristopher (Kormanyos), I’m aware of your CRC catalog library (https://github.com/ckormanyos/crc-catalog <https://github.com/ckormanyos/crc-catalog>) I like it because of how in particular it is implemented. I would really like to merge it into our CRC implementation in Crypto3.Hash. By the way. I wonder if we could sometime remove separate Boost.CRC library and use CRC from more generic Boost.Crypto3? Hypothetically. Just asking. Yeah, we have checksums being implemented in Crypto3.Hash (and in Boost.Crypto3 as well), but they were simply not verified yet, so they are hanging in a PR (https://github.com/NilFoundation/crypto3-hash/pull/68 <https://github.com/NilFoundation/crypto3-hash/pull/68>). 4. Other modules? Roadmap? Yes, there will be other modules being proposed in some time. Public Key cryptography in Boost? Yes. Based on enhanced for cryptographic purposes Boost.Multiprecision (https://github.com/NilFoundation/crypto3-multiprecision <https://github.com/NilFoundation/crypto3-multiprecision>). SSL based on Boost.Crypto3? Yes. Why not. We are already in progress. Major updates are going to be performed with merging in the Boost-ified version of bunch of cross-dependent modules of the original suite. For example, public key cryptography would require enhancing Boost.Multiprecision (which is already in progress) and introducing finite fields library (https://github.com/NilFoundation/crypto3-algebra <https://github.com/NilFoundation/crypto3-algebra>). By the way. The original =nil; Crypto3 suite is highly modular. Every cryptography field is emplaced in separate submodule. There are more than 30 of them now in total (counting closed ones)(yeah, it is huge). Would you like to keep this modularity (like introducing Boost.BlockCiphers, Boost.Hashes and Boost.Codec instead of Boost.Crypto3)? Or do we need to keep everything in header-only Boost.Crypto3? I would vote for the second option. John (Maddock), do you understand now why I was attempting to get accepted those strange PRs in Boost.Multiprecision for the last year? Degski (Sorry, I’m not aware of your name), I remember you’ve mentioned some kind of compression library of yours. I would like to see the compression in the suite. Do you think I could fork it and fit into our architecture? 5. Why do we think we can implement properly design and cryptography? Ask me :). 6. Conclusion. In some time, this would definitely result in C++ Standardization Committee proposal. It is a long way, but I hope to bring it there. What we would like to hear from Boost community is some guidance about what in particular from this field would be useful for Boost, proposals and suggestions about architecture and concepts. Let us standardize some cryptography together (starting with this). Sincerely yours, Mikhail Komarov nemo(a)nil.foundation

6 8

Re: [boost] ASIO SSL Memory Leaks
by abrarov＠mail.ru 01 Sep '20

01 Sep '20

Hi John, Welcome to this mailing list! Doesn't Asio call SSL_shutdown here: https://github.com/chriskohlhoff/asio/blob/master/asio/include/asio/ssl/deta il/impl/engine.ipp#L325 ? Regarding memory leaks - cold you please provide a minimal example reproducing memory leak including the way you detect memory leak? Could chriskohlhoff/asio GitHub issue #368 (https://github.com/chriskohlhoff/asio/issues/368) be your case? Regards, Marat Abrarov.

1 0

ASIO SSL Memory Leaks
by John A 01 Sep '20

01 Sep '20

Hello y'all, This is my first post to this listserv though I've been on others for ~30 years. I have a Boost question, ob duh. Using the SSL_shutdown() outlined here: https://www.openssl.org/docs/man1.0.2/man3/SSL_shutdown.html from within Boost, there isn't an object to associate with this shutdown call. Secondly, we have memory leaks (~10k bytes) using the SSL library via Boost socket calls using the ASIO SSL streams. What am I missing? What advisement can be offered? Ideally, we want to call SSL_shutdown() from within Boost, though I am missing something. Best regards, John Aronetics.com We Speak ITR +1-216/307-5760

3 3