Re: [boost] A question for folks here

11 Jul 2024

      Hi,

I have been involved with C++ since its inception from C and BCPL (anyone
else remember that one?), but only involved with Boost when Vinnie offered
me a role as a writer with CppAlliance. I used to write computer wargames
in C++, and Battalia - a simulation of Napoleonic and American Civil War
battles - is still around on some free sites (
https://www.homeoftheunderdogs.net/game.php?id=3635). Wish I had known
about Boost at that time. Looking to the future I recently heard the
electric car company Rivian used Unreal Engine (a C++ based game engine) to
write its in-car infotainment software. I own a Rivian, am a big believer
in EVs and cleaner sources of energy, and there is a role for C++, the
Boost Library, the Standard Library, and all that goes with it - in a more
efficient future - if we do indeed want such a role.

Best Regards

- Peter Turcan

On Wed, Jul 10, 2024 at 5:22 PM Kenneth Reitz via Boost <
boost@lists.boost.org> wrote:
...
Hi all,
My name is Kenneth Reitz, and I got involved with Boost when Vinnie Falco
asked me to join the C++ Alliance to work on the new boost website.
You might recognize my name from the Python community, where I’m known for
making Requests, Pipenv, and other tools.
It’s nice to meet everyone!
—
Kenneth Reitz
...
1. A question for folks here (David Sankel)
  2. Re: A secure string library? (Andrey Semashev)
  3. Re: A secure string library? (Andrey Semashev)
  4. Re: A question for folks here (Richard Hodges)
  5. Re: A secure string library? (Daniela Engert)
  6. Re: A secure string library? (Andrey Semashev)
  7. Re: A secure string library? (Christian Mazakas)
  8. Re: A question for folks here (Matt Borland)
  9. Re: A question for folks here (Christopher Kormanyos)
 10. Re: A question for folks here (Louis Tatta)
----------------------------------------------------------------------
Message: 1
Date: Wed, 10 Jul 2024 10:47:50 -0400
From: David Sankel 
To: Boost 
Subject: [boost] A question for folks here
Message-ID:

Content-Type: text/plain; charset="UTF-8"
For those who got involved in Boost within the last couple years, how did
you hear about boost? What attracted you to it?
For those who have been around for a while, what keeps you here? Why do
you
stay engaged?
------------------------------
Message: 2
Date: Wed, 10 Jul 2024 18:26:20 +0300
From: Andrey Semashev 
To: boost@lists.boost.org
Subject: Re: [boost] A secure string library?
Message-ID: <6265cabd-c0b9-405e-98f4-8a47683e5472@gmail.com>
Content-Type: text/plain; charset=UTF-8
On 7/10/24 14:59, Peter Dimov via Boost wrote:
...
Ruben Perez wrote:
...
There is OPENSSL_Cleanse, which does a similar job. I found it
surprisingly
simple - I wonder if this is just enough or there are any corner cases
where this
doesn't work:
https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85...
That's an interesting approach. I can't offhand think of a reason why
it wouldn't
work.
The compiler might convert OPENSSL_cleanse to something like this:
void OPENSSL_cleanse(void *ptr, size_t len)
 {
   memset_t func = memset_func;
   if (func != memset)
     func(ptr, 0, len);
   else
     memset(ptr, 0, len);
 }
The purpose here is that, if memset_func is actually memset most of the
time, it can further optimize the call to memset, including to
completely remove it, depending on the call context. The
(well-predictable) branch is typically cheaper than an indirect call. I
think I've seen compilers do something along those lines as a result of
call devirtualization, especially with IPO and LTCG.
I'm not saying that's what actually happens in OpenSSL, just that
something like this is possible. I think, a dummy asm statement is more
reliable and more efficient.
void secure_cleanse(void *ptr, size_t len)
 {
   memset(ptr, 0, len); // a normal memset, optimizations are welcome
   __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
 }
You can even make that function inline and it'll work, and in an optimal
way, too.
------------------------------
Message: 3
Date: Wed, 10 Jul 2024 18:37:00 +0300
From: Andrey Semashev 
To: boost@lists.boost.org
Subject: Re: [boost] A secure string library?
Message-ID: <57326885-13ab-4a73-ac5a-5aa6357e799c@gmail.com>
Content-Type: text/plain; charset=UTF-8
On 7/10/24 18:26, Andrey Semashev wrote:
...
On 7/10/24 14:59, Peter Dimov via Boost wrote:
...
Ruben Perez wrote:
...
There is OPENSSL_Cleanse, which does a similar job. I found it
surprisingly
simple - I wonder if this is just enough or there are any corner
cases where this
doesn't work:
https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85...
That's an interesting approach. I can't offhand think of a reason why
it wouldn't
work.
The compiler might convert OPENSSL_cleanse to something like this:
void OPENSSL_cleanse(void *ptr, size_t len)
 {
   memset_t func = memset_func;
   if (func != memset)
     func(ptr, 0, len);
   else
     memset(ptr, 0, len);
 }
The purpose here is that, if memset_func is actually memset most of the
time, it can further optimize the call to memset, including to
completely remove it, depending on the call context. The
(well-predictable) branch is typically cheaper than an indirect call. I
think I've seen compilers do something along those lines as a result of
call devirtualization, especially with IPO and LTCG.
I'm not saying that's what actually happens in OpenSSL, just that
something like this is possible. I think, a dummy asm statement is more
reliable and more efficient.
void secure_cleanse(void *ptr, size_t len)
 {
   memset(ptr, 0, len); // a normal memset, optimizations are welcome
   __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
 }
You can even make that function inline and it'll work, and in an optimal
way, too.
And for compilers that don't support __asm__, I think you could replace
it with:
std::atomic_signal_fence(std::memory_order::acq_rel);
------------------------------
Message: 4
Date: Wed, 10 Jul 2024 18:01:03 +0200
From: Richard Hodges 
To: boost@lists.boost.org
Subject: Re: [boost] A question for folks here
Message-ID:
      <
CALvx3hYSO6svxz_aEw_LoaraweTguqW+jJScTyfymZAveOrXZg@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
On Wed, 10 Jul 2024 at 16:48, David Sankel via Boost <
boost@lists.boost.org>
wrote:
...
For those who got involved in Boost within the last couple years, how
did
you hear about boost? What attracted you to it?
I've been using boost as an essential force multiplier of C++ since
version
1.35 when Asio appeared, courtesy of Chris Kohlhoff.
More recently in 2017, I needed an asynchronous websocket library for C++
and lo-and-behold, boost had one, courtesy of Vinnie Falco.
...
For those who have been around for a while, what keeps you here? Why do
you
...
stay engaged?
After finishing a contract in which I used Boost.Beast et.al. in
production
code, I took up an offer by the C++ Alliance to become a maintainer of
it.
A bit of a "Victor Kiam" moment, if you will.
While working for the C++ Alliance I was also involved in the design and
first cut of Boost.Json and had some input into Boost.Url, another two
very
keenly needed libraries sponsored by the C++ Alliance.
I am no longer working for the C++ Alliance, but maintain a warm
relationship with the organisation.
I am currently using Boost.Cobalt in a financial trading software suite.
I stay engaged because:
- I don't want to have to switch languages - C++ is perfectly fine for
me.
- Boost polyfills all the essential functionality that the children
running
WG21 continue to refuse to provide.
- It's convenient to have one library that provides (almost) every tool I
need.
My very great thanks to the original authors and contributors, and more
recently to the C++ Alliance, which as far as I can see is the only major
sponsor of Boost.
Without this organisation's involvement, I believe the libraries would
have
been allowed to fall into disrepair a long time ago.
I don't think I have met more than one team in the financial industry
which
uses C++ but which does not use Boost.
On a separate note, I was deeply disappointed by your recent public
outbursts, David.
Sincerely,
Richard Hodges
A C++phile
ex maintainer of Boost.Beast
contributor to Boost.Json, Boost.Url
Advisor to Boost.MySQL, Boost.Cobalt
...
_______________________________________________
Unsubscribe & other changes:
http://lists.boost.org/mailman/listinfo.cgi/boost
------------------------------
Message: 5
Date: Wed, 10 Jul 2024 18:13:41 +0200
From: Daniela Engert 
To: boost@lists.boost.org
Subject: Re: [boost] A secure string library?
Message-ID: <98140e71-ef1e-431a-a089-e24f56e390bd@ngrt.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Am 10.07.2024 um 17:37 schrieb Andrey Semashev via Boost:
...
On 7/10/24 18:26, Andrey Semashev wrote:
...
On 7/10/24 14:59, Peter Dimov via Boost wrote:
...
Ruben Perez wrote:
...
There is OPENSSL_Cleanse, which does a similar job. I found it
surprisingly
simple - I wonder if this is just enough or there are any corner
cases where this
doesn't work:
https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85...
That's an interesting approach. I can't offhand think of a reason why
it wouldn't
work.
The compiler might convert OPENSSL_cleanse to something like this:
void OPENSSL_cleanse(void *ptr, size_t len)
  {
    memset_t func = memset_func;
    if (func != memset)
      func(ptr, 0, len);
    else
      memset(ptr, 0, len);
  }
The purpose here is that, if memset_func is actually memset most of the
time, it can further optimize the call to memset, including to
completely remove it, depending on the call context. The
(well-predictable) branch is typically cheaper than an indirect call. I
think I've seen compilers do something along those lines as a result of
call devirtualization, especially with IPO and LTCG.
I'm not saying that's what actually happens in OpenSSL, just that
something like this is possible. I think, a dummy asm statement is more
reliable and more efficient.
void secure_cleanse(void *ptr, size_t len)
  {
    memset(ptr, 0, len); // a normal memset, optimizations are welcome
    __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
  }
You can even make that function inline and it'll work, and in an
optimal
way, too.
And for compilers that don't support __asm__, I think you could replace
it with:
std::atomic_signal_fence(std::memory_order::acq_rel);
I think Peter is actually right in his assessment:
While 'memset_func' is TU-local, it is also 'volatile'. This implies
every look at it might reveal a different content. Therefore the
necessity for non-constness and dynamic initialization. We humans see
that the variable can't change its value other than in the
initialization. The compiler can't reason that because said properties
are the only ones known during the compilation of 'OPENSSL_cleanse()'.
It can't perform unbounded look-ahead until the end of the TU like we do.
Dani
--
PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
------------------------------
Message: 6
Date: Wed, 10 Jul 2024 19:24:48 +0300
From: Andrey Semashev 
To: boost@lists.boost.org
Subject: Re: [boost] A secure string library?
Message-ID: <905dd5f7-5d29-48ab-87aa-6aa0a23e60c6@gmail.com>
Content-Type: text/plain; charset=UTF-8
...
Am 10.07.2024 um 17:37 schrieb Andrey Semashev via Boost:
...
On 7/10/24 18:26, Andrey Semashev wrote:
...
On 7/10/24 14:59, Peter Dimov via Boost wrote:
...
Ruben Perez wrote:
> There is OPENSSL_Cleanse, which does a similar job. I found it
> surprisingly
> simple - I wonder if this is just enough or there are any corner
> cases where this
> doesn't work:
>
https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85...
That's an interesting approach. I can't offhand think of a reason
why it wouldn't
work.
The compiler might convert OPENSSL_cleanse to something like this:
?? void OPENSSL_cleanse(void *ptr, size_t len)
?? {
???? memset_t func = memset_func;
???? if (func != memset)
?????? func(ptr, 0, len);
???? else
?????? memset(ptr, 0, len);
?? }
The purpose here is that, if memset_func is actually memset most of
On 7/10/24 19:13, Daniela Engert via Boost wrote:
the
...
...
...
time, it can further optimize the call to memset, including to
completely remove it, depending on the call context. The
(well-predictable) branch is typically cheaper than an indirect call.
I
think I've seen compilers do something along those lines as a result
of
call devirtualization, especially with IPO and LTCG.
I'm not saying that's what actually happens in OpenSSL, just that
something like this is possible. I think, a dummy asm statement is
more
reliable and more efficient.
?? void secure_cleanse(void *ptr, size_t len)
?? {
???? memset(ptr, 0, len); // a normal memset, optimizations are
welcome
???? __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
?? }
You can even make that function inline and it'll work, and in an
optimal
way, too.
And for compilers that don't support __asm__, I think you could replace
it with:
?? std::atomic_signal_fence(std::memory_order::acq_rel);
I think Peter is actually right in his assessment:
While 'memset_func' is TU-local, it is also 'volatile'. This implies
every look at it might reveal a different content. Therefore the
necessity for non-constness and dynamic initialization. We humans see
that the variable can't change its value other than in the
initialization. The compiler can't reason that because said properties
are the only ones known during the compilation of 'OPENSSL_cleanse()'.
It can't perform unbounded look-ahead until the end of the TU like we
do.
With volatile, the compiler is not allowed to optimize away or reorder
loads and stores of the variable. There's no restriction on what the
compiler is allowed to do with the loaded value.
------------------------------
Message: 7
Date: Wed, 10 Jul 2024 10:10:43 -0700
From: Christian Mazakas 
To: boost@lists.boost.org
Subject: Re: [boost] A secure string library?
Message-ID:

Content-Type: text/plain; charset="UTF-8"
...
Andrey Semashev via Boost:
With volatile, the compiler is not allowed to optimize away or reorder
loads and stores of the variable. There's no restriction on what the
compiler is allowed to do with the loaded value.
It could insert a branch against `memset` but that seems really only
like a
theoretical
optimization.
This is an example where the volatile preserves the secure_erase() call:
https://godbolt.org/z/854jfarb9
Kind of neat trying to get the compiler to elide this stuff. You were
right, Andrey, you really need
LTO and LTCG and the like to get this kind of behavior. Super neat stuff.
Idk, seems like whatever OpenSSL is doing is Sound Enough in practice.
I'd be curious to know if it's actually sound for a compiler to make that
kind of optimization or at least
what are the limits on how far a compiler can actually go to ensure
soundness. If you know those limits,
you can probably make this pattern work reliably.
- Christian
------------------------------
Message: 8
Date: Wed, 10 Jul 2024 17:57:57 +0000
From: Matt Borland 
To: boost@lists.boost.org
Subject: Re: [boost] A question for folks here
Message-ID:

...
Content-Type: text/plain; charset="utf-8"
...
For those who got involved in Boost within the last couple years, how
...
...
you hear about boost?
I heard about it from a university professor as a freshman undergraduate
student in 2011. Since C++11 was just coming about, and the speed of
updating toolchains in academia is slow so we were stuck in the C++03 days
on old RHEL systems.
...
What attracted you to it?
I was studying aeronautical engineering so having pre-made libraries
did
like Boost.Math, Boost.Multiprecision, and ODEint were useful to me. I
could focus on the problem at hand rather than having to roll my own
special functions, solvers, big numbers, constants, etc.
...
...
For those who have been around for a while, what keeps you here? Why do
you stay engaged?
...
I started to get involved with development during the COVID lockdowns
because like many people I found myself with quite a bit of extra free
time. What hooked me was learning from and working with extremely competent
programmers and mathematicians like John Maddock, Chris Kormanyos, and Nick
Thompson. I was hired by the C++ Alliance at the beginning of 2023 and they
have sponsored my development since. This allows me to work on new (
https://github.com/cppalliance/decimal), and existing libraries
(charconv, math, multiprecision, random, etc) in a way that I was never
able to on a volunteer basis with a full time job. The community is large,
knowledgeable, and helpful so I continue to enjoy working on Boost.
...
I think it would be beneficial to the developer community for the Boost
Foundation board members to also answer these questions. Frankly some even
require introductions with their answers. There are members of the board
like Peter, Glen, and Zach that are Titans of Boost, and those like Kristen
who I had not heard of until her post about wanting to cut ties with the
C++ Alliance. Having unknown members of the Boost Foundation runs counter
to their mission statement whereby the board intends to achieve its mission
by "fostering community engagement, [and] nurturing leaders..."[1].
...
Matt
[1] https://sites.google.com/boost.org/boost-foundation/home