On 4/6/19 7:46 PM, John Maddock via Boost wrote:

On 05/04/2019 00:52, Riff J via Boost wrote:

...

Hi everyone,

I am a developer from Microsoft and currently using boost in our project. We recently get noticed by our security team, that the boost library we use, are not compiled with spectre mitigation (/Qspectre) enabled. Since boost is super powerful, it might not be a good idea to write our own or maintain our own version, so we are reaching out for help. Could anyone please help see if we could enable /Qspectre option in the official build of boost?

We tend to use default compiler flags for official builds, but it's relatively easy for you to build Boost with whatever other flags you may want:

cd boost-root-dir

bootstrap

b2 --build-type=complete cxxflags=-Qspectre

Will build the libraries with the latest installed msvc version and the /Qspectre flag enabled.  And of course for header only libraries you don't need to do anything at all anyway.

One thing we could look at for future releases would be to provide differently-named binaries for /Qspectre.  Anyone else have thoughts on that?

I don't think replacing built binaries with Spectre-protected ones is what we want, but adding them as an option might be possible. However, doubling the number of configurations of the built libraries would be too taxing. E.g. one package of Boost 1.69 (which corresponds to one version of MSVC at one bitness) is already 200-250 MiB, and that would have to double for the newer MSVC versions that support the flag. If we're going to package these binaries, maybe we should package them separately. As for whether we should actually provide such binaries, I'm not sure. I've never used the pre-built binaries. I would guess anyone working on security-sensitive software would want to build everything from source code (ideally, after reviewing the said source code), so I'm not sure how useful those binaries would be.

On Sat, Apr 6, 2019, 1:11 PM Robert Ramey via Boost wrote:

On 4/6/19 9:46 AM, John Maddock via Boost wrote:

...

...

One thing we could look at for future releases would be to provide differently-named binaries for /Qspectre. Anyone else have thoughts on that?

Hmmm - I question I'm too lazy to investigate so I'll just ask you. Does boost build not provide the option of specifying that the invocation specify the name of the binaries or perhaps some root? Would this not get boost off the hook for addressing special requirements like this?

Yes, both of those features are available.

Forgot to mention: I don't think any kind of important code should be depending on binaries. For this reason, I don't think that Boost should be distributing binaries at all. To me, it's bad practice. I realize that in some cases it's unavoidable, (libc, etc. zlib, et.all). But still, I should be less preferred option for code which requires any consideration of modern security problems. Robert Ramey

On Sat, Apr 6, 2019 at 4:12 PM Riff J via Boost wrote:

Hi Robert, I get what you're saying, and we don't mind to build our own version. But this issue is not a regular issue, like some minor macros/flag we want to enable to fix some build errors. It is a serious one. And for mitigating it, MSVC team even released a special version of CRT ( https://devblogs.microsoft.com/cppblog/spectre-mitigations-in-msvc/). We do have other issues while fitting boost into our build system, but we never reached out for those things. We only reached out for this one, as we believe this is the better way to fix it and can help the entire community. Please help reconsider it again.

I understand that this is a serious issue. But it's not a mass developer problem. The majority of the software written is not impacted by this problem as it does not allow the manipulation through external end-user means and/or have security sensitive information to deal with. Hence such precompile Boost libraries would see a significant lower use than the current binaries. And the current binaries see a significant lower use than the source code (i.e. developer compiled). And hence I don't see the value of Boost itself providing such precompiled libraries. If Microsoft feels this is truly an important concern that needs to be addressed Microsoft could build Boost in that configuration and provide them for the rest of the world to use. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

I understand that this is a serious issue. But it's not a mass developer problem. The majority of the software written is not impacted by this problem as it does not allow the manipulation through external end-user means and/or have security sensitive information to deal with. Hence such precompile Boost libraries would see a significant lower use than the current binaries. And the current binaries see a significant lower use than the source code (i.e. developer compiled). And hence I don't see the value of Boost itself providing such precompiled libraries. If Microsoft feels this is truly an important concern that needs to be addressed Microsoft could build Boost in that configuration and provide them for the rest of the world to use.

+1 Security conscious end users are going to recompile everything according to their own verification and audit processes in any case. They won't use precompiled binaries from external parties unless utterly unavoidable. So to me, apart from fixing the build errors mentioned when compiling with spectre mitigations enabled (pull requests welcome), this is not an issue the release managers need solve. I might also add that simply flipping on /Qspectre is horribly incomplete, and provides a false sense of security. An actually robust spectre mitigation involves very significant rearchitecture of an application, particularly to eliminate as many shared libraries as possible, and to hand-walk every single code path controllable by outside parties looking for timing-related information leak vectors. This is not something which Boost can, or should, do. Niall

On Sun, 7 Apr 2019 at 11:03, Rainer Deyke via Boost wrote:

I would go even further than that. If Microsoft, as an organization, feels that libraries should be compiled by with spectre mitigation by default, then it's up to the MSVC team to actually make that the default, without requiring extra command line arguments. Asking every user of MSVC to modify their build scripts in order to turn on spectre mitigation doesn't scale very well when there are millions of such users.

Hola, wait a sec here, most software doesn't need this mitigation. This mostly relates to browsers from what I understand https://en.wikipedia.org/w/index.php?title=Spectre_(security_vulnerability)&action=edit§ion=3 or local machines that are compromised [but hey, then there are simpler methods of getting your data]. degski -- *Microsoft, please kill Paint3D*

On Sun, 7 Apr 2019 at 13:57, Rainer Deyke via Boost wrote:

... versus the cost of failing to compile with spectre mitigation where it really matters.

You don't think that the people who compile something relevant to the world and relevant to them, know that this would be a good idea. As a general rule, I tend to favor a safe-by-default approach (with the

option to turn the safeties off if you know what you're doing).

The best way to drive a bicycle is obviously with side-wheels, a helmet on, knee paddings and to never leave your drive-way. Once the [a] lib is compiled with Spectre-Mitigations, there is no way of "turning it off". In reality the problem is highly hypothetical as most (Windows) Boost users seem to prefer to use out-dated compilers [and out-dated Boost for that matter] and will not [be able to] use these spectre-mitigated-libs anyway. degski -- *Microsoft, please kill Paint3D*

On Sun, Apr 7, 2019 at 9:11 AM Rainer Deyke via Boost wrote:

Like I said, I'm mostly neutral on the issue of whether spectre mitigation should be turned on by default or not. I just find it strange that someone from Microsoft should complain about Boost being compiled with the compiler options that someone else at Microsoft decided to make the default options for that compiler. From an outsider perspective, it looks like a case of the right hand not knowing what the left hand is doing.

First, sorry, I need to apologize for mentioning I am working in Microsoft, as it might give you all a wrong impression that I speak for Microsoft. I was not doing that. I am just a regular developer trying to fix the project I am working on... All the replies in this thread makes me re-evaluate how serve this issue is. I thought this is a really severe one. Although /Qspectre flag is not magic can fix everything, it is a low hanging fruit and good to have. So instead of just fixing my project, I decided to start this thread and see if we can add this flavor and benefit others as well (compiling boost does take time.... although I have already done it for what I considered as a short term solution for my project). But what Rene, Rainer and Niall said below noticed me and I cannot agree more. Although I am not a security expert, I can still imagine security is a long fight. And there will be more issues and more fixes in the future. It is up to the compiler team and user to decide which one to enable, and we cannot satisfy all the cases, e.g. control flow guard is good to have but not enabled by default as well. While some people are willing to enable more checks or even add their own, some people might not even want stack check enabled as their program is so time sensitive and maybe only runs on their local machine. Providing pre-compiled binaries with all default flags while having a way for user to build their own flavor is definitely good enough. And this is what boost has already done. And thank you all again for all the help and great discussions! On Sun, Apr 7, 2019 at 4:03 AM Rainer Deyke via Boost wrote:

On 07.04.19 04:34, Rene Rivera via Boost wrote:

...

If Microsoft feels this is truly an important concern that needs to be addressed Microsoft could build Boost in that configuration and provide them for the rest of the world to use.

I would go even further than that. If Microsoft, as an organization, feels that libraries should be compiled by with spectre mitigation by default, then it's up to the MSVC team to actually make that the default, without requiring extra command line arguments. Asking every user of MSVC to modify their build scripts in order to turn on spectre mitigation doesn't scale very well when there are millions of such users.

On Sun, Apr 7, 2019 at 1:21 AM Niall Douglas via Boost wrote:

Security conscious end users are going to recompile everything according to their own verification and audit processes in any case. They won't use precompiled binaries from external parties unless utterly unavoidable.

So to me, apart from fixing the build errors mentioned when compiling with spectre mitigations enabled (pull requests welcome), this is not an issue the release managers need solve.

On Sun, Apr 7, 2019 at 9:11 AM Rainer Deyke via Boost wrote:

On 07.04.19 16:10, degski via Boost wrote:

...

The best way to drive a bicycle is obviously with side-wheels, a helmet on, knee paddings and to never leave your drive-way. Once the [a] lib is compiled with Spectre-Mitigations, there is no way of "turning it off". In reality the problem is highly hypothetical as most (Windows) Boost users seem to prefer to use out-dated compilers [and out-dated Boost for that matter] and will not [be able to] use these spectre-mitigated-libs anyway.

Yes, I think we're all aware that any heuristic can lead to absurdity when taken to the extreme. We're also all aware that lots of code is shipped with serious security flaws, so clearly a lot of programmers are erring on the side of "not enough security".

Like I said, I'm mostly neutral on the issue of whether spectre mitigation should be turned on by default or not. I just find it strange that someone from Microsoft should complain about Boost being compiled with the compiler options that someone else at Microsoft decided to make the default options for that compiler. From an outsider perspective, it looks like a case of the right hand not knowing what the left hand is doing.

-- Rainer Deyke (rainerd@eldwood.com)

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Sat, Apr 6, 2019 at 1:48 PM Robert Ramey via Boost wrote:

Is the information easily available in the Boost Build documentation to do this? I think it should be. In fact I think every time a question like this comes it, the boost library and tool maintainers should add a section to the the boost build documentation: Example ... or Case Study ... . Since the same questions tend to be posted again and again, eventually the need to make such additions to the manual will will taper off. At that point all these kinds of questions can be answered with a simple "see example ... in the documentation (you lazy moron). So, though my suggestion seems like it would be creating more work, it's actually an investment of effort designed to diminish total amount of work required to support the tool.

These are not options pertaining to B2 itself. And hence have no mention in the B2 documentation. You can see the options by doing: $ cd boost-root $ b2 --help There are numerous install and build directory options listed in that. Use the appropriate one. For the tagging there's a concept of "buildid" for which there are two options present. The appropriate place to document these in addition to the "--help" printout would be in the Getting Started Boost pages. Or perhaps some other "Building Boost" documentation. Or optionally the wiki. And it would seem that such "Case Study" examples would lend themselves well to having on the wiki. Especially since it's really, really, really, easy for everyone to contribute such documentation through direct edit suggestion PRs to the wiki. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

On Sat, Apr 6, 2019 at 11:23 PM Robert Ramey via Boost < boost@lists.boost.org> wrote:

On 4/6/19 7:22 PM, Rene Rivera via Boost wrote:

...

These are not options pertaining to B2 itself. And hence have no mention

in

...

the B2 documentation. You can see the options by doing:

$ cd boost-root $ b2 --help

There are numerous install and build directory options listed in that. Use the appropriate one. For the tagging there's a concept of "buildid" for which there are two options present. The appropriate place to document these in addition to the "--help" printout would be in the Getting Started Boost pagesOr perhaps some other "Building Boost" documentation. Or optionally the wiki. And it would seem that such "Case Study" examples would lend themselves well to having on the wiki. Especially since it's really, really, really, easy for everyone to contribute such documentation through direct edit suggestion PRs to the wiki.

I think this view is very typical of library and tool developers. And I think it's a big problem.

TL;DR: You injected way more meaning into what I said. To the point of mischaracterizing my intentions of simple declarative information. a) Technically you're correct in that all the required information is

available if one is willing to dig deep enough and invest some time. I think the time required is more than you think it is though.

Yes, I gave technical declarative information. I don't understand how you ascertained anything past my technical statement.

b) You frame task is building the software and the job of "spoon feeding" the user as beyond the developers responsibility and perhaps his ability.

I fixed the spelling mistake in that for you ;-) c) I think you're dead wrong on this. Your task, should you choose to

accept it, is to create something that lots of people will use and will spread.

Yes, what I chose to create was "Boost Build" and "Boost Predef".

d) So, if the task isn't accomplished, the developer isn't done. He's failed to complete the required task.

Correct, I'm not done. e) So this means putting the document on par with the code. The code

programs the machine, and the document programs the user. If one doesn't work - nothing works.

Yep, I continually work on documentation improvements. For example spending a year converting Boost Build documentation to a format that was a magnitude easier for myself and others to contribute to. Which has worked splendidly as we've gotten a handful of outside contributions in both general improvement and new features with accompanying documentation. f) Defining the task in this way and approaching it accordingly will

result in a work flow which shuttles bank and forth between code and document resulting in better and simpler versions of both. This is because when problems are discovered, we can fix it in the simplest place to fix it.

Right. Which is why Boost Build and Boost Predef Documentation is partially embedded in the source code and is in a simple text format. As it makes it's more likely to get updated and be accurate. And that has been proven to work rather well as experience with it has shown. I know you guys are tired of me preaching this repeatedly. I'm sorry

about this.

I'm not tired of that. But let me clarify my original statement.. The installation and buildid features of "Boost C++ Libraries" are not part of "Boost Build" (B2). B2 implements a build system that provides supporting mechanisms to those features but does not contain them itself. They are properly features of "Boost C++ Libraries" only. As a concrete delineation of that consider this github search < https://github.com/search?q=org%3Aboostorg+buildid&type=Code> for "buildid" in the entirety of all the Boost libraries. You can observe that the places where that is present are in the super-project and in Boost.Config. Hence _not_ in Boost Build. As such, my original key point, that such documentation belongs where it currently is. And perhaps, and almost certainly, as additions to the "Getting Started" documentation. But also that it could be added in the wiki where it is more amenable public contribution. I did not at any time say that such documentation should not be written. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

-----Original Message----- From: Boost On Behalf Of John Maddock via Boost Sent: 6 April 2019 17:47 To: Riff J via Boost Cc: John Maddock Subject: Re: [boost] Enabling spectre mitigation in boost libraries On 05/04/2019 00:52, Riff J via Boost wrote:

Hi everyone,

I am a developer from Microsoft and currently using boost in our project. We recently get noticed by our security team, that the boost library we use, are not compiled with spectre mitigation (/Qspectre) enabled. Since boost is super powerful, it might not be a good idea to write our own or maintain our own version, so we are reaching out for help. Could anyone please help see if we could enable /Qspectre option in the official build of boost?

We tend to use default compiler flags for official builds, but it's relatively easy for you to build Boost with whatever other flags you may want: cd boost-root-dir bootstrap b2 --build-type=complete cxxflags=-Qspectre Will build the libraries with the latest installed msvc version and the /Qspectre flag enabled. And of course for header only libraries you don't need to do anything at all anyway. One thing we could look at for future releases would be to provide differently-named binaries for /Qspectre. Anyone else have thoughts on that? That works for me for a small subset of 'core' libraries. Before we get too excited about this, perhaps we should see if using the /Qspectre option does anything by checking how many warnings we get from https://devblogs.microsoft.com/cppblog/spectre-diagnostic-in-visual-studio-2... by enabling C5045 ? I've done this building a few core libraries for msvc-14.2 Adding cxxflags=/w35045 And I get a blizzard of warnings, almost exclusively from MS code mainly about buffer sizes in handling of strings. However, date-time and filesystem do produce some warnings ☹ Calculation of buffer and string sizes mainly. Unit test framework also has some warnings - but one would probably not care about some testing code that should never 'hits the streets'. (C strings are the worst idea ever). I wonder if some of these could be 'fixed' by making the lengths static and/or const or constexpr? But anyone using this is going to have to ensure that absolutely everything is compiled with the /Qspectre option. MS say that the performance cost is negligible I think we should handle this by changing the documentation to provides links to this and, for now at least, leave users to handle building their own libraries and using with their own code. If there proves to be popular demand, we could consider a new set of libraries. My two pennyworth. Paul Paul A. Bristow Prizet Farmhouse Kendal, Cumbria LA8 8AB UK

And here is the vcblog about the spectre mitigation, if anyone is interested: https://blogs.msdn.microsoft.com/vcblog/2018/01/15/spectre-mitigations -in-msvc/