Hi Michael, [I've rearranged your email slightly] On 3/21/2016 9:15 PM, Michael Witten wrote:

I'm new to this community, so forgive my ignorance if I've missed an existing solution.

This topic was raised about a week ago.

In short, I request that the maintainers start publishing cryptographically signed, strong hashes of:

* downloadable files.

This specific question, even.

* git objects (tags, and even commits).

It seems a low-priority idea to me. Say, you have a github commit by me, which means that somebody in possession of my RSA private key has pushed it. What does PGP signature adds - the fact that somebody also has my PGP private key? If I can't protect one, I probably can't protect the other, so the only thing we'll be protecting is GitHub security issues?

A cryptographic signature should probably be a personal signature of a relevant maintainer (rather than some generic project-level signature for which nobody has a sufficiently strong incentive to maintain the trustworthiness).

Perhaps, each repository should include a collection of relevant public keys, so as to compound trustworthiness and ease dissemination.

For better or for worse, this won't ever happen due to number of repositories and maintainers.

With each release, there should be included conspicuously (and widely) published means by which to check the authenticity of the relevant content.

[...]

Fortunately, all parties can be satisfied simultaneously and cheaply: Provide the list of hashes as a cryptographically signed message.

If you propose that a release announcement includes a list of checksums along with a PGP signature, then it's doable, as discussed previously. Though, I don't know which release managers actually have PGP keys and how widely they are known. Say, my PGP key is signed by about 3 other people.

In any case, something must be done; this project sits at the core of much critical software, and its integrity should be ensured with greater zeal.

That's true, but it's not clear whether tampered source archives is the biggest risk. If you look at other open-source projects, all the huge security problems were either genuine bugs, or government-mandated "export crypto", not so much of directly evil code. If one wanted to use Boost as attack vector, he'd probably try to introduce buffer overflow inside otherwise reasonable patch, for which the above solutions would not help. -- Vladimir Prus http://vladimirprus.com

Please also see this thread about release integrity and authenticity:

http://lists.boost.org/Archives/boost/2016/03/228428.php

On 03/21/2016 07:15 PM, Michael Witten wrote:

In short, I request that the maintainers start publishing cryptographically signed, strong hashes of:

* downloadable files. * git objects (tags, and even commits).

A cryptographic signature should probably be a personal signature of a relevant maintainer (rather than some generic project-level signature for which nobody has a sufficiently strong incentive to maintain the trustworthiness).

Perhaps, each repository should include a collection of relevant public keys, so as to compound trustworthiness and ease dissemination.

------------------------------------------------------------------------

I'm new to this community, so forgive my ignorance if I've missed an existing solution.

With each release, there should be included conspicuously (and widely) published means by which to check the authenticity of the relevant content.

As far as I can tell, this is not yet the case.

At the very least, within an email that announces a release, there should be a list of suitably safe (e.g., SHA-256) hashes for the downloadable files in question, including the hash of the relevant git commit object[s].

Now, SourceForge does indeed list SHA-1 and MD5 hashes of each file it offers for download, but they are not conspicuous; the user must know to click on an `i' symbol that sits next to each link (the `i' presumably stands for `information', but I wouldn't be surprised if it actually stood for `ignore me'). To make matters worse, that very icon and its associated functionality is only available when JavaScript is enabled, which is absurd.

Furthermore, it's important that this list of hashes appear in as many independent places as possible, so that it becomes increasingly difficult to alter the association; for instance, if Gmane picks up an announcement email that includes such hashes, then an attacker will also have to compromise Gmane's servers in order to forge a new record of the intended payload.

That is, widely published integrity information strongly suggests a [reasonable] means by which to calculate authenticity; certainly, the converse is true: Authenticity implies integrity.

Of course, it's irritating (and unlikely) for a user to take the time to check multiple publishers for such integrity information every time a new release is made. Thus, it makes more sense to abstract away this process into a one-time dissemination of integrity information with which a user may determine the authenticity of other, related data that is provided by any publisher.

Such an abstraction is provided by cryptographic signatures.

By widely publishing a public key, a maintainer provides a [reasonable] means by which a user may calculate the authenticity (and thus also the integrity) of any other data that is signed by the associated private key.

Now, that calculation is only as trustworthy as the private key itself, so it's important that there be a strong incentive to keep that private key trustworthy; this possibly implies that the private key in question should be intimately associated with the individual who uses it; this suggests that, as a matter of explicit policy, there should not be some generic project-level private key shared by multiple individuals across space or time.

The easiest solution is to require that some *individual* maintainer of data be responsible for signing that data with his own personal signature.

Of course, not every user has the machinery or desire to work with cryptographic signatures; some people are satisifed with merely checking that the hash of data matches the result provided by a trusted publisher.

Fortunately, all parties can be satisfied simultaneously and cheaply: Provide the list of hashes as a cryptographically signed message.

Furthermore, the source-control management tool `git' (which this project already uses) provides support for integrating cryptographic signatures; not only can a tag object be cryptographically signed, but every commit object can also be cryptographically signed. It's probably enough just to sign tags (those of releases, at the very least), but perhaps it would be worthwhile to sign a commit object that sits some sufficient number of commits ahead of the object last signed, so as to give users a kind of checkpoint of authenticity.

It might be worthwhile to include in every repository the public key of each relevant maintainer (say, one file for each maintainer); this kind of publication would allow for compounding a key's trustworthiness with each new commit, and it would make dissemination of that information both easier and more specialized to its purpose.

In any case, something must be done; this project sits at the core of much critical software, and its integrity should be ensured with greater zeal.

Sincerely, Michael Witten

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Before I begin, I would like to note that the engineering of source code is separate from the engineering of the release of that source code. If you are bored by the issues of the release process, then that's fine; however, it's important to find people who *are* interested in honing that process ever more. What follows is meant to be both informative and speculative; I'm just sharing ideas that might prove useful to discussion. Now, consider the following: $ git clone https://github.com/boostorg/boost.git repo $ cd repo $ git describe master fatal: No annotated tags can describe '11da8f8fafc4ba6b3e577e63abb7dc45cb902e2f'. However, there were unannotated tags: try --tags. Yikes! The super-project doesn't seem to use any annotated tags; indeed: $ git for-each-ref --format='%(tag)' | grep -v ^$ || echo none none Why does this matter? Recall the following from the `git help tag' page: Tag objects (created with -a, -s, or -u) are called "annotated" tags; they contain a creation date, the tagger name and e-mail, a tagging message, and an optional GnuPG signature. Whereas a "lightweight" tag is simply a name for an object (usually a commit object). Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels. For this reason, some git commands for naming objects (like `git describe') will ignore lightweight tags by default. Pay particular attention to the second paragraph: Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels. By using just lightweight tags, this project is neglecting to publish valuable information about a release: * When was the release officially tagged? * Who tagged the release? * Is there any important information particular to this tagging? The tag body could include the `Message-ID' of the announcement email. * Is this tagging *authentic*? Is this really the tag developers intended? Getting the tags correct is foundational to further improvements in the security of release management; it will certainly make it much easier for the release team to integrate cryptographic signatures into the process, because git supports them directly. Given that this project uses centralized repositories to which multiple people have write access, it would probably be prudent to designate officially one person to be the lead release manager, who by widespread convention will be the only one allowed to tag a release (including a release candidate); naturally, each submodule could have its own lead release manager. To facilitate this designation, the lead release manager should publish conspicuously (and widely) the public key associated with his or her personal identity in this role (an impersonal, project-level private key is unlikely to enjoy the inherently intense security incentives that surround a personal private key); this publishing includes: * Emailing the Boost developer mailing list with the plaintext version of the public key. * Disseminating the public key via specialized services (such as PGP keyservers). * Providing on the Boost website links to independently archived versions of the public key (such as the aforementioned email as hosted on Gmane, and a keyserver's file, and the webpage's own file). * Making sure community members with whom he or she officially meets (say, at an industry conference), have the opportunity to review, to accept, and to publicly validate the public key (i.e., to create a web of trust). * etc. I know, it sounds terrible; but, it's really not that bad. Then, every public tag should probably be an annotated tag; ideally, the lead release manager would cryptographically sign every such tag with his or her personal private key. Separately, this same key could (and probably should) also be used to produce a single plaintext cryptographic signature with an embedded plaintext message that includes a listing of SHA-256, SHA-1, and MD5 hashes for the relevant release archives; this signature (and embedded message) could be the entire body of the release's announcement email, or it could be included in the body of the announcement email (and this email could be explicitly referenced in the body of the annotated release tag, etc.). Certainly, that signature and its associated message should be published conspicuously alongside the release's downloadable files. For now, I would suggest concentrating on improving the procedures of the super-project, and then eventually using the resulting insight to help the [release] managers of the various subprojects introduce similar improvements. After all, it would appear that 116 submodules of the super-project also do not employ any annotated tags whatsoever; to get a list of those submodules, remove the final `| wc -l' from the following: $ git submodule update --init $ git submodule foreach --quiet ' git for-each-ref --format="%(tag)" | grep -q -v ^$ || echo "$name" ' | wc -l 116 Of the ones that *do* use annotated tags, the majority probably use them inadequately, irregularly, or possibly without realizing it; just 3 submodules publish tags that are exclusively annotated: compute dll libs/hana Those were found in the following list of submodules, each of which uses at least one annotated tag: $ git submodule foreach --quiet ' total=$(git tag | wc -l) git for-each-ref --format="%(tag)" | awk -v n="$name" -v t="$total" " /./ {tags[i++] = \$0} END { if (i==0) exit; printf(\"%s(%i/%i):\n\",n,i,t); for (x in tags) print(\" \" tags[x]); print(\"\"); } " ' asio(4/48): asio-1.10.1 asio-1.10.2 asio-1.10.4 asio-1.10.5 compute(5/5): v0.1 v0.2 v0.3 v0.4 v0.5 dll(5/5): boost-review v0.1 v0.1_docs v0.2 v0.2_docs filesystem(1/61): filesystem-2014-07-23 geometry(2/21): geometry-1.56.0 geometry-1.56.0-beta1 libs/hana(7/7): v0.1 v0.2.0 v0.3.0 v0.4.0 v0.5.0 v0.6.0 v0.7.0 icl(1/23): icl-2015-11-22 interprocess(1/43): interprocess-1.56.00.b0 intrusive(1/43): intrusive-1.56.00.b0 odeint(4/15): v2.1 v2.2 v2.3 v2.4 optional(1/61): optional-2016-02-22 system(2/44): system-2014-06-02 system-2014-07-17 type_index(4/11): v1.0 v1.0_docs v3.0 v3.0_html build(2/73): converted-develop converted-master Sincerely, Michael Witten