Moving away from SourceForge

newer
Boost XML Serialization Stability...

Louis Dionne

22 Jan 2017 22 Jan '17

4:35 p.m.

Hi, I know this has been discussed in the past, but I would like to revive the discussion about moving away from SourceForge as Boosts' release hosting provider. SourceForge has at least the following issues: - There is no way of controlling the format when downloading an archive of the latest release. This can break people that rely on this in their CI scripts. For example, Metabench's[1] CI was broken for a while when the archive changed from .bz2 to .zip. - SourceForge has been in the middle of some controversies[2]. - Their reliability has not been so great. In the past year, I've had quite a few of my Travis jobs fail due to SourceForge being unresponsive. There are most likely other concerns, but these are the ones that really bite me as a user right now. Am I the only one? If there's some push to do the move, the two main candidates I see would be - GitHub's native hosting - Bintray Thoughts? Louis [1]: https://github.com/ldionne/metabench [2]: https://en.wikipedia.org/wiki/SourceForge#Controversies -- View this message in context: http://boost.2283326.n4.nabble.com/Moving-away-from-SourceForge-tp4691184.ht... Sent from the Boost - Dev mailing list archive at Nabble.com.

Show replies by date

Andrey Semashev

22 Jan 22 Jan

5:27 p.m.

On 01/22/17 19:35, Louis Dionne wrote:

...

Hi,

I know this has been discussed in the past, but I would like to revive the discussion about moving away from SourceForge as Boosts' release hosting provider. SourceForge has at least the following issues:

- There is no way of controlling the format when downloading an archive of the latest release. This can break people that rely on this in their CI scripts. For example, Metabench's[1] CI was broken for a while when the archive changed from .bz2 to .zip.

- SourceForge has been in the middle of some controversies[2].

- Their reliability has not been so great. In the past year, I've had quite a few of my Travis jobs fail due to SourceForge being unresponsive.

There are most likely other concerns, but these are the ones that really bite me as a user right now. Am I the only one? If there's some push to do the move, the two main candidates I see would be

- GitHub's native hosting

I don't have a strong opinion re SourceForge, but I have negative experience with GitHub. I had problems with checking out Boost git repos multiplie times in the past, their web interface is regularly down (last time was a few days ago, actually). Thus I have little trust in GitHub reliability. Also, does GitHub allow publishing source releases in the form other than .tar.gz? I imagine, it's not very convenient for Windows users and is not efficient for everyone else.

Brittany Friedman

6:57 p.m.

...

On Sun, Jan 22, 2017 at 11:27 AM, Andrey Semashev < andrey.semashev@gmail.com> wrote:

...

Also, does GitHub allow publishing source releases in the form other than .tar.gz? I imagine, it's not very convenient for Windows users and is not efficient for everyone else.

Yes, you can publish .zip files. Here's an example on my repo: https://github.com/briterator/drpdb/releases

Niall Douglas

5:35 p.m.

...

There are most likely other concerns, but these are the ones that really bite me as a user right now. Am I the only one? If there's some push to do the move, the two main candidates I see would be

- GitHub's native hosting - Bintray

I've found neither sourceforge nor github to be particularly reliable. They're ok for free. I ended up having a cronjob dump a known good tarball into my own server https://dedi4.nedprod.com/static/files/. That's reliable. I did experiment for a bit with Bintray, but I found its design not well suited for Boost type libraries. It could be forced to work, but it's pretty unnatural and clunky. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Tom Kent

7:52 p.m.

On Sun, Jan 22, 2017 at 10:35 AM, Louis Dionne wrote:

...

Hi,

I know this has been discussed in the past, but I would like to revive the discussion about moving away from SourceForge as Boosts' release hosting provider.

One thing that we found that couldn't be replicated at any of the other major OSS hosting sites was the ability to have the windows binaries in the same location. These files are quite large (200MB x 12 visual studio version + 1.4GB for the combined archive), and none of the sites that we looked at could support them. There was also (I believe) a general consensus that we wanted to keep those at the same place as the source distribution. That said, I think we have other (secondary?) locations that serve the source distro, so maybe it would be worthwhile to point your travis jobs there instead of sourceforge? Tom

Rene Rivera

8:37 p.m.

On Sun, Jan 22, 2017 at 10:35 AM, Louis Dionne wrote:

...

Hi,

I know this has been discussed in the past, but I would like to revive the discussion about moving away from SourceForge as Boosts' release hosting provider. SourceForge has at least the following issues:

- There is no way of controlling the format when downloading an archive of the latest release. This can break people that rely on this in their CI scripts. For example, Metabench's[1] CI was broken for a while when the archive changed from .bz2 to .zip.

True.. But what you observed was a human error. The release team forgot to set the default OS download file. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Diego Rodriguez-Losada

23 Jan 23 Jan

5:44 p.m.

On Sun, Jan 22, 2017 at 5:35 PM, Louis Dionne wrote:

...

Hi,

I know this has been discussed in the past, but I would like to revive the discussion about moving away from SourceForge as Boosts' release hosting provider. SourceForge has at least the following issues:

- There is no way of controlling the format when downloading an archive of the latest release. This can break people that rely on this in their CI scripts. For example, Metabench's[1] CI was broken for a while when the archive changed from .bz2 to .zip.

- SourceForge has been in the middle of some controversies[2].

- Their reliability has not been so great. In the past year, I've had quite a few of my Travis jobs fail due to SourceForge being unresponsive.

There are most likely other concerns, but these are the ones that really bite me as a user right now. Am I the only one? If there's some push to do the move, the two main candidates I see would be

- GitHub's native hosting - Bintray

Thoughts? Louis

Hi all, As part of JFrog, I would like to offer Bintray services for this migration. Bintray counts with several features that could be relevant for the Boost community and users: * Not stagnating - 2 billion downloads a month and growing * Backed up by Akamai – the most powerful CDN in the world for faster downloads * Watches and badges – uses can follow and get notifications when new versions are released * Organizations support – people that create and publish Boost can be part of the organization with powerful permissions * REST API – tooling around resolution and deployments of Boost, it's very easy and powerful with Bintray Bintray will sponsor the hosting, so no costs for the Boost community, in a similar way as JFrog did for hosting the Mac OSX Homebrew binary bottles. We might be able to also provide support for executing the migration, writing scripts for copying the artifacts from sourceforge, etc. Best Diego Diego Rodriguez-Losada Senior SW Engineer @JFrog

...

[1]: https://github.com/ldionne/metabench [2]: https://en.wikipedia.org/wiki/SourceForge#Controversies

-- View this message in context: http://boost.2283326.n4. nabble.com/Moving-away-from-SourceForge-tp4691184.html Sent from the Boost - Dev mailing list archive at Nabble.com.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Rene Rivera

6:02 p.m.

On Mon, Jan 23, 2017 at 11:44 AM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...

On Sun, Jan 22, 2017 at 5:35 PM, Louis Dionne wrote:

Hi all,

As part of JFrog, I would like to offer Bintray services for this migration. Bintray counts with several features that could be relevant for the Boost community and users:

* Not stagnating - 2 billion downloads a month and growing * Backed up by Akamai – the most powerful CDN in the world for faster downloads * Watches and badges – uses can follow and get notifications when new versions are released * Organizations support – people that create and publish Boost can be part of the organization with powerful permissions * REST API – tooling around resolution and deployments of Boost, it's very easy and powerful with Bintray

Bintray will sponsor the hosting, so no costs for the Boost community, in a similar way as JFrog did for hosting the Mac OSX Homebrew binary bottles. We might be able to also provide support for executing the migration, writing scripts for copying the artifacts from sourceforge, etc.

FYI.. We already use Bintray, and it was the first alternative I tried after evaluating various https://bintray.com/boostorg. But there where various problems we encountered to use it for general releases. The key one being what Tom mentioned: === One thing that we found that couldn't be replicated at any of the other major OSS hosting sites was the ability to have the windows binaries in the same location. These files are quite large (200MB x 12 visual studio version + 1.4GB for the combined archive), and none of the sites that we looked at could support them. There was also (I believe) a general consensus that we wanted to keep those at the same place as the source distribution. === Is it possible to support that use case? As an example here are the files we are talking about we had problems with < https://sourceforge.net/projects/boost/files/boost-binaries/1.63.0/>. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Diego Rodriguez-Losada

7:44 p.m.

On Mon, Jan 23, 2017 at 7:02 PM, Rene Rivera wrote:

...

On Mon, Jan 23, 2017 at 11:44 AM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
On Sun, Jan 22, 2017 at 5:35 PM, Louis Dionne wrote:

Hi all,

As part of JFrog, I would like to offer Bintray services for this migration. Bintray counts with several features that could be relevant for the Boost community and users:

* Not stagnating - 2 billion downloads a month and growing * Backed up by Akamai – the most powerful CDN in the world for faster downloads * Watches and badges – uses can follow and get notifications when new versions are released * Organizations support – people that create and publish Boost can be part of the organization with powerful permissions * REST API – tooling around resolution and deployments of Boost, it's very easy and powerful with Bintray

Bintray will sponsor the hosting, so no costs for the Boost community, in a similar way as JFrog did for hosting the Mac OSX Homebrew binary bottles. We might be able to also provide support for executing the migration, writing scripts for copying the artifacts from sourceforge, etc.

FYI.. We already use Bintray, and it was the first alternative I tried after evaluating various https://bintray.com/boostorg. But there where various problems we encountered to use it for general releases. The key one being what Tom mentioned:

=== One thing that we found that couldn't be replicated at any of the other major OSS hosting sites was the ability to have the windows binaries in the same location. These files are quite large (200MB x 12 visual studio version + 1.4GB for the combined archive), and none of the sites that we looked at could support them. There was also (I believe) a general consensus that we wanted to keep those at the same place as the source distribution. ===

Is it possible to support that use case? As an example here are the files we are talking about we had problems with < https://sourceforge.net/projects/boost/files/boost-binaries/1.63.0/>.

Yes, the issue is that you probably hit the OSS quota, not prepared for such large binaries. Did you contact bintray for support on this? The sponsoring would consist of raising the quota, specifically for the Boost project. I am working on the details and checking that there aren't other limitations, I will contact back when I have more info. Diego

...

-- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Rene Rivera

7:53 p.m.

On Mon, Jan 23, 2017 at 1:44 PM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...

On Mon, Jan 23, 2017 at 7:02 PM, Rene Rivera wrote:

...
On Mon, Jan 23, 2017 at 11:44 AM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
On Sun, Jan 22, 2017 at 5:35 PM, Louis Dionne wrote:

Hi all,

As part of JFrog, I would like to offer Bintray services for this migration. Bintray counts with several features that could be relevant for the Boost community and users:

* Not stagnating - 2 billion downloads a month and growing * Backed up by Akamai – the most powerful CDN in the world for faster downloads * Watches and badges – uses can follow and get notifications when new versions are released * Organizations support – people that create and publish Boost can be part of the organization with powerful permissions * REST API – tooling around resolution and deployments of Boost, it's very easy and powerful with Bintray

Bintray will sponsor the hosting, so no costs for the Boost community, in a similar way as JFrog did for hosting the Mac OSX Homebrew binary bottles. We might be able to also provide support for executing the migration, writing scripts for copying the artifacts from sourceforge, etc.

FYI.. We already use Bintray, and it was the first alternative I tried after evaluating various https://bintray.com/boostorg. But there where various problems we encountered to use it for general releases. The key one being what Tom mentioned:

=== One thing that we found that couldn't be replicated at any of the other major OSS hosting sites was the ability to have the windows binaries in the same location. These files are quite large (200MB x 12 visual studio version + 1.4GB for the combined archive), and none of the sites that we looked at could support them. There was also (I believe) a general consensus that we wanted to keep those at the same place as the source distribution. ===

Is it possible to support that use case? As an example here are the files we are talking about we had problems with < https://sourceforge.net/projects/boost/files/boost-binaries/1.63.0/>.

Yes, the issue is that you probably hit the OSS quota, not prepared for such large binaries. Did you contact bintray for support on this?

I don't remember if I did.. I know I contacted them for various issues but don't know if that was one of them. The sponsoring would consist of raising the quota, specifically for the

...

Boost project.

OK.

...

I am working on the details and checking that there aren't other limitations, I will contact back when I have more info.

Great. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Diego Rodriguez-Losada

24 Jan 24 Jan

9:42 p.m.

On Mon, Jan 23, 2017 at 8:53 PM, Rene Rivera wrote:

...

On Mon, Jan 23, 2017 at 1:44 PM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
On Mon, Jan 23, 2017 at 7:02 PM, Rene Rivera wrote:

...
On Mon, Jan 23, 2017 at 11:44 AM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
On Sun, Jan 22, 2017 at 5:35 PM, Louis Dionne wrote:

Hi all,

As part of JFrog, I would like to offer Bintray services for this migration. Bintray counts with several features that could be relevant for the Boost community and users:

* Not stagnating - 2 billion downloads a month and growing * Backed up by Akamai – the most powerful CDN in the world for faster downloads * Watches and badges – uses can follow and get notifications when new versions are released * Organizations support – people that create and publish Boost can be part of the organization with powerful permissions * REST API – tooling around resolution and deployments of Boost, it's very easy and powerful with Bintray

Bintray will sponsor the hosting, so no costs for the Boost community, in a similar way as JFrog did for hosting the Mac OSX Homebrew binary bottles. We might be able to also provide support for executing the migration, writing scripts for copying the artifacts from sourceforge, etc.

FYI.. We already use Bintray, and it was the first alternative I tried after evaluating various https://bintray.com/boostorg. But there where various problems we encountered to use it for general releases. The key one being what Tom mentioned:

=== One thing that we found that couldn't be replicated at any of the other major OSS hosting sites was the ability to have the windows binaries in the same location. These files are quite large (200MB x 12 visual studio version + 1.4GB for the combined archive), and none of the sites that we looked at could support them. There was also (I believe) a general consensus that we wanted to keep those at the same place as the source distribution. ===

Is it possible to support that use case? As an example here are the files we are talking about we had problems with < https://sourceforge.net/projects/boost/files/boost-binaries/1.63.0/>.

Yes, the issue is that you probably hit the OSS quota, not prepared for such large binaries. Did you contact bintray for support on this?

I don't remember if I did.. I know I contacted them for various issues but don't know if that was one of them.

The sponsoring would consist of raising the quota, specifically for the

...
Boost project.

Hi again, The quotas have been raised for bintray/boostorg organization, up to 100Gb storage, 20Tb traffic and 2.5Gb file size. It should be good for hosting the Windows binaries too in the same place. Just a question (from my ignorance), why the 1.5Gb msvc-all archive? The download stats clearly show a high preference for specific installers, maybe that one could be dropped without much impact. Please tell me if you have any other issues, feedback, or need further help. Best, Diego

...

...
I am working on the details and checking that there aren't other limitations, I will contact back when I have more info.

Great.

-- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Tom Kent

26 Jan 26 Jan

3:21 a.m.

On Tue, Jan 24, 2017 at 3:42 PM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...

Just a question (from my ignorance), why the 1.5Gb msvc-all archive? The download stats clearly show a high preference for specific installers, maybe that one could be dropped without much impact.

There are lots of use cases where users will need multiple different versions of the libraries. Commonly, both the 32 and 64 bit version of a single visual studio type. However, if you are building a library for distribution, you will probably want to support multiple versions of visual studio with your library, so you'll need the boost libraries for all those. The lack of a package that supported all the different versions is what originally drove me to start volunteering to do the windows builds...I'd really like to see it stick around :-) Tom

Diego Rodriguez-Losada

8:39 a.m.

On Thu, Jan 26, 2017 at 4:21 AM, Tom Kent wrote:

...

On Tue, Jan 24, 2017 at 3:42 PM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
Just a question (from my ignorance), why the 1.5Gb msvc-all archive? The download stats clearly show a high preference for specific installers, maybe that one could be dropped without much impact.

There are lots of use cases where users will need multiple different versions of the libraries. Commonly, both the 32 and 64 bit version of a single visual studio type. However, if you are building a library for distribution, you will probably want to support multiple versions of visual studio with your library, so you'll need the boost libraries for all those.

The lack of a package that supported all the different versions is what originally drove me to start volunteering to do the windows builds...I'd really like to see it stick around :-)

Sure, that makes sense. I am not advocating at all to remove the msvc-all archive, just understanding the context, thanks :) Diego

...

Tom

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Olaf van der Spek

8:43 a.m.

On Thu, Jan 26, 2017 at 4:21 AM, Tom Kent wrote:

...

On Tue, Jan 24, 2017 at 3:42 PM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
Just a question (from my ignorance), why the 1.5Gb msvc-all archive? The download stats clearly show a high preference for specific installers, maybe that one could be dropped without much impact.

There are lots of use cases where users will need multiple different versions of the libraries. Commonly, both the 32 and 64 bit version of a single visual studio type. However, if you are building a library for distribution, you will probably want to support multiple versions of visual studio with your library, so you'll need the boost libraries for all those.

The lack of a package that supported all the different versions is what originally drove me to start volunteering to do the windows builds...I'd really like to see it stick around :-)

Why is building from source not an option? -- Olaf

Tom Kent

12:55 p.m.

On Thu, Jan 26, 2017 at 2:43 AM, Olaf van der Spek wrote:

...

On Thu, Jan 26, 2017 at 4:21 AM, Tom Kent wrote:

...
On Tue, Jan 24, 2017 at 3:42 PM, Diego Rodriguez-Losada < diego.rlosada@gmail.com> wrote:

...
Just a question (from my ignorance), why the 1.5Gb msvc-all archive? The download stats clearly show a high preference for specific installers, maybe that one could be dropped without much impact.

There are lots of use cases where users will need multiple different versions of the libraries. Commonly, both the 32 and 64 bit version of a single visual studio type. However, if you are building a library for distribution, you will probably want to support multiple versions of visual studio with your library, so you'll need the boost libraries for all those.

The lack of a package that supported all the different versions is what originally drove me to start volunteering to do the windows builds...I'd really like to see it stick around :-)

Why is building from source not an option?

It is definitely an option, I often recommend it to people because it isn't very hard (a couple extra dependencies and a new tool to learn, but I'm not being sarcastic when I say it isn't hard). However that isn't how windows development often happens. I know that where I work, we have static libs pre-built for all our dependencies. Then when a developer makes a build of the product, they just link in all those dependencies. Tom

Louis Dionne

4:25 p.m.

Diego Rodriguez-Losada wrote

...

The quotas have been raised for bintray/boostorg organization, up to 100Gb storage, 20Tb traffic and 2.5Gb file size. It should be good for hosting the Windows binaries too in the same place.

Rene, It seems like the main issues that were preventing our transition to Bintray are lifted now, right? If so, what would be the path forward to moving away from SourceForge? How can I help? Louis -- View this message in context: http://boost.2283326.n4.nabble.com/Moving-away-from-SourceForge-tp4691184p46... Sent from the Boost - Dev mailing list archive at Nabble.com.

Jeff Trull

6:39 a.m.

On Sun, Jan 22, 2017 at 8:35 AM, Louis Dionne wrote:

...

the two main candidates I see would be

- GitHub's native hosting - Bintray

I think someone may have gotten there already! https://bintray.com/boostorg I understand they are moving to support conan.io packages (and have hired the developers!). Cheers, Jeff

Diego Rodriguez-Losada

9:17 a.m.

On Thu, Jan 26, 2017 at 7:39 AM, Jeff Trull wrote:

...

On Sun, Jan 22, 2017 at 8:35 AM, Louis Dionne wrote:

...
the two main candidates I see would be

- GitHub's native hosting - Bintray

I think someone may have gotten there already! https://bintray.com/boostorg

I understand they are moving to support conan.io packages (and have hired the developers!).

Hi Jeff, Yes, that boostorg account was created before this discussion by René, Tom & others, but failed due to make it work due to Boost large sizes and default Bintray OSS quotas. We just raised the OSS quota for boostorg, so they can try again. Please tell me any other possible issue you might have. I think there was also some interrupted transfers. To be honest, in my experience that is a typical thing depending on the network, not the service. I experience failures in AWS S3 transfers more often than desired too. But please tell me, and I will handle the issues with the Bintray team. Yes, JFrog have hired the Conan developers, in fact one of them is me :) It is possible that we will provide Bintray support for conan packages, as this is one of the long standing feature request for conan.io, as bintray is much more powerful than conan.io. It might take some time. So far, conan has already been integrated with Artifactory. However, this is an independent effort, I saw Louis Dionne email proposing github or Bintray (I agree that SourceForge might not be the best distribution channel), and some responses about github limitations. So I decided to try to promote this sponsoring for the Boost community inside JFrog. Other relevant projects, as OSX homebrew did it in the past, and I thought Boost deserved it too. Cheers, Diego

...

Cheers, Jeff

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

2845

Age (days ago)

2849

Last active (days ago)

List overview

Download

17 comments

9 participants

participants (9)

Andrey Semashev
Brittany Friedman
Diego Rodriguez-Losada
Jeff Trull
Louis Dionne
Niall Douglas
Olaf van der Spek
Rene Rivera
Tom Kent