NuGet is a package manager that has traditionally been focused on .NET, but it has recently added support for C++ libraries. Here is a presentation by the lead developer adding C++ support to NuGet: http://channel9.msdn.com/Events/GoingNative/2013/Find-Build-Share-Use-Using-... I would like to see boost packages added to NuGet, and to have these packages be "official" packages. I am a contributor on another project (Thrift) and I would like to be able to produce NuGet packages for Thrift, and I would like that package to have a "magically" working Boost dependency. Sergey Shandar has made some unofficial packages. Blog describing his effort: http://sergey-shandar.blogspot.com/2013/08/boost-on-nugetorg.html Nuget packages he has submitted: https://www.nuget.org/profiles/sergey.shandar/ If someone (perhaps me) authored the boost NuGet package build process, would the person in charge of releasing Boost be willing to publish the NuGet packages that get created? There are going to be plenty of follow up questions if Boost releases are willing to publish NuGet packages, but I figured I would get the big question out of the way first.
2013/9/16 Ben Craig <ben.craig@ni.com> <...>
I would like to see boost packages added to NuGet, and to have these packages be "official" packages.
Sounds interesting. Try forwarding your letter to Boost.Build developer's list: boost-build@lists.boost.org Also take a look at this thread: http://boost.2283326.n4.nabble.com/Pre-build-MSVC-Boost-binaries-td4650161.h... It contains discussion about the Pre-build-MSVC-Boost-binaries, which are now on the official site. -- Best regards, Antony Polukhin
I will continue this conversation on the boost-build list then. From: Antony Polukhin <antoshkka@gmail.com> To: "boost@lists.boost.org List" <boost@lists.boost.org>, Date: 09/17/2013 10:12 AM Subject: Re: [boost] NuGet packages for boost Sent by: "Boost" <boost-bounces@lists.boost.org> 2013/9/16 Ben Craig <ben.craig@ni.com> <...>
I would like to see boost packages added to NuGet, and to have these packages be "official" packages.
Sounds interesting. Try forwarding your letter to Boost.Build developer's list: boost-build@lists.boost.org Also take a look at this thread: http://boost.2283326.n4.nabble.com/Pre-build-MSVC-Boost-binaries-td4650161.h... It contains discussion about the Pre-build-MSVC-Boost-binaries, which are now on the official site. -- Best regards, Antony Polukhin _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Tue, Sep 17, 2013 at 2:08 PM, Ben Craig <ben.craig@ni.com> wrote:
I will continue this conversation on the boost-build list then.
IMO it should stay on this list. The topic is about users being able to discover, download, and install packages for use with Visual C++, and IIUC would Microsoft's build engine rather than Boost.Build. The boost-build list would be a good resource for questions about how we currently build and test on VC++, but at least for the moment I'd rather keep discussion on this list. There are some policy questions best rhandled on the main list. For example, * While we might want help packaging maintainers, do we actually want to host package management on Boost? If so, which package management tools? There are a number of popular ones out in the wild. * How does NuGet relate to Ryppl? AFAIK, NuGet is just for Windows/Visual Studio users. So while we wish NuGet well, our long term interest is in something much broader than any one platform and much broader that just package management. --Beman
On 17 Sep 2013 at 16:36, Beman Dawes wrote:
* While we might want help packaging maintainers, do we actually want to host package management on Boost? If so, which package management tools? There are a number of popular ones out in the wild.
* How does NuGet relate to Ryppl? AFAIK, NuGet is just for Windows/Visual Studio users. So while we wish NuGet well, our long term interest is in something much broader than any one platform and much broader that just package management.
It seems to me that if you could convert bootstrap.bat to MSBuild, then packaging up Boost as NuGet ought to be fairly straightforward. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
It seems to me that if you could convert bootstrap.bat to MSBuild, then packaging up Boost as NuGet ought to be fairly straightforward.
Niall
It depends on how you want to present things to users. I think that you are suggesting having consuming NuGet clients rebuild boost on their machine. That might work, but that seems like it would require porting a lot of the existing bjam code. A really straightforward approach was prototyped by Sergey Shandar. You build 32-bit Boost and 64-bit Boost, adjust the consuming packages lib path and include path, and you're done. That package is more than 200 MB though, and it didn't include a lot of the build variants that clients might expect. So here are some general build questions that would need to be answered: 1. Which build variants / pivots would we support for "normal" libraries? This has been discussed some here: http://boost.2283326.n4.nabble.com/Pre-build-MSVC-Boost-binaries-td4650161.h... My inclination is to pivot on the following axes for most libraries: release, debug Win32, x64, Arm static, dynamic static-crt, dynamic-crt vc10, vc11_xp, vc12 (xp?) (I don't think earlier IDEs don't support NuGet) The main pivot this is omitting is single threaded vs. multi threaded. I, personally, don't think that single threaded CRTs make much sense these days. 2. Which build variants / pivots would we support for "unusual" libraries? I would call the python, locale, and regex libraries "unusual" in that they have dependencies on non-boost libraries. We need to figure out which versions of the external libraries to use. If these external libraries have dependencies at runtime, then we will need to make sure that they have NuGet packages as well. 3. How should we package this in NuGet? One mega "boost" library, or lots of little "boost-thread" style libraries? My preference is to do lots of little libraries. This is more work. Ideally, we would also mention the dependencies between boost libraries in the NuGet package, but I don't currently see any place where these dependencies are documented. The CoApp tools that make NuGet packaging "easy" are supposed to get an update soon that will allow for redirector packages (my term). These packages don't contain binaries, but they can have conditional dependencies on the particular configuration that you do use. If we used that capability, then users would only download the particular configurations they care about, and not 5 GB worth of binaries.
On 17 Sep 2013 at 16:10, Ben Craig wrote:
It seems to me that if you could convert bootstrap.bat to MSBuild, then packaging up Boost as NuGet ought to be fairly straightforward.
It depends on how you want to present things to users. I think that you are suggesting having consuming NuGet clients rebuild boost on their machine. That might work, but that seems like it would require porting a lot of the existing bjam code.
Not really. If you can get NuGet to spit out a b2.exe, the rest of the build is very easy. My issue with supplying prebuilt binaries via NuGet is that there are a lot of combinations, and a lot of variants because of MSVCRT and STL incompatibilities - for example, VS Express doesn't support some technologies such as OpenMP, so OpenMP using Boost libraries may have an issue. Prebuilt binaries means needing server bandwidth to deliver binaries and a meaty suite of CI build VMs with various Visual Studios, and bear in mind > VS2010 needs Windows 7, so there's a licence fee in there too. If someone of course wants to set all that up and pay for it, I have no objection. Prebuilt binaries are easy on the end user. Niall -- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/
If someone of course wants to set all that up and pay for it, I have no objection. Prebuilt binaries are easy on the end user. Garrett Serack has offered up hosted server VMs with Visual Studio
Responding almost bottom up... preinstalled. So for now, I'm going to pretend that the compilers and licenses won't be an issue.
My issue with supplying prebuilt binaries via NuGet is that there are a lot of combinations, and a lot of variants because of MSVCRT and STL incompatibilities - for example, VS Express doesn't support some technologies such as OpenMP, so OpenMP using Boost libraries may have an issue. I see this as a major challenge as well. However, right now clients have to figure it out on their own. It's even worse for libraries that want to use Boost, but require Boost to be built "correctly". If some library needs Boost built with OpenMP support, I want that library to be able to express that dependency cleanly.
It seems to me that if you could convert bootstrap.bat to MSBuild, then packaging up Boost as NuGet ought to be fairly straightforward.
It depends on how you want to present things to users. I think that you are suggesting having consuming NuGet clients rebuild boost on their machine. That might work, but that seems like it would require porting a lot of the existing bjam code.
Not really. If you can get NuGet to spit out a b2.exe, the rest of the build is very easy. I'm going to rephrase this to check my own understanding. I think that you are suggesting that the bootstrap / b2 build process should get ported to msbuild. Then, when a user installs the Boost NuGet package, b2 will get built, and then the Boost libraries themselves will get built, all using the current compiler.
I think that could work. I don't think NuGet makes it easy. I think it also will cause long install times. But it has the significant benefit of sidestepping download size issues, and pre-building all the configuration options. I think this could also be packaged up in a way that clients that want boost-thread for the static CRT could just build the libraries they need, and no others. So this idea has significant merits, but my current preference is still to have built binaries.
It seems to me that if you could convert bootstrap.bat to MSBuild, then packaging up Boost as NuGet ought to be fairly straightforward.
Niall It depends on how you want to present things to users. I think that you are suggesting having consuming NuGet clients rebuild boost on their machine. That might work, but that seems like it would require porting a lot of the existing bjam code.
A really straightforward approach was prototyped by Sergey Shandar. You build 32-bit Boost and 64-bit Boost, adjust the consuming packages lib path and include path, and you're done. That package is more than 200 MB though, and it didn't include a lot of the build variants that clients might expect. The last version of my Boost NuGet package uses a little bit different approach. The main boost package includes all Boost headers files but compiled files. So it's about 15Mb only. Then there are other packages, such as boost_atomic, boost_chrono, etc. They depends on main boost
On 9/17/2013 2:10 PM, Ben Craig wrote: package and includes lib and dll files. In this case, we can add as many configuration as we like. For example, boost_python27_vc12-1.54.0 which depends on boost-1.54.0, python-2.7.* and Visual Studio 2013. We can add Python as a NuGet package as well.
On 09/17/2013 11:10 PM, Ben Craig wrote:
3. How should we package this in NuGet? One mega "boost" library, or lots of little "boost-thread" style libraries? My preference is to do lots of little libraries. This is more work. Ideally, we would also mention the dependencies between boost libraries in the NuGet package, but I don't currently see any place where these dependencies are documented.
Are you aware of the modularization plan for boost? https://svn.boost.org/trac/boost/wiki/CMakeModularizationStatus The latest effort I'm aware of is this repo which also contains a port to CMake: https://github.com/boost-cmake/boost-cmake Note also that even with code in separate git repos, there is currently a complex mesh of interdependencies between the repos/libraries. The real work is to do actual modularization of the code: http://thread.gmane.org/gmane.comp.lib.boost.devel/243094 Just FYI. There's not likely to be any progress on 'actual modularization' until I get an svn account, and that is proving difficult. I don't know the current schedule for moving to split git repos. Thanks, Steve.
Are you aware of the modularization plan for boost?
https://svn.boost.org/trac/boost/wiki/CMakeModularizationStatus I'm aware now :). I have seen Jens Weller's Boost dependency analyzer ( http://www.meetingcpp.com/index.php/br/items/boost-dependency-analyzer.html ) and I was pretty sure that it was related to efforts like this.
Note also that even with code in separate git repos, there is currently a complex mesh of interdependencies between the repos/libraries. The real work is to do actual modularization of the code: I think that the modularization effort has a lot of value. I don't think it is necessary for NuGet though. As I understand the modularization effort, it is also trying to express the dependencies between headers-only libraries. For NuGet, I would likely just put all the headers in a "boost_includes" package. Sergey has already demonstrated that such an approach is possible.
That said, Google hasn't shown me any documentation on binary-library dependencies. This can be determined empirically, but I am afraid of the information getting stale.
* While we might want help packaging maintainers, do we actually want to host package management on Boost? If so, which package management tools? There are a number of popular ones out in the wild. The boost devs certainly can't provide packages for every package management tool out there. There are almost certainly too many. The question then becomes does it make sense to support -any-, and which ones.
If any package managers are going to be supported for Windows, I think that the obvious choice for September 2013 is NuGet. Next year the answer may be very different. I don't know of any competing package managers on Windows that have shipped, but I may just be ignorant on this point. It would be my preference that the packages be hosted on NuGet.org instead of boost.org though, as I expect most packages to be in that central repository. I don't know how easy it is for a package in one repository to depend on a package in a different repo.
* How does NuGet relate to Ryppl? AFAIK, NuGet is just for Windows/Visual Studio users. So while we wish NuGet well, our long term interest is in something much broader than any one platform and much broader that just package management. Dave Abrahams had similar concerns in ryppl-dev: https://groups.google.com/d/msg/ryppl-dev/aJyyfMftW5E/g9pw6b1NZIUJ
Ryppl does look like it is trying to solve more problems, but NuGet has the distinct advantage of having actually shipped. Nuget is also already integrated into Visual Studio.
On Tue, Sep 17, 2013 at 10:36 PM, Beman Dawes <bdawes@acm.org> wrote:
* How does NuGet relate to Ryppl? AFAIK, NuGet is just for Windows/Visual Studio users. So while we wish NuGet well, our long term interest is in something much broader than any one platform and much broader that just package management.
First, I pointed the subject on the Ryppl mailing list some time ago, there have been a short discussion: https://groups.google.com/forum/#!topic/ryppl-dev/aJyyfMftW5E Second, there is where the efforts are going on (supporting Visual Studio first) and what the communcation says (in theory, it could be used for other platforms and editors, but just installing source code would be enough). I suspect that efforts will not follow communication but there is a recent precedent that might suggest that it might happen, that is the REST API aka Casablanca library, which is supposed to be both open source and cross-platform right now. I also suspect that if a tool like pip or gem, for any language, is not open source then it might be a big no-no for a lot of users.
On Mon, Sep 16, 2013 at 11:59 AM, Ben Craig <ben.craig@ni.com> wrote:
NuGet is a package manager that has traditionally been focused on .NET, but it has recently added support for C++ libraries. Here is a presentation by the lead developer adding C++ support to NuGet:
http://channel9.msdn.com/Events/GoingNative/2013/Find-Build-Share-Use-Using-...
Interesting. It looks like NuGet is Microsoft's attempt at providing the Debian "apt-get" Advance Packaging Tool for Windows. If NuGet comes even close the usefulness of apt-get, it will be a big step forward for Windows developers who want to use open source software, IMO.
I would like to see boost packages added to NuGet, and to have these packages be "official" packages. I am a contributor on another project (Thrift) and I would like to be able to produce NuGet packages for Thrift, and I would like that package to have a "magically" working Boost dependency.
Sergey Shandar has made some unofficial packages. Blog describing his effort: http://sergey-shandar.blogspot.com/2013/08/boost-on-nugetorg.html Nuget packages he has submitted: https://www.nuget.org/profiles/sergey.shandar/
If someone (perhaps me) authored the boost NuGet package build process, would the person in charge of releasing Boost be willing to publish the NuGet packages that get created?
There are going to be plenty of follow up questions if Boost releases are willing to publish NuGet packages, but I figured I would get the big question out of the way first.
I'm very interested. --Beman
participants (7)
-
Antony Polukhin -
Beman Dawes -
Ben Craig -
Klaim - Joël Lamotte -
Niall Douglas -
Sergey Shandar -
Stephen Kelly