Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

newer
[thread] Errors when building wave

older
[container] Experimental support...

Boris Schäling

9 Feb 2014 9 Feb '14

7:06 p.m.

Create a Bjam clone based on the Boost libraries 1. Write a parser for the most important rules (exe, lib and so on) based on Boost.Spirit 2. Use the parser's actions to create a dependency tree based on Boost.Graph 3. Attach Python functions to the nodes in the dependency tree based on Boost.Python 4. Implement Python functions for one toolset (like g++) The project promises a lot of variety as different Boost libraries have to be used. It's also an opportunity to get familiar with various Boost libraries. As it can be broken down into tasks it shouldn't be that difficult to make a realistic plan for a summer project. If there is anyone interested have a look at https://github.com/boostcon/2011_presentations/raw/master/mon/Boost.Build.pd... and http://www.highscore.de/cpp/boostbuild/ for more information about the build system. And of course the idea is not to create a fully featured Bjam clone in a few months. If there is a binary at the end of the project which can build a software based on a simple Jamfile the project is a success. It's more important to have all the required components in place even if they are far from complete (but once they are in place it shouldn't be too difficult to extend them later and add support for another rule for example). Boris

Show replies by date

Mateusz Łoskot

9 Feb 9 Feb

8:43 p.m.

On 9 February 2014 19:06, Boris Schäling wrote:

...

Create a Bjam clone based on the Boost libraries

1. Write a parser for the most important rules (exe, lib and so on) based on Boost.Spirit 2. Use the parser's actions to create a dependency tree based on Boost.Graph 3. Attach Python functions to the nodes in the dependency tree based on Boost.Python 4. Implement Python functions for one toolset (like g++)

The project promises a lot of variety as different Boost libraries have to be used. It's also an opportunity to get familiar with various Boost libraries. As it can be broken down into tasks it shouldn't be that difficult to make a realistic plan for a summer project.

I think it's an interesting idea. It could open some new opportunities for Boost.Build. There are a few additional related ideas which have been discussed here and on IRC: - Boost.Build run as a daemon (there is b2 --daemon prototype, based on dbus) - Boost.Build command line interface could be improved - Boost.Build API as a regular library, as lightweight network protocol based on JSON - Boost.Build API could allow implementations not only in Python, but also in <your favourite scripting language here> Some of opportunities could be integration with IDEs. I have a functional prototype for Qt Creator, but it has obvious limitations due to lack of complete Boost.Build parser, or better, Boost.Build API: https://github.com/mloskot/qt-creator-plugin-boostbuild Best regards, -- Mateusz Łoskot, http://mateusz.loskot.net

Alligand Edouard

10 Feb 10 Feb

7:20 a.m.

...

1. Write a parser for the most important rules (exe, lib and so on) based on Boost.Spirit 2. Use the parser's actions to create a dependency tree based on Boost.Graph 3. Attach Python functions to the nodes in the dependency tree based on Boost.Python 4. Implement Python functions for one toolset (like g++)

What about using that parser to create a tool that would transform bjam files into different files (ninja, make, cmake, etc.)? That would be very useful in order to integrate Boost libraries into existing projects. --Edouard

Roland Bock

8:38 a.m.

On 2014-02-10 08:20, Alligand Edouard wrote:

...

...
1. Write a parser for the most important rules (exe, lib and so on) based on Boost.Spirit 2. Use the parser's actions to create a dependency tree based on Boost.Graph 3. Attach Python functions to the nodes in the dependency tree based on Boost.Python 4. Implement Python functions for one toolset (like g++) What about using that parser to create a tool that would transform bjam files into different files (ninja, make, cmake, etc.)?

That would be very useful in order to integrate Boost libraries into existing projects.

--Edouard

Nice idea, +1 And for people like me who cannot wrap their head around the logic of bjam files: Translate from ninja, cmake and co to bjam...

Steven Watanabe

5:15 p.m.

AMDG On 02/10/2014 12:38 AM, Roland Bock wrote:

...

On 2014-02-10 08:20, Alligand Edouard wrote:

...
What about using that parser to create a tool that would transform bjam files into different files (ninja, make, cmake, etc.)?

That would be very useful in order to integrate Boost libraries into existing projects.

Nice idea, +1

And for people like me who cannot wrap their head around the logic of bjam files: Translate from ninja, cmake and co to bjam...

I don't believe that either of these is really viable. CMake and Boost.Build both use complete programming languages and there is not a 1-to-1 mapping between the target models. Not to mention that Boost.Build and CMake handle relative paths differently and translating them correctly requires the ability to know what is a path and what is some other string, which is basically impossible. Boost.Build to ninja or Boost.Build to make is probably possible, as long as you're willing to accept that all configuration is handled during the translation (just like CMake to X), since it's just a matter of dumping the dependency graph. In Christ, Steven Watanabe

Niall Douglas

3:47 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

On 9 Feb 2014 at 20:06, Boris Schäling wrote:

...

Create a Bjam clone based on the Boost libraries

That is a huge project, one which would take many summers and a hugely dedicated mentor or mentors, even to deliver a demonstration prototype of reasonable quality. Designing a correct build tool is incredibly hard. As I like the API of scons, and we have scons build files for most Boost libraries, I'd suggest instead helping out with scons-plusplus which is Boost.Python based. In truth though, an O(N) scalable but not-painful-to-write-for build system almost certainly needs a persistently stored graph database. Which just gave you a hint as to my C++ Now Presentation if I'm chosen to present ... Niall --- Boost C++ Libraries Google Summer of Code 2014 admin https://svn.boost.org/trac/boost/wiki/SoC2014

Hartmut Kaiser

4:42 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

...

1. Write a parser for the most important rules (exe, lib and so on) based on Boost.Spirit 2. Use the parser's actions to create a dependency tree based on Boost.Graph 3. Attach Python functions to the nodes in the dependency tree based on Boost.Python 4. Implement Python functions for one toolset (like g++)

The project promises a lot of variety as different Boost libraries have to be used. It's also an opportunity to get familiar with various Boost libraries. As it can be broken down into tasks it shouldn't be that difficult to make a realistic plan for a summer project.

If there is anyone interested have a look at <https://github.com/boostcon/2011_presentations/raw/master/mon/Boost.Build .pdf> and http://www.highscore.de/cpp/boostbuild/ for more information about the build system.

And of course the idea is not to create a fully featured Bjam clone in a few months. If there is a binary at the end of the project which can build a software based on a simple Jamfile the project is a success. It's more important to have all the required components in place even if they are far from complete (but once they are in place it shouldn't be too difficult to extend them later and add support for another rule for example).

What about getting the CMake support finalized instead? Who needs yet another build system for Boost? Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

Boris Schäling

5:36 p.m.

On Mon, 10 Feb 2014 17:42:25 +0100, Hartmut Kaiser wrote:

...

[...]What about getting the CMake support finalized instead? Who needs yet

That's also a good idea - will you add it to https://svn.boost.org/trac/boost/wiki/SoC2014 with a few more details for students to understand what will need to be done? Boris

Hartmut Kaiser

5:39 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

...

On Mon, 10 Feb 2014 17:42:25 +0100, Hartmut Kaiser wrote:

...
[...]What about getting the CMake support finalized instead? Who needs yet

That's also a good idea - will you add it to https://svn.boost.org/trac/boost/wiki/SoC2014 with a few more details for students to understand what will need to be done?

Sorry, I don't know nearly enough about this to be able to tell what needs to be done here. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

Stephen Kelly

16 Feb 16 Feb

8:50 a.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

Hartmut Kaiser wrote:

...

...
On Mon, 10 Feb 2014 17:42:25 +0100, Hartmut Kaiser wrote:

...
[...]What about getting the CMake support finalized instead? Who needs yet

That's also a good idea - will you add it to https://svn.boost.org/trac/boost/wiki/SoC2014 with a few more details for students to understand what will need to be done?

Sorry, I don't know nearly enough about this to be able to tell what needs to be done here.

Technically, there is not much to do. Certainly not enough for a GSOC, and certainly stuff that should be done by experts (eg me, Daniel). The blocker to moving to CMake is acceptance of it as a goal by the boost community. That's not something for a GSoC to resolve. Thanks, Steve.

Boris Schäling

12:03 p.m.

On Sun, 16 Feb 2014 09:50:56 +0100, Stephen Kelly wrote:

...

[...]The blocker to moving to CMake is acceptance of it as a goal by the boost community. That's not something for a GSoC to resolve.

Steve, do you know whether this is blocking the next release of the Boost libraries (1.56.0)? Or does this have more to do with the move from Subversion to Git and is not related to CMake? Boris

Stephen Kelly

12:44 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

Boris Schäling wrote:

...

On Sun, 16 Feb 2014 09:50:56 +0100, Stephen Kelly wrote:

...
[...]The blocker to moving to CMake is acceptance of it as a goal by the boost community. That's not something for a GSoC to resolve.

Steve,

do you know whether this is blocking the next release of the Boost libraries (1.56.0)?

I don't understand the question. I can't tell you anything about what is or is not blocking Boost 1.56.0. My impression that moving to CMake is not a goal for the Boost community come from mails like these: http://thread.gmane.org/gmane.comp.lib.boost.devel/248928/focus=248956 http://thread.gmane.org/gmane.comp.lib.boost.devel/248928/focus=249009 http://thread.gmane.org/gmane.comp.lib.boost.build/26227/focus=26252 Clearly, even if some people want to move to CMake, others do not. I am not here to convince anyone. I designed and implemented many features in CMake 3.0 (due RealSoonNow) specifically for boost use-cases, eg INTERFACE_LIBRARY: http://www.cmake.org/cmake/help/git-next/manual/cmake-buildsystem.7.html#int... The innuendo on the boost.build list that a CMake migration would go anything like the migration to fractured (not modularized!) git repos is unfounded. Technically, there are no blockers. The only blocker is that it is not actually a goal of Boost as far as I can tell, and no one is championing changing that (I am certainly not volunteering to attempt to convince anyone). If someone decides to champion that, the technical backing is there. I expect there are minor issues to solve which we have not solved yet, but I have no reason to think that there are blockers. Thanks, Steve.

Edward Diener

3:31 p.m.

On 2/16/2014 7:44 AM, Stephen Kelly wrote:

...

Boris Schäling wrote:

...
On Sun, 16 Feb 2014 09:50:56 +0100, Stephen Kelly wrote:

...
[...]The blocker to moving to CMake is acceptance of it as a goal by the boost community. That's not something for a GSoC to resolve.

Steve,

do you know whether this is blocking the next release of the Boost libraries (1.56.0)?

I don't understand the question. I can't tell you anything about what is or is not blocking Boost 1.56.0.

My impression that moving to CMake is not a goal for the Boost community come from mails like these:

http://thread.gmane.org/gmane.comp.lib.boost.devel/248928/focus=248956 http://thread.gmane.org/gmane.comp.lib.boost.devel/248928/focus=249009 http://thread.gmane.org/gmane.comp.lib.boost.build/26227/focus=26252

Clearly, even if some people want to move to CMake, others do not. I am not here to convince anyone.

I designed and implemented many features in CMake 3.0 (due RealSoonNow) specifically for boost use-cases, eg INTERFACE_LIBRARY:

http://www.cmake.org/cmake/help/git-next/manual/cmake-buildsystem.7.html#int...

The innuendo on the boost.build list that a CMake migration would go anything like the migration to fractured (not modularized!) git repos is unfounded. Technically, there are no blockers. The only blocker is that it is not actually a goal of Boost as far as I can tell, and no one is championing changing that (I am certainly not volunteering to attempt to convince anyone). If someone decides to champion that, the technical backing is there. I expect there are minor issues to solve which we have not solved yet, but I have no reason to think that there are blockers.

I have found understanding bjam and Boost Build largely impenetrable based on its documentation. I am not criticizing its functionality or how well it works for the vast majority of normal cases. It is when one needs to do something outside of the norm that I have found it very difficult, ie. actually writing bjam files with an undestanding of how they fit in with the Boost Build system. I know nothing about CMake. If it could provide the functionality that bjam/Boost Build provides in an understandable and flexible way so that it would be easier for non-experts to manipulate the build of libraries, test cases, documentation, I would love it as at least an alternative to current Boost Build. Without knowing anything about CMake my one worry about it is that it is a generalized build system and I do not know if it could be made as flexible as Boost Build in being tailored toward Boost developers. This is just purely my personal opinion and I have no idea how the majority of Boost library developers or users feel about it, much less the acknowledged leaders in the Boost community of developers.

Stephen Kelly

3:39 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

Edward Diener wrote:

...

I know nothing about CMake. If it could provide the functionality that bjam/Boost Build provides in an understandable and flexible way so that it would be easier for non-experts to manipulate the build of libraries, test cases, documentation, I would love it as at least an alternative to current Boost Build.

Without knowing anything about CMake my one worry about it is that it is a generalized build system and I do not know if it could be made as flexible as Boost Build in being tailored toward Boost developers.

If you wish to get answers to those concerns and educate yourself about it, I suggest you (or another champion) ask about them on the cmake users mailing list. It is very good. Having answers to share would be a lot better than sharing vague concerns out of not knowing the answers. I don't want to engage with such vagueness, but if you decide to write to the cmake users mailing list, you'll naturally need to answer some of your own questions in order to ask the right question on the cmake users list. Thanks, Steve.

Klaim - Joël Lamotte

3:41 p.m.

On Sun, Feb 16, 2014 at 4:31 PM, Edward Diener wrote:

...

I have found understanding bjam and Boost Build largely impenetrable based on its documentation. I am not criticizing its functionality or how well it works for the vast majority of normal cases. It is when one needs to do something outside of the norm that I have found it very difficult, ie. actually writing bjam files with an undestanding of how they fit in with the Boost Build system.

I know nothing about CMake. If it could provide the functionality that bjam/Boost Build provides in an understandable and flexible way so that it would be easier for non-experts to manipulate the build of libraries, test cases, documentation, I would love it as at least an alternative to current Boost Build.

Without knowing anything about CMake my one worry about it is that it is a generalized build system and I do not know if it could be made as flexible as Boost Build in being tailored toward Boost developers.

This is just purely my personal opinion and I have no idea how the majority of Boost library developers or users feel about it, much less the acknowledged leaders in the Boost community of developers.

A quick informative note to complete that: I'm not a CMake advocate or against boost being converted to it, but one important change to note with CMake compared to b2 is that bootstrapping from just boost source code clone would not be possible as it is now (except if you add cmake code into boost source repo). I don't see this as a problem because almost all my C++ projects already use CMake, but maybe that's one of the arguments used Against putting more efforts in a CMake conversion (because some people see CMake as an additional dependency instead of a meta-buildsystem). I'll just add a quick thanks to Stephen Kelly who added a lot of interesting features to recent and future CMake versions which will make my life less miserable when setting up complex projects.

Niall Douglas

5:15 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

On 16 Feb 2014 at 16:41, Klaim - Joël Lamotte wrote:

...

A quick informative note to complete that: I'm not a CMake advocate or against boost being converted to it, but one important change to note with CMake compared to b2 is that bootstrapping from just boost source code clone would not be possible as it is now (except if you add cmake code into boost source repo).

In my past, non-Boost, projects I simply included the CMake output in the source repo. That solved the bootstrapping problem for those without CMake installed i.e. usually the end users, not library developers. I certainly found no problem with this approach, and I've seen plenty of other open source libraries which bolted on CMake on top of their preexisting build system do the same. (My only request is that if you do do this, please stick a great big auto-generated warning on the top of every bjam file output. I've got nobbled by my build script changes getting eaten too many times in my life!) Niall -- Currently unemployed and looking for work in Ireland. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/

Stephen Kelly

9:16 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

Niall Douglas wrote:

...

In my past, non-Boost, projects I simply included the CMake output in the source repo. That solved the bootstrapping problem for those without CMake installed i.e. usually the end users, not library developers. I certainly found no problem with this approach,

I'm surprised that approach worked in any case at all. CMake buildsystems invoke the cmake executable for various things. You can find all kinds of trolling because of that. Thanks, Steve.

Stephen Kelly

9:06 p.m.

New subject: Just another GSoC project idea: Create a Bjam clone based on the Boost libraries

Klaim - Joël Lamotte wrote:

...

I'll just add a quick thanks to Stephen Kelly who added a lot of interesting features to recent and future CMake versions which will make my life less miserable when setting up complex projects.

I'm glad it's useful for you, and thanks for the appreciation. For others, some (definitely not all :) ) of the interesting features are described here: http://www.kdab.com/modern-cmake-with-qt-and-boost Thanks, Steve.

Lars Viklund

12:45 p.m.

On Sun, Feb 16, 2014 at 01:03:50PM +0100, Boris Schäling wrote:

...

On Sun, 16 Feb 2014 09:50:56 +0100, Stephen Kelly wrote:

...
[...]The blocker to moving to CMake is acceptance of it as a goal by the boost community. That's not something for a GSoC to resolve.

Steve,

do you know whether this is blocking the next release of the Boost libraries (1.56.0)? Or does this have more to do with the move from Subversion to Git and is not related to CMake?

I'm not Steve, but I'd like to offer my input on this. As I gather from lists and IRC it seems that the delay of 1.56.0 is largely from the major upheaval in the release process that the VCS migration and modularisation has caused. In particular, it seems rather hard to develop and make releases when you: * don't know what branches to work in; * don't have the regression test suite working; * have your work disturbed by accidents in other libraries; * get tripped up by changes in build (b2 headers). Many of those teething problems have been resolved in the time since the move, but some of it still seems in flux. Migrating the build system to CMake and/or the introduction of the mysterious Ryppl would be something disjoint from the current changes in process and preferably something embarked on when the releases start rolling again. As for the grand plan of things, I guess that's something that the Steering Committee could clear up, as I can't find any roadmap documented anywhere in public. -- Lars Viklund | zao@acc.umu.se

Vladimir Prus

17 Feb 17 Feb

4:33 a.m.

On 09.02.2014 23:06, Boris Schäling wrote:

...

Create a Bjam clone based on the Boost libraries

1. Write a parser for the most important rules (exe, lib and so on) based on Boost.Spirit 2. Use the parser's actions to create a dependency tree based on Boost.Graph 3. Attach Python functions to the nodes in the dependency tree based on Boost.Python 4. Implement Python functions for one toolset (like g++)

Boris, personally, it seems it would be more useful to instead improve Boost.Build/Python; for example API with IDE will be certainly useful and sufficiently local project. - Volodya

3922

Age (days ago)

3930

Last active (days ago)

List overview

Download

19 comments

12 participants

participants (12)

Alligand Edouard
Boris Schäling
Edward Diener
Hartmut Kaiser
Klaim - Joël Lamotte
Lars Viklund
Mateusz Łoskot
Niall Douglas
Roland Bock
Stephen Kelly
Steven Watanabe
Vladimir Prus