[context] Don't hard-code the assembler
AMDG This commit: https://github.com/boostorg/context/commit/7534fe5d1c01e7f543d86607770e9c46a... makes it impossible for me to run b2 --with-context msvc-12.0 outside of the visual studio command prompt. Please just use the assembler that the toolset provides. In Christ, Steven Watanabe
2014-04-01 20:39 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:
Please just use the assembler that the toolset provides.
I've to restrict the assembler implementation for the combination of architecture/address-model/binary-format/ABI to one assembler tool - otherwise I would be forced to provide asm-files for the assembler tools available (masm, nasm, flat asm, ...) - impossible. On Windows MASM is used to compile boost.context - it was shipped with MS Visual Studio an at some point MS decided to move it from the MS Visual Studio to the MS WDK. I'm not responsible how MS installs the development tools - probably you could adjust your search path so you can call masm from the command prompt. but I don't know if masm needs some environment variables set by using the visual studio command prompt.
AMDG On 04/01/2014 02:00 PM, Oliver Kowalke wrote:
2014-04-01 20:39 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:
Please just use the assembler that the toolset provides.
I've to restrict the assembler implementation for the combination of architecture/address-model/binary-format/ABI to one assembler tool - otherwise I would be forced to provide asm-files for the assembler tools available (masm, nasm, flat asm, ...) - impossible.
This is, of course, why you shouldn't be writing assembler in the first place... I think I'd prefer it if Boost.Context just gave an error if I use a toolset that it doesn't support, instead of trying (and failing badly) to switch silently to a different toolset.
On Windows MASM is used to compile boost.context - it was shipped with MS Visual Studio an at some point MS decided to move it from the MS Visual Studio to the MS WDK.
It's still installed along with Visual Studio.
I'm not responsible how MS installs the development tools - probably you could adjust your search path so you can call masm from the command prompt. but I don't know if masm needs some environment variables set by using the visual studio command prompt.
- With <toolset>msvc, the assembler can't be found automatically, even though Boost.Build already knows how to find it. - If the assembler has a name other then as, it cannot be used, even if it would actually work. This is especially likely to be a problem for cross compiling. In Christ, Steven Watanabe
2014-04-02 0:20 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:This is, of course, why you shouldn't be
writing assembler in the first place...
but C++ doesn't provide the required features (for instance swapping the stack pointer) - I'm forced to use assembler. some new compilers (MSVC for instance) do not support inline-assembler :^/
I think I'd prefer it if Boost.Context just gave an error if I use a toolset that it doesn't support, instead of trying (and failing badly) to switch silently to a different toolset.
it does not switch to another toolset - boost.context contains assembler code only and the convention is: - on i386-Windows its uses MASM - on x86_64-Windows its uses MASM64 - on arm-Windows armasm - on all other platforms GNU as
On Windows MASM is used to compile boost.context - it was shipped with MS Visual Studio an at some point MS decided to move it from the MS Visual Studio to the MS WDK.
It's still installed along with Visual Studio.
masm64 too?
- With <toolset>msvc, the assembler can't be found automatically, even though Boost.Build already knows how to find it.
hmm - it works for me - 'b2 toolset=msvc-10.0' builds boost.context on Win32
- If the assembler has a name other then as, it cannot be used, even if it would actually work. This is especially likely to be a problem for cross compiling.
what would be a solution? If I would let the toolset choose the assembler tool I would be forced to support the different syntax of those assembler tools (which would be a lot if you know how many assemblers are available). MASM and GNU as are the default assemblers an the related platforms
some new compilers (MSVC for instance) do not support inline-assembler :^/ http://msdn.microsoft.com/en-us/library/4ks26t93.aspx
As far as I can tell MSVC still supports inline assembler, see above link. Joshua Scholar
On 03/04/14 04:17, Joshua Scholar wrote:
some new compilers (MSVC for instance) do not support inline-assembler :^/ http://msdn.microsoft.com/en-us/library/4ks26t93.aspx
As far as I can tell MSVC still supports inline assembler, see above link.
MSVC doesn't support inline assembler for ARM and x86-64, two architectures supported by Boost.Context. Even if it did, multiple versions would still be required because different compilers use different dialects of assembler. I think it's a good idea to stick to a single version of the file for maintenance purposes.
Oops I should read more carefully, it said that MSVC only supports inline assembler on 32 bit code. Sorry for that.
AMDG On 04/01/2014 11:13 PM, Oliver Kowalke wrote:
2014-04-02 0:20 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:This is,
On Windows MASM is used to compile boost.context - it was shipped with MS Visual Studio an at some point MS decided to move it from the MS Visual Studio to the MS WDK.
It's still installed along with Visual Studio.
masm64 too?
Yes. My msvc 12 express installation has VC\bin\ml.exe and VC\bin\x86_amd64\ml64.exe In Christ, Steven Watanabe
AMDG On 04/01/2014 11:13 PM, Oliver Kowalke wrote:
2014-04-02 0:20 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:
I think I'd prefer it if Boost.Context just gave an error if I use a toolset that it doesn't support, instead of trying (and failing badly) to switch silently to a different toolset.
it does not switch to another toolset - boost.context contains assembler code only and the convention is:
- on i386-Windows its uses MASM - on x86_64-Windows its uses MASM64 - on arm-Windows armasm - on all other platforms GNU as
How is this not switching to a different toolset?
On Windows MASM is used to compile boost.context - it was shipped with MS Visual Studio an at some point MS decided to move it from the MS Visual Studio to the MS WDK.
It's still installed along with Visual Studio.
masm64 too?
masm used to be a standalone tool, but was integrated into visual studio (in vc6, I think). The current version is no longer distributed separately, although it is also included in a few other packages as well (including the WDK). I've seen claims on the internet that Microsoft stopped shipping masm with Visual Studio, but AFAICT, it simply isn't true. It's definitely included in VC Express (I checked all the versions I have installed, 9.0, 10.0, 11.0, and 12.0), and I can't believe that Microsoft would leave it out of the paid versions, if they include it in the free version.
- With <toolset>msvc, the assembler can't be found automatically, even though Boost.Build already knows how to find it.
hmm - it works for me - 'b2 toolset=msvc-10.0' builds boost.context on Win32
Meaning that you've added masm to your PATH.
- If the assembler has a name other then as, it cannot be used, even if it would actually work. This is especially likely to be a problem for cross compiling.
what would be a solution? If I would let the toolset choose the assembler tool I would be forced to support the different syntax of those assembler tools (which would be a lot if you know how many assemblers are available).
I think you're vastly exaggerating the problem. You don't need to support every assembler in existence. Most of the assemblers out there are standalone tools that have nothing to do with the toolset, and are thus completely irrelevant. msvc, gcc, clang, intel, and darwin, should all be okay, and these are by far the most common and the only toolsets that are run in the regression matrix. tasm is obsolete, but claims to be compatible with masm. I couldn't find any information on xlc, sun, pathscale, or pgi. The only real problem that I'm aware of is MinGW, which you should support directly anyway. See https://svn.boost.org/trac/boost/ticket/7262 which (a) describes a concrete use case where your proposed solution of using masm cannot possibly work, and (b) includes a patch. In Christ, Steven Watanabe
2014-04-08 4:13 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:
it does not switch to another toolset - boost.context contains assembler code only and the convention is:
- on i386-Windows its uses MASM - on x86_64-Windows its uses MASM64 - on arm-Windows armasm - on all other platforms GNU as
How is this not switching to a different toolset?
related to combination of architectue/ABI/address-model/binary-format - only one toolset
masm64 too?
masm used to be a standalone tool, but was integrated into visual studio (in vc6, I think).
for MSVC6 32bit masm wash shipped with
The current version is no longer distributed separately, although it is also included in a few other packages as well (including the WDK).
yes
I've seen claims on the internet that Microsoft stopped shipping masm with Visual Studio
that was my information too
It's definitely included in VC Express (I checked all the versions I have installed, 9.0, 10.0, 11.0, and 12.0), and I can't believe that Microsoft would leave it out of the paid versions, if they include it in the free version.
I've to rely on the information I get from other developers
- With <toolset>msvc, the assembler can't be found automatically, even though Boost.Build already knows how to find it.
hmm - it works for me - 'b2 toolset=msvc-10.0' builds boost.context on Win32
Meaning that you've added masm to your PATH.
I compile boost on Windows+MSVC always inside the MSVC-cmd prompt
what would be a solution? If I would let the toolset choose the assembler tool I would be forced to support the different syntax of those assembler tools (which would be a lot if you know how many assemblers are available).
I think you're vastly exaggerating the problem. You don't need to support every assembler in existence. Most of the assemblers out there are standalone tools that have nothing to do with the toolset, and are thus completely irrelevant. msvc, gcc, clang, intel, and darwin, should all be okay, and these are by far the most common and the only toolsets that are run in the regression matrix. tasm is obsolete, but claims to be compatible with masm. I couldn't find any information on xlc, sun, pathscale, or pgi.
and what is the conclusion regarding to xlc, sun, pathscale etc.?
The only real problem that I'm aware of is MinGW, which you should support directly anyway. See https://svn.boost.org/trac/boost/ticket/7262 which (a) describes a concrete use case where your proposed solution of using masm cannot possibly work, and (b) includes a patch.
that's the problem! - MinGW's assembler tool does have a different syntax than MASM. The defacto standard on Windows is MASM - why not use the standard tool on the platform to compile the object files (boost.context consists of assembler only). why must be the different assembler syntax' of the different assembler tools be supported?
----- Original Message -----
From: "Oliver Kowalke" <oliver.kowalke@gmail.com>
The only real problem that I'm aware of is MinGW, which you should support directly anyway. See https://svn.boost.org/trac/boost/ticket/7262 which (a) describes a concrete use case where your proposed solution of using masm cannot possibly work, and (b) includes a patch.
that's the problem! - MinGW's assembler tool does have a different syntax than MASM. The defacto standard on Windows is MASM - why not use the standard tool on the platform to compile the object files (boost.context consists of assembler only). why must be the different assembler syntax' of the different assembler tools be supported?
I am no expert when it comes to assembler variations, but I'd rather not have to have a dependency on MS tools. VC++ et al being the "standard tools on the platform" is part of the problem, not something to be encouraged. Regards, Paul Mensonides
2014-04-08 8:36 GMT+02:00 <pmenso57@comcast.net>:
I am no expert when it comes to assembler variations, but I'd rather not have to have a dependency on MS tools. VC++ et al being the "standard tools on the platform" is part of the problem, not something to be encouraged.
would burden to provide an implementation for each toolset out there (with its own assembler syntax) on my shoulders - and I can imagine that I'll get emails like "... my toolset on this platform with this OS is very important - why are you so lazy not to provide an implementation for it? you must do it immediately ..." - even if I don't have the architecture/OS at home nore documenation or any information at my hand.
----- Original Message -----
From: "Oliver Kowalke" <oliver.kowalke@gmail.com>
would burden to provide an implementation for each toolset out there (with its own assembler syntax) on my shoulders - and I can imagine that I'll get emails like "... my toolset on this platform with this OS is very important - why are you so lazy not to provide an implementation for it? you must do it immediately ..." - even if I don't have the architecture/OS at home nore documenation or any information at my hand.
That is exactly what the rest of Boost has to do with C++ compilers. Workarounds and alternate implementations abound. I am fully aware of the burden this places on the typical Boost developer. Personally, I would prefer "unsupported" over fallback to a different toolset (or assumption of said toolset). Regards, Paul Mensonides
2014-04-08 9:31 GMT+02:00 <pmenso57@comcast.net>:
That is exactly what the rest of Boost has to do with C++ compilers. Workarounds and alternate implementations abound. I am fully aware of the burden this places on the typical Boost developer. Personally, I would prefer "unsupported" over fallback to a different toolset (or assumption of said toolset).
it is not exactly as with C++ - the C++ syntax is standardized and you have to fix only the code on some small place for some C++ compilers. with assembler I've to rewrite the complete code - for instance how values are stored in a register varies from toolset to toolset, char defining a comment varies etc.
On Tue, Apr 8, 2014 at 12:18 PM, Oliver Kowalke <oliver.kowalke@gmail.com> wrote:
2014-04-08 9:31 GMT+02:00 <pmenso57@comcast.net>:
That is exactly what the rest of Boost has to do with C++ compilers. Workarounds and alternate implementations abound. I am fully aware of the burden this places on the typical Boost developer. Personally, I would prefer "unsupported" over fallback to a different toolset (or assumption of said toolset).
it is not exactly as with C++ - the C++ syntax is standardized and you have to fix only the code on some small place for some C++ compilers. with assembler I've to rewrite the complete code - for instance how values are stored in a register varies from toolset to toolset, char defining a comment varies etc.
I wonder if it's possible to translate between masm and gas syntaxes on the fly. I know gas understands Intel notation for x86 instructions but I'm not sure if that extends to other aspects of the syntax. Also, as a side note, many opensource projects with asm code limit themselves with just one assembler (I'd say yasm is one of the most popular ones). This isn't really a problem as long as the required tool is available and free for all target platforms. I think it is ok if Boost.Context limits the set of supported assemblers, although I'm not sure that supporting only masm on Windows is a good idea. I'd prefer an open alternative, even if it is not the standard one shipped with VS. Otherwise Boost.Context becomes vendor-locked on that particular platform. yasm is really good in this respect - it is widely available, lightweight and opensource. Disclaimer: I'm not a user of Boost.Context.
2014-04-08 11:27 GMT+02:00 Andrey Semashev <andrey.semashev@gmail.com>:
Also, as a side note, many opensource projects with asm code limit themselves with just one assembler (I'd say yasm is one of the most popular ones). This isn't really a problem as long as the required tool is available and free for all target platforms. I think it is ok if Boost.Context limits the set of supported assemblers, although I'm not sure that supporting only masm on Windows is a good idea. I'd prefer an open alternative, even if it is not the standard one shipped with VS. Otherwise Boost.Context becomes vendor-locked on that particular platform. yasm is really good in this respect - it is widely available, lightweight and opensource.
I'm not against to support GNU as on Windows - but I don't want to be forced to support the assembler tools coming along with all the toolsets (sun, xlc, pathscale, pgi, ...) on the variation of architecture, ABI, binary-format etc. In the future I might provide an implementation for GNU as on Windows but at the moment I have not the time.
On 08/04/14 11:40, Oliver Kowalke wrote:
2014-04-08 11:27 GMT+02:00 Andrey Semashev <andrey.semashev@gmail.com>:
Also, as a side note, many opensource projects with asm code limit themselves with just one assembler (I'd say yasm is one of the most popular ones). This isn't really a problem as long as the required tool is available and free for all target platforms. I think it is ok if Boost.Context limits the set of supported assemblers, although I'm not sure that supporting only masm on Windows is a good idea. I'd prefer an open alternative, even if it is not the standard one shipped with VS. Otherwise Boost.Context becomes vendor-locked on that particular platform. yasm is really good in this respect - it is widely available, lightweight and opensource.
I'm not against to support GNU as on Windows - but I don't want to be forced to support the assembler tools coming along with all the toolsets (sun, xlc, pathscale, pgi, ...) on the variation of architecture, ABI, binary-format etc.
In the future I might provide an implementation for GNU as on Windows but at the moment I have not the time.
So for x86 we have: Linux/Mac 32-bit GAS Linux/Mac 64-bit GAS Windows 32-bit GAS Windows 32-bit MASM Windows 64-bit GAS Windows 64-bit MASM That's 6 versions of the code. Possibly Windows GAS and Linux/Mac GAS can be the same in certain situations, further reducing the number of combinations. Is 6 really much worse than 4 versions?
On Tue, Apr 8, 2014 at 11:16 AM, Oliver Kowalke <oliver.kowalke@gmail.com> wrote:
2014-04-08 8:36 GMT+02:00 <pmenso57@comcast.net>:
I am no expert when it comes to assembler variations, but I'd rather not have to have a dependency on MS tools. VC++ et al being the "standard tools on the platform" is part of the problem, not something to be encouraged.
would burden to provide an implementation for each toolset out there (with its own assembler syntax) on my shoulders - and I can imagine that I'll get emails like "... my toolset on this platform with this OS is very important - why are you so lazy not to provide an implementation for it? you must do it immediately ..." - even if I don't have the architecture/OS at home nore documenation or any information at my hand.
I think that you'd be in your rights to request a patch to support that platform. This is opensource, you don't owe anything to anyone.
AMDG On 04/08/2014 02:32 AM, Andrey Semashev wrote:
I think that you'd be in your rights to request a patch to support that platform. This is opensource, you don't owe anything to anyone.
+1. There is a patch attached to http://svn.boost.org/trac/boost/ticket/7262, but it no longer works, as the interface of Boost.Context has changed since it was written. In Christ, Steven Watanabe
Hi Oliver, Am Dienstag, 8. April 2014, 06:36:19 schrieb pmenso57@comcast.net:
From: "Oliver Kowalke" <oliver.kowalke@gmail.com>
The only real problem that I'm aware of is MinGW, which you should support directly anyway. See https://svn.boost.org/trac/boost/ticket/7262 which (a) describes a concrete use case where your proposed solution of using masm cannot possibly work, and (b) includes a patch.
that's the problem! - MinGW's assembler tool does have a different syntax than MASM. The defacto standard on Windows is MASM - why not use the standard tool on the platform to compile the object files (boost.context consists of assembler only).
The central point is "on the platform". Requiring MASM does not work when cross-compiling (mostly form Linux or some other *nix). And this is something which is demanded by more and more users.
why must be the different assembler syntax' of the different assembler tools be supported?
I am no expert when it comes to assembler variations, but I'd rather not have to have a dependency on MS tools. VC++ et al being the "standard tools on the platform" is part of the problem, not something to be encouraged.
Yes, a lot of people are trying to switch from vc++ to gcc/clang for a lot of reasons. This affects not only Boost, but every popular cross platform framework. Maybe you should look at "support" and "maintenance" from another point of view. I think it is a sure bet that the question "why can't I use gnu assembler for context and why can't I cross-compile from my favourite platform using gcc/clang" will crop up every few months. See the aforementioned https://svn.boost.org/trac/boost/ticket/7262 and the mailing archives. This is at least the 4. or 5. discussion about supporting gnu assembler on windows and won't be the last. What about simply accepting the fact that this is what more and more users *need* and *want* and provide at least the feature most demanded? This would save everyone a lot of timeand effort in the long run. I can understand you can't provide corde for assembler variant out there, but supporting the two most important ones on Windows, including one free implementation, will make a lot of people happy. And I really doubt there are gnu incompatible *nix assemblers out there any more. And people using obsolete compilers and tools have hard times ahead for a lot of other reasons... And no, I don't use context, I'm just tired of ignoring build errors when using mingw or not running vcvarsall.bat. And ignoring discussions wasting everyones time. And don't underestimate the community. As soon as some working code is available, patches tend be committed. Been there, seen that :-) Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
Am 08.04.2014 08:27 schrieb Oliver Kowalke:
2014-04-08 4:13 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:
It's definitely included in VC Express (I checked all the versions I have installed, 9.0, 10.0, 11.0, and 12.0), and I can't believe that Microsoft would leave it out of the paid versions, if they include it in the free version.
I've to rely on the information I get from other developers
I can confirm that Visual Studio Professional 10, 11 and 12 come with ml.exe and ml64.exe at the locations pointed out by Steven. Ciao Dani
AMDG On 04/01/2014 02:00 PM, Oliver Kowalke wrote:
I've to restrict the assembler implementation for the combination of architecture/address-model/binary-format/ABI to one assembler tool - otherwise I would be forced to provide asm-files for the assembler tools available (masm, nasm, flat asm, ...) - impossible.
<snip>
Okay. Here's what I can do: - define new types ASM_GAS, ans ASM_MASM which inherit from ASM. (It's also possible to switch on a feature, but I think different types is a more correct abstraction.) - define two new tools gas and masm which can be initialized like this and which override the default assembler for the corresponding types using gas : [command] ; using masm : [command] ; - in your Jamfile, specify the types of the asm files with [ cast make_i386_ms_pe_masm asm-masm : asm/make_i386_ms_pe_masm.asm ] At this point: - If the assembler provided by the toolset can handle the format, then everything should work. (As is the case now.) - If not, then it can be fixed by adding a line to user-config.jam or project-config.jam. It would also be possible for you to try to auto-configure the assembler, but I'm not sure that's a good idea, since it overrides the toolset which is supposed to be the primary means of compiling. In Christ, Steven Watanabe
2014-04-02 4:01 GMT+02:00 Steven Watanabe <watanabesj@gmail.com>:
Okay. Here's what I can do:
- define new types ASM_GAS, ans ASM_MASM which inherit from ASM. (It's also possible to switch on a feature, but I think different types is a more correct abstraction.) - define two new tools gas and masm which can be initialized like this and which override the default assembler for the corresponding types
using gas : [command] ; using masm : [command] ;
- in your Jamfile, specify the types of the asm files with [ cast make_i386_ms_pe_masm asm-masm : asm/make_i386_ms_pe_masm.asm ]
At this point: - If the assembler provided by the toolset can handle the format, then everything should work. (As is the case now.) - If not, then it can be fixed by adding a line to user-config.jam or project-config.jam.
It would also be possible for you to try to auto-configure the assembler, but I'm not sure that's a good idea, since it overrides the toolset which is supposed to be the primary means of compiling.
OK - thx, I'll try to add your suggestion (don't know if it will be finished till the 1.56 release).
participants (8)
-
Andrey Semashev -
Daniela Engert -
Joshua Scholar -
Jürgen Hunold -
Mathias Gaunard -
Oliver Kowalke -
pmenso57＠comcast.net -
Steven Watanabe