The code below fails a compile-time assertion in boost 1.32.0 on MSVC 7.1. This is because, internally, boost::optional uses boost::alignment_of, which produces the number 12, which is then passed as a template parameter to boost::type_with_alignment, which won't accept 12. Not sure where the blame lies. Is it with boost::alignment_of, for producing 12 in the first place? Perhaps not, since 12 is a multiple of 4, and the doco says boost::alignment_of is allowed to produce any multiple of the actual alignment value. So is the fault with boost::type_with_alignment, for not accepting 12? Or is there something wrong with the app? #include <boost/optional.hpp> class D { virtual ~D(); }; class E : virtual public D { virtual ~E(); unsigned long long m_i; }; boost::optional<E> s_optE; Cheers, --- Michael Shepanski
Michael Shepanski <mshepanski@gmail.com> writes:
The code below fails a compile-time assertion in boost 1.32.0 on MSVC 7.1.
This is because, internally, boost::optional uses boost::alignment_of, which produces the number 12, which is then passed as a template parameter to boost::type_with_alignment, which won't accept 12. Not sure where the blame lies. Is it with boost::alignment_of, for producing 12 in the first place? Perhaps not, since 12 is a multiple of 4, and the doco says boost::alignment_of is allowed to produce any multiple of the actual alignment value. So is the fault with boost::type_with_alignment, for not accepting 12? Or is there something wrong with the app?
I think the problem is that type_with_alignment can't find a type that has an alignment of 12. Producing an error in that case is the right behavior. We could add the type whose alignment_of<> is 12 to the reperotoire of types being tested by type_with_alignment...
#include <boost/optional.hpp>
class D { virtual ~D(); };
class E : virtual public D { virtual ~E(); unsigned long long m_i; };
boost::optional<E> s_optE;
Cheers, --- Michael Shepanski _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- Dave Abrahams Boost Consulting www.boost-consulting.com
On 10/26/05, David Abrahams <dave@boost-consulting.com> wrote:
I think the problem is that type_with_alignment can't find a type that has an alignment of 12. Producing an error in that case is the right behavior. We could add the type whose alignment_of<> is 12 to the reperotoire of types being tested by type_with_alignment...
Don't we need to be sure that type_with_alignment<> can accept *any* number that alignment_of<> produces -- or, conversely, make sure that alignment_of<> only produces numbers that type_with_alignment<> can accept? (I'm assuming that we want boost::optional to work all the time, not just for my example.) --- Michael Shepanski
Michael Shepanski <mshepanski@gmail.com> writes:
On 10/26/05, David Abrahams <dave@boost-consulting.com> wrote:
I think the problem is that type_with_alignment can't find a type that has an alignment of 12. Producing an error in that case is the right behavior. We could add the type whose alignment_of<> is 12 to the reperotoire of types being tested by type_with_alignment...
Don't we need to be sure that type_with_alignment<> can accept *any* number that alignment_of<> produces -- or, conversely, make sure that alignment_of<> only produces numbers that type_with_alignment<> can accept? (I'm assuming that we want boost::optional to work all the time, not just for my example.)
Sure, that's what we want. Can you figure out *how*? That's the problem. -- Dave Abrahams Boost Consulting www.boost-consulting.com
On 10/27/05, David Abrahams <dave@boost-consulting.com> wrote:
Sure, that's what we want. Can you figure out *how*? That's the problem.
I can't think of a perfect solution, but here are some imperfect ones: 1. Inside type_traits/alignment_of.hpp, the expression: sizeof(alignment_of_hack<T>)-sizeof(T) is what generates the number 12 in the first place. I think the 12 comes about because this expression includes the size of padding both before and after the t field in alignment_of_hack<T>. If you replace this expression with: offsetof(alignment_of_hack<T>, t) then it seems to work better. The downside is that the standard says you can't use offsetof with non-PODs, and gcc gives warnings if you try. This may be because offsetof relies on "&" --in which case boost::addressof will save the day-- but there may be more fundamental reasons to do with unpredictable layout of non-PODs. 2. We could assume that the alignment of int64_t is good enough alignment for anything, and then set up type_with_alignment so that it gives you an int64_t by default, i.e. in the cases where it's currently failing. Downside: the assumption is probably not guaranteed by any C++ standard, so if you meet a platform where the assumption fails, then you'll need platform-specific code that sets a different default type. 3. boost::optional could put the data on the heap (since operator new always knows how to get properly aligned memory), and then boost::optional could manage it with a smart pointer. Downside: it defeats one of the attractions of boost::optional over smart pointers, which is that it saves the allocation and indirection overhead. 4. Maybe it's possible to write code that selects option 3 only for non-PODs, and otherwise either uses offsetof (i.e. option 1) or does what it does now. Downside: there's still the downside of 3, when it's selected. Also I'm not sure whether it's possible to do the template metaprogramming to choose one implementation for PODs and another for non-PODs. --- Michael Shepanski
Michael Shepanski wrote:
On 10/27/05, David Abrahams <dave@boost-consulting.com> wrote:
Sure, that's what we want. Can you figure out *how*? That's the problem.
I can't think of a perfect solution, but here are some imperfect ones:
1. Inside type_traits/alignment_of.hpp, the expression: sizeof(alignment_of_hack<T>)-sizeof(T) is what generates the number 12 in the first place. I think the 12 comes about because this expression includes the size of padding both before and after the t field in alignment_of_hack<T>. If you replace this expression with: offsetof(alignment_of_hack<T>, t) then it seems to work better. The downside is that the standard says you can't use offsetof with non-PODs, and gcc gives warnings if you try. This may be because offsetof relies on "&" --in which case boost::addressof will save the day-- but there may be more fundamental reasons to do with unpredictable layout of non-PODs.
Well, you said it yourself... we can't use this solution becasue it won't work for any T.
2. We could assume that the alignment of int64_t is good enough alignment for anything, and then set up type_with_alignment so that it gives you an int64_t by default, i.e. in the cases where it's currently failing. Downside: the assumption is probably not guaranteed by any C++ standard, so if you meet a platform where the assumption fails, then you'll need platform-specific code that sets a different default type.
Again, as you said, that assumption is not a std requirement so we can't use it.
3. boost::optional could put the data on the heap (since operator new always knows how to get properly aligned memory), and then boost::optional could manage it with a smart pointer. Downside: it defeats one of the attractions of boost::optional over smart pointers, which is that it saves the allocation and indirection overhead.
Is more than an attraction, it's a fundamental feature. I can't change that. FWIW, there is sound (IMO at least) alignment proposal for the std, so we may not need to worry anymore about this in the not so distant future. -- Fernando Cacciola SciSoft http://fcacciola.50webs.com/
"Fernando Cacciola" <fernando_cacciola@hotmail.com> writes:
Michael Shepanski wrote:
On 10/27/05, David Abrahams <dave@boost-consulting.com> wrote:
FWIW, there is sound (IMO at least) alignment proposal for the std, so we may not need to worry anymore about this in the not so distant future.
I don't think that proposal gives us any more guarantees than we have today. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
"Fernando Cacciola" <fernando_cacciola@hotmail.com> writes:
Michael Shepanski wrote:
On 10/27/05, David Abrahams <dave@boost-consulting.com> wrote:
FWIW, there is sound (IMO at least) alignment proposal for the std, so we may not need to worry anymore about this in the not so distant future.
I don't think that proposal gives us any more guarantees than we have today.
Oh.. I was refering to n1546.. Do you as well? I interpreted that the idea there is to be able to spell: char align_as<T> storage[sizeof(T)]; And let the compiler do the magic. Isn't that all we need? Or is that the proposal was rejected? Note for the curious: This article by Tom Lokovic, although totally unrelated to boost AFAICS, explains how boost::alignment_of<> works (because he proposes the same technique) http://www.monkeyspeak.com/alignment/ -- Fernando Cacciola SciSoft http://fcacciola.50webs.com/
"Fernando Cacciola" <fernando_cacciola@hotmail.com> writes:
David Abrahams wrote:
"Fernando Cacciola" <fernando_cacciola@hotmail.com> writes:
Michael Shepanski wrote:
On 10/27/05, David Abrahams <dave@boost-consulting.com> wrote:
FWIW, there is sound (IMO at least) alignment proposal for the std, so we may not need to worry anymore about this in the not so distant future.
I don't think that proposal gives us any more guarantees than we have today.
Oh.. I was refering to n1546..
Oh, Attila's proposal?
Do you as well?
No, sorry, I was referring to the TR1 type traits spec.
I interpreted that the idea there is to be able to spell:
char align_as<T> storage[sizeof(T)];
And let the compiler do the magic.
Isn't that all we need? Or is that the proposal was rejected?
Many proposals don't make it without ever really being rejected. It's probably not -- as of yet -- accepted, though. -- Dave Abrahams Boost Consulting www.boost-consulting.com
Michael Shepanski wrote:
The code below fails a compile-time assertion in boost 1.32.0 on MSVC 7.1.
class D { virtual ~D(); };
class E : virtual public D { virtual ~E(); unsigned long long m_i; };
FYI: The following article by Tom Lokovic explains a technique for calculating the alignment of a data type. His technique is exactly the same used by boost::alignment_of<>, so we can consider the article an explanation of how alignment_of<> works. http://www.monkeyspeak.com/alignment/ Basically: ********** Explanation of alignment_of<> ********************** Appendix 1 of the article contains a proof of the following interesting claim: "For every type T, sizeof(T) is an integer multiple of T's stride S" (Tom calls "stride of T" to the "alignment requirements of T") Read the proof if you're not convinced. It basically says that C/C++ compilers chooses sizeof(T) as a multiple of the alignment of T because that allows you to arrange instances of T one right after another in an array without violating T's alignment requirements. And Appendix 2 contains a proof of: "For every struct type T, sizeof(T) is an integer multiple of the stride of each of T's members" It basically says that when the compiler creates an object containing both T and U, it must give it a size such that, if 2 instances of the T+U are placed one after another in an array, each member T or U must be itself properly aligned, and for that, the total size must be a multiple of each member's alignment. So, Say we have a type 'T' and we form a new type, Tchar, adding a 'char' to it. Now say we put 2 instances of Tchar into an array. However the char is added into Tchar, the T member of the second Tchar must be properly aligned w.r.t to T's requirement, so the compiler needs to arrange Tchar as to guarantee that. Furhtermore, since a char can be placed anywhere in memory, that is, it's alignment is 1, we know that the compiler has really just 2 choices: either it reuses the padding within T to store the 'char', in which case sizeof(T)==sizeof(Tchar), or it extends the storage needed by an amount equivalent to the alignment of T, or a multiple of that (it has to be like that to make sure each T member is properly placed in an array of Tchar) That is, sizeof(Tchar)-sizeof(T) must be a multiple of the alignment of T. boost::alignment_of<> returns either sizeof(T) or sizeof(Tchar)-sizeof(T), and both are required to be multiples of the alignment of T. It uses the following template to generate Tchar template <typename T> struct alignment_of_hack { char c; T t; alignment_of_hack(); }; ********** End of Explanation of alignment_of<> ********************** Now, I checked some facts with your E class in VC7.1 and the results are interesting: The sizeof(E) is 20 And according to the explanation of how to calculate the alignment of E, a struct adding a 'char' to E should have a size larger than 20 in an amount multiple of the alignment of T. struct Echar { E e ; char c; } ; struct charE { char c; E e ; } ; sizeof(Echar) -> 24 sizeof(charE) -> 32 As you can see, the true alignment of T is 4, while 12 is just 3 times that. As you know by now, the real problem here is that type_with_alignment<> cannot really take as argument "any" multiple of the alignment becasue it cannot match any multiple with its type list. As Martin Bonner suggested, one solution is to factorize the result of alignment_of<> so it returns not just any multiple. Unfortunately, we can't say that the alignment _ought_ to be a power of 2 can we? An alternative solution that, at least, would work with this very specific example, is to compute it as the minimun of sizeof(charT)-sizeof(T) and sizeof(Tchar)-sizeof(T) That is, adding the char both before and after, and taking the least difference. HTH -- Fernando Cacciola SciSoft http://fcacciola.50webs.com/
Fernando Cacciola wrote:
struct Echar { E e ; char c; } ;
struct charE { char c; E e ; } ;
sizeof(Echar) -> 24 sizeof(charE) -> 32
As you can see, the true alignment of T is 4, while 12 is just 3 times that.
The "preferred" alignment of E seems to be 8 (makes sense because of the long long), which is why the sizes are a multiple of 8. Why is sizeof(E) 20, however, I don't know. Alignments are tricky.
Peter Dimov wrote:
The "preferred" alignment of E seems to be 8 (makes sense because of the long long),
Just curious, how do tell that? -- Fernando Cacciola SciSoft http://fcacciola.50webs.com/
Peter Dimov wrote:
Fernando Cacciola wrote:
Peter Dimov wrote:
The "preferred" alignment of E seems to be 8 (makes sense because of the long long),
Just curious, how do tell that?
Because the compiler tends to place it on an offset within structs that is a multiple of 8?
Hmmm, if a char is appended before the E, it is offseted 4, not 8, according to my tests. -- Fernando Cacciola SciSoft http://fcacciola.50webs.com/
Hello again. I'd like to reawaken this thread because It didn't seem to reach a conclusion, and the bug that started it continues to cause me pain. On 10/29/05, Fernando Cacciola <fernando_cacciola@hotmail.com> wrote:
As Martin Bonner suggested, one solution is to factorize the result of alignment_of<> so it returns not just any multiple. Unfortunately, we can't say that the alignment _ought_ to be a power of 2 can we?
An alternative solution that, at least, would work with this very specific example, is to compute it as the minimun of sizeof(charT)-sizeof(T) and sizeof(Tchar)-sizeof(T) That is, adding the char both before and after, and taking the least difference.
These both sound like interesting solutions; what do others think? Are they theoretically correct? A policy question: what happens if no theoretically correct solution is found? Would boost::optional then be discontinued? Or would it be reimplemented in a way that happens to work on all popular platforms? Or would it switch at compile-time between overtly platform-specific implementations? I do hope that the policy is not to keep the current solution until a theoretically correct solution is comes to light. Not to put too fine a point on it, the current solution is theoretically incorrect too; the only thing that distinguishes it from other theoretically incorrect solutions that have been discussed is that this one fails in practice. --- Michael Shepanski
Michael Shepanski wrote:
Hello again. I'd like to reawaken this thread because It didn't seem to reach a conclusion, and the bug that started it continues to cause me pain.
On 10/29/05, Fernando Cacciola <fernando_cacciola@hotmail.com> wrote:
As Martin Bonner suggested, one solution is to factorize the result of alignment_of<> so it returns not just any multiple. Unfortunately, we can't say that the alignment _ought_ to be a power of 2 can we?
An alternative solution that, at least, would work with this very specific example, is to compute it as the minimun of sizeof(charT)-sizeof(T) and sizeof(Tchar)-sizeof(T) That is, adding the char both before and after, and taking the least difference.
These both sound like interesting solutions; what do others think? Are they theoretically correct?
A policy question: what happens if no theoretically correct solution is found? Would boost::optional then be discontinued? Or would it be reimplemented in a way that happens to work on all popular platforms? Or would it switch at compile-time between overtly platform-specific implementations?
I do hope that the policy is not to keep the current solution until a theoretically correct solution is comes to light. Not to put too fine a point on it, the current solution is theoretically incorrect too; the only thing that distinguishes it from other theoretically incorrect solutions that have been discussed is that this one fails in practice.
And FWIW my solution works in practice.. specifically in your case, which is the first and only case reported so far. So if I where you, I would just implement it into your version of the code.. (If you don't know how let me know and I'll do it) HTH -- Fernando Cacciola SciSoft http://fcacciola.50webs.com/
Michael Shepanski <mshepanski@gmail.com> writes:
Hello again. I'd like to reawaken this thread because It didn't seem to reach a conclusion, and the bug that started it continues to cause me pain.
On 10/29/05, Fernando Cacciola <fernando_cacciola@hotmail.com> wrote:
As Martin Bonner suggested, one solution is to factorize the result of alignment_of<> so it returns not just any multiple. Unfortunately, we can't say that the alignment _ought_ to be a power of 2 can we?
An alternative solution that, at least, would work with this very specific example, is to compute it as the minimun of sizeof(charT)-sizeof(T) and sizeof(Tchar)-sizeof(T) That is, adding the char both before and after, and taking the least difference.
These both sound like interesting solutions; what do others think? Are they theoretically correct?
The second one certainly won't work. It's very possible that sizeof(Tchar) - sizeof(T) is zero, while T may still have alignment requirements. The first one isn't theoretically correct, because an arbitrary machine might have non-power-of-2 alignments. However, it's very likely to work for on any machine we target in the forseeable future.
A policy question: what happens if no theoretically correct solution is found? Would boost::optional then be discontinued?
Of course not.
Or would it be reimplemented in a way that happens to work on all popular platforms?
That's what we do now. Except that someone found a case where it fails.
Or would it switch at compile-time between overtly platform-specific implementations?
How to address it is up to the maintainer.
I do hope that the policy is not to keep the current solution until a theoretically correct solution is comes to light.
I doubt it, because there is no such solution.
Not to put too fine a point on it, the current solution is theoretically incorrect too; the only thing that distinguishes it from other theoretically incorrect solutions that have been discussed is that this one fails in practice.
That's one way to look at it. The other way is to say that it refuses to silently return a wrong answer, preferring instead to cause a compile-time error so we will notice the problem and have this discussion about addressing it. I think that even if we take the biggest factor of 2 that evenly divides the detected alignment, it's still possible to come out with a number so large that none of the fundamental types we use to force alignment will handle it. I hope in that case we'll continue to produce a compile-time error. -- Dave Abrahams Boost Consulting www.boost-consulting.com
participants (4)
-
David Abrahams -
Fernando Cacciola -
Michael Shepanski -
Peter Dimov