On 08/30/2006 04:27 PM, Paul Mensonides wrote:

...

-----Original Message----- [snip]

I see that you updated your library! A couple of quick notes...

1) Because the macro names are uppercased (annoying, but necessary), [snip]

2) The #error directive on line 18 has an unmatched double-quote. [snip]

Thanks, Paul. I've updated the vault with the hopefully corrected code.

Regarding what the generated code is doing (as opposed to what the generator code is doing), the major problem with forwarding without language support is that it is a combinatorial problem--rather than linear. I.e. you really need various combinations of const and-or [snip]

Ouch! Do you know of anyone working on a compiler to solve this forwarding problem?

In any case, it is still reasonable up to 5 parameters if you're ignoring volatile. I know that Dave implemented this kind of forwarding for an operator new workalike that returned a smart pointer. (With perfect forwarding, you can design a reference-counted smart pointer so that it doesn't require a double allocation.)

I assume you mean one which doesn't use an intrusive refcount, IOW, uses a "detached" refcount like the existing shared_ptr? IOW, with the current shared_ptr, an one allocation is for the pointed-to object, the other is for the refcount. If so, then this is what I was trying to do with: http://tinyurl.com/obnls However, instead of using: template < class T0 , class T1 , ... , class Tn > ctor_this_type ( T0 a0 , T1 a1 , ... , Tn an ) : ctor_base_type ( a0 , a1 , ... , an ) {} the workaround implemented with the help of managed_ptr_ctor_forwarder.hpp: http://tinyurl.com/o7zlg generates CTOR's of the form: template < class VecOfTypes > ctor_this_type ( VecOfTypes const& , mpl::at_c<VecOfTypes,0>::type a0 , mpl::at_c<VecOfTypes,1>::type a1 , ... , mpl::at_c<VecOfTypes,n>::type an ) : ctor_base_type ( a0 , a1 , ... , an ) {} IOW, the client has to specify the "signature" of the ctor_base_type CTOR via a single VecOfTypes template arg to the ctor_this_type CTOR. The justification for this was that the client would have to know which ctor_base_type CTOR he wanted anyway, so it would not be that much of a burden for him to name it, indirectly, by supplying the "signature". I've uploaded prototype code demonstrating the idea to the same vault directory, in file typelist_fwding.cpp. -regards, Larry

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Paul Mensonides

Revisions inline...

You need a forwarding contructor on the 'data' structure to directly initialize the 'value' member. You also need a forwarding function on whatever takes the arguments from clients--so the resulting syntax looks something like:

Actually, you only need combinatorial forwarding on the function that takes arguments from the client. You don't need it elsewhere because then you don't have temporaries.

smart_ptr<X> p = make_ptr<X>(a, b, 123);

The make_ptr<X> function has to construct a 'smart_ptr<X>::data' object and initialize a smart_ptr with it and return that smart_ptr.

One problem of forwarding is arity (which can be more-or-less solved with code generation). The more stubborn problem with forwarding is temporaries. They cannot be bound to a non-const reference, and the template mechanism cannot deduce T as const U in a parameter of type T&. OTOH, non-temporaries don't have any problem deducing T as U or const U in a parameter of type T&. Unfortunately, the only way to handle it transparently is to overload via cv-qualifier--which leads the the combinatorial explosion when multiple parameters are involved.

Given the above, you could solve (i.e. avoid) the combinatorial problem by linearizing the input. If you have no temporaries, you can just use T& and T will bind correctly to a cv-qualified type. To get rid of temporaries (which can't bind to T&) you need: template<class T> inline T& identity(T& x) { return x; } template<class T> inline const T& identity(const T& x) { return x; } template<class T> inline volatile T& identity(volatile T& x) { return x; } template<class T> inline const volatile T& identity(const volatile T& x) { return x; } What was called before as f(a, b, c); can be called as f(identity(a), identity(b), identity(c)); ...which is relatively easily generated via the preprocessor--especially with variadic macros: F(a, b, c) -> f(identity(a), identity(b), identity(c)) Without variadic macros, you'd have to specify the number of arguments or do something like: F((a)(b)(c)) Regards, Paul Mensonides

Larry Evans wrote:

On 08/31/2006 04:05 PM, Paul Mensonides wrote:

...

Given the above, you could solve (i.e. avoid) the combinatorial problem by linearizing the input.

Paul, could explain what's meant by "linearizing the input"? I'm unfamiliar with that phrase :(

Bad wording on my part. What I meant was converting the problem from combinatorial to linear complexity (in relation to the number of required template functions) by modifying the input in such a way that the combinatorial problem is avoided. The rules that we have now for overloading and template argument deduction disallow the following: template<class T> void f(T&); int main(void) { f(123); return 0; } The problem here is that the T cannot be deduced as "const T" which would allow the binding of a temporary to a reference. However, if the argument is *already* a reference to const--instead of a temporary--then the T will be bound to (e.g.) const int. For example: int main(void) { f((const int&)123); return 0; } This one isn't a problem. The T will bind to const int and we avoid the temporary problem. What this means is that we can make the problem linear by restricting the input. We don't need every possible combination of cv-qualifiers on various parameters, such as (ignoring volatile): template<class T0, class T1> void f(T0&, T1&); template<class T0, class T1> void f(T0&, const T1&); template<class T0, class T1> void f(const T0&, T1&); template<class T0, class T1> void f(const T0&, const T1&); Instead, we only need template<class T0, class T1> void f(T0&, T1&); iff the arguments are guaranteed not to be temporaries--namely because the T0 and T1 in the parameter types will be deduced as cv-qualified as necessary. Given the above, we need a way to cause each argument to preserve it's cv-qualification, yet have temporaries be "promoted" to reference to const. We can do that with overloading similar to what we were doing with the combinatorial overloads--but this time it isn't combinatorial because it is only with one argument at a time--it is linear, but along a different axis (namely cv-qualifications as opposed to arity). Simple way to accomplish that is with identity functions for every possible cv-qualification: template<class T> T& identity(T&); template<class T> const T& identity(const T&); template<class T> volatile T& identity(volatile T&); template<class T> const volatile T& identity(const volatile T&); If the identity function is applied to a temporary, it has the same effect as the c-style cast used above: int main(void) { f(identity(123)); return 0; } The advantages of using something like identity are that we don't have to manually try to deduce the type in order to do the cast and that every other kind of argument passes through preserving cv-qualification. Ultimately then, we can avoid the combinatorial forwarding problem by simply applying identity to each argument before the argument gets the our function(s). E.g. template<class T0, class T1> void g(T0&, T1&); int main(void) { int x = 0; const int y = 0; g(identity(x), identity(y)); g(identity(123), identity(x)); // etc. return 0; } Though this works, it is obviously not a complete solution because it is flagrantly intrusive on the client code. However, the problem is solved successfully avoided if we can find a way to automatically add the identity calls to each argument. That can be done with a macro interface instead of a function. The macro interface won't be perfect. You have the usual all-caps library-prefixed macro name, and without variadic macros (C99 + C++0x) you have to specify either the number of arguments or use a different invocation syntax. E.g. G(2)(x, y) G(2)(123, x) -or- G((x)(y)) G((123)(x)) With variadics, you can make it... G(x, y) G(123, x) ...and the interface ultimately becomes something like: smart_ptr<T>::MAKE_SMART_PTR(x, y) smart_ptr<T>::MAKE_SMART_PTR(123, x) I don't know if that is better than just dealing with the combinatorial problem (for a low maximum arity) or not. However, it is better, IMO, than having to actually specify the signature at the call site.

I'm afraid I don't understand. Could you provide some code, maybe for just 2 to 3 parameters, which is like the ctor_template_test.cpp code in the vault. IOW, show:

Or could you provide some other code or sketch of code to make it a little clearer?

Say you have a smart_ptr implemented something like: template<class T> class smart_ptr { private: struct data { data(void) : rc(1), x() { } template<class T0> data(T0& p0) : rc(1), x(p0) { } template<class T0, class T1> data(T0& p0, T1& p1) : rc(1), x(p0, p1) { } template<class T0, class T1, class T2> data(T0& p0, T1& p1, T2& p2) : rc(1), x(p0, p1, p2) { } // etc. unsigned long rc; T x; } * p_; smart_ptr(data* p) : p_(p) { }; public: // ... static smart_ptr make(void) { return new data(); } template<class T0> static smart_ptr make(T0& p0) { return new data(p0); } template<class T0> static smart_ptr make(T0& p0, T1& p1) { return new data(p0, p1); } template<class T0> static smart_ptr make(T0& p0, T1& p1, T2& p2) { return new data(p0, p1, p2); } // etc. }; template<class T> inline T& identity(T& x) { return x; } template<class T> inline const T& identity(const T& x) { return x; } template<class T> inline volatile T& identity(volatile T& x) { return x; } template<class T> inline const volatile T& identity(const volatile T& x) { return x; } #define MAKE_SMART_PTR(n) BOOST_PP_CAT(MAKE_SMART_PTR_, n) #define MAKE_SMART_PTR_0() make() #define MAKE_SMART_PTR_1(a) make(identity(a)) #define MAKE_SMART_PTR_2(a, b) make(identity(a), identity(b)) #define MAKE_SMART_PTR_3(a, b, c) make(identity(a), identity(b), identity(c)) // etc. Usage becomes: smart_ptr<XYZ> p = smart_ptr<XYZ>::MAKE_SMART_PTR(x, y, z); Another way that you could do it is will chained invocations that each take one parameter: smart_ptr<XYZ> p = smart_ptr<XYZ>::make(x)(y)(z); This last is doable, but would be less efficient. Of course, the real solution is language support. Rvalue references, AFAICT, take the problem from a combinatorial function of arity and cv-qualification to just a linear function of arity. Variadic templates (if specified in the right way) + rvalue references take the problem from a linear function of arity to a constant function--that returns two (nullary and non-nullary). Regards, Paul Mensonides

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Larry Evans

The 01.09.2006 04:12 ctor_template.zip file in the vault is my attempt at doing what you've outlined above. BTW, instead of something like the G macro, it uses:

g( BOOST_FWD_SEQ_FOR_EACH_IDENTITY ( (x) (y) ) )

Please let me know if you see any errors.

You can do it a little more efficiently regarding the comma handling. E.g. #define SEQ_FOR_EACH_IDENTITY(seq) \ BOOST_PP_SEQ_FOR_EACH_I(APPLY_IDENTITY, ~, seq) \ /**/ #define APPLY_IDENTITY(r, _, i, x) \ BOOST_PP_COMMA_IF(i) boost::forwarder::identity(x) \ /**/ You could also let the pp-lib do the work: #define APPLY_IDENTITY(s, _, x) boost::forwarder::identity(x) #define SEQ_FOR_EACH_IDENTITY(seq) \ BOOST_PP_SEQ_ENUM( \ BOOST_PP_SEQ_TRANSFORM(APPLY_IDENTITY, ~, seq) \ ) \ /**/ Either way, the speed difference between the above two is neglible for the number of arguments we're talking about (I tested up to 50).

The newest zip also includes a Makefile which I've found useful in seeing what's produced by cpp. I was wondering how you've gotten bjam to do something similar to that Makefile. If so, could you post the code or a link?

I don't use bjam for this. I run the compiler (or a set of compilers) in preprocess-only mode and look at the output. I make small proof-of-concept scenarios (regular code) in order to establish what I want to generate. Once I have decided that, then I write the generator and debug any errors I might have made. At this point, I'm specifically testing the generator--not the generated code. Once I'm satisfied that the generator is generating what I want, I then go on to full compiles and debugging the generated code. When I'm writing and testing the generator, I either do it outside of a normal "heavyweight" translation unit, or I comment out header inclusions that are not necessary for the generator itself--namely so I don't have to wade through enormous amounts of output to find my generated code. E.g. say I'm generating something that uses std::cout, when I'm writing and testing the generator, I'll comment out the inclusion of <iostream> because the generator itself doesn't need it to function. On most compilers, the preprocess-only command line switch is -E. Several compilers also support a -P command line switch that suppresses #line directives in the output and generally compacts vertical space. Regards, Paul Mensonides

On 08/31/2006 03:07 PM, Paul Mensonides wrote:

...

-----Original Message----- [snip]

...

smart pointer so that it doesn't require a double allocation.)

I assume you mean one which doesn't use an intrusive refcount, IOW, uses a "detached" refcount like the existing shared_ptr? IOW, with the current shared_ptr, an one allocation is for the pointed-to object, the other is for the refcount.

Yes, basically you have the smart pointer allocate the pointed-to-object itself as a data member of a struct that also contains the reference count. E.g.

template<class T> class smart_ptr { // ... private: struct data { unsigned long rc; T value; } * p_; smart_ptr(data* p); };

You need a forwarding contructor on the 'data' structure to directly initialize the 'value' member. You also need a forwarding function on whatever takes the arguments from clients--so the resulting syntax looks something like:

smart_ptr<X> p = make_ptr<X>(a, b, 123);

The make_ptr<X> function has to construct a 'smart_ptr<X>::data' object and initialize a smart_ptr with it and return that smart_ptr.

The above make_ptr<X> is close to the managed_ptr:: auto_overhead<X,ulong_rc> where: template<class> struct ulong_rc { unsigned long rc; }; and instead of: smart_ptr(data* p) there would only be: smart_ptr(auto_overhead<T,ulong_rc>& a_auto) IOW, auto_overhead is simply derived from auto_ptr with the pointee being an instance of the managed_ptr:: overhead_referent_vals<ulong_rc,T>. IOW, overhead_referent_vals<ulong_rc,T> corresponds to your smart_ptr<T>::data. The reason for the elaborate overhead_referent_* classes is to allow different smart_ptr policies, e.g. to allow strong and weak pointers. The reason for disallowing: smart_ptr(overhead_referent_vals<ulong_rc,T>*) is to never expose a raw pointer to the client.

One problem of forwarding is arity (which can be more-or-less solved with code generation). The more stubborn problem with forwarding is temporaries. They cannot be bound to a non-const reference, and the template mechanism cannot deduce T as const U in a parameter of type T&. OTOH, non-temporaries don't have any problem deducing T as U or const U in a parameter of type T&.

Yeah, I had that problem which is why the a_auto above is not 'const &'. I remember some attempt at a workaround, but can't remember the details or even if it worked.

Unfortunately, the only way to handle it transparently is to overload via cv-qualifier--which leads the the combinatorial explosion when multiple parameters are involved.

:(

...

the workaround implemented with the help of managed_ptr_ctor_forwarder.hpp:

http://tinyurl.com/o7zlg

generates CTOR's of the form:

template < class VecOfTypes

...

ctor_this_type ( VecOfTypes const& , mpl::at_c<VecOfTypes,0>::type a0 , mpl::at_c<VecOfTypes,1>::type a1 , ... , mpl::at_c<VecOfTypes,n>::type an ) : ctor_base_type ( a0 , a1 , ... , an ) {}

IOW, the client has to specify the "signature" of the ctor_base_type CTOR via a single VecOfTypes template arg to the ctor_this_type CTOR. The justification for this was that the client would have to know which ctor_base_type CTOR he wanted anyway, so it would not be that much of a burden for him to name it, indirectly, by supplying the "signature".

I'm not sure why you need to add this into each class (rather than just to the smart-pointer-related interfaces).

Sorry I was unclear. This above is only needed for the auto_overhead class. IOW, replace ctor_this_type with auto_overhead and ctor_base_type with Referent from auto_overhead<Referent,Overhead>. Of course it's a little more complicated, but that's essentially what's happening with the MANAGED_PTR_CTOR_FORWARDER_OVERLOAD macro invokation in the middle of the auto_overhead class definition.

It may be that we're not talking about the same thing, but the smart_ptr facilities above could be designed without *any* intrusive requirements on the type pointed-to

Likewise, auto_overhead does not have *any* intrusive requirements since the only allocation is for overhead_referent_vals<ulong_rc,pointee_type>.

and without *any* manual listing of argument types by the client. Granted, you can only support a fairly low number of parameters (larger if you ignore volatile).

And this, AFAICT, is the advantage of make_ptr vs. auto_overhead. IOW, auto_overhead does have to supply, as mentioned in my previous post, the signature (VecOfTypes in the above code) of the desired "target" CTOR.

On 08/30/2006 05:18 AM, Paul Mensonides wrote: [snip]

normal one), as it is *intended* to be included multiple times. Also, it is using what the pp-lib calls "named external arguments", which, for all intents and purposes are arguments to the "function call". The header should #undef those macros:

// boost/forwarder/ctor_template.hpp #if BOOST_PP_IS_ITERATING

// ----- point 1 ----- //

// "include guard" (to save time) #ifndef BOOST_FORWARD_CTOR_TEMPLATE_INCLUDED #define BOOST_FORWARD_CTOR_TEMPLATE_INCLUDED #include <boost/preprocessor/iterate.hpp> #include <boost/preprocessor/repetition/enum_binary_params.hpp> #include <boost/preprocessor/repetition/enum_params.hpp> #endif

Ah! This is where I went wrong. The ctor_template.hpp file was my modification of the SEGMENTED_FUSION_V2 file: http://tinyurl.com/zgxho which had the #include-guard guarding the complete then branch of the #if BOOST_PP_IS_ITERATING. Your above comment: // boost/forwarder/ctor_template.hpp makes clear the purpose of the #include-guard. I'd failed to understand the purpose of the #include-guard (somehow I'd figured it was needed for the boost-pp file-iteration to work). [snip]

After this, the client code looks like:

#include <boost/forwarder/ctor_template.hpp> #include <boost/preprocessor/repetition/enum_params.hpp>

// ...

#define BOOST_FORWARDER_CTOR_TEMPLATE_THIS_TYPE sub #define BOOST_FORWARDER_CTOR_TEMPLATE_BASE_TYPE(n) : Super(BOOST_PP_ENUM_PARAMS(n, _)) #include BOOST_FORWARDER_CTOR_TEMPLATE()

Great! I've followed your suggestions and the code works. The modified code was uploaded to the same place in the vault. I've also corrected the minor problems you noted in your first reply. [snip]

BOOST_PP_IS_ITERATING. What the file-iteration mechanism is basically doing is defining a for-loop mechanism that calls a user-defined function:

namespace boost { namespace preprocessor { bool is_iterating;

template<class F> void iterate(int a, int z, F func) { is_iterating = true; for (; a <= z; ++a) { func(a); } is_iterating = false; return; } }}

The "ctor_template.hpp" header (as it currently exists) is such a function, but its initial call (the one that starts the iteration) is done by the client, not the file-iteration mechanism. You have to write that file as if it is a function--not a header. But you should also provide a normal header that defines the interface to the function (i.e. BOOST_FORWARDER_CTOR_TEMPLATE()) and includes the files needed by the function.

Make sense?

Yes. Makes much better sense to me now. Thanks Paul :) -regards, Larry