[fusion] undocumented sequence requirement: convert
Say I have a valid fusion sequence. I want to use an mpl algorithm on it. I include boost/fusion/mpl.hpp and put a nested tag type in my sequence that is an alias for fusion::fusion_sequence_tag. So far, so good. Then I try to call mpl::pop_back on this sequence, and I find that, despite fulfilling all the documented requirements for Fusion ForwardSequence, the code fails to compile. That's because of this in boost/fusion/mpl/pop_back.hpp: template <> struct pop_back_impl<fusion::fusion_sequence_tag> { template <typename Sequence> struct apply { typedef typename fusion::result_of::pop_back<Sequence>::type result; typedef typename fusion::result_of::convert< typename fusion::detail::tag_of<Sequence>::type, result>::type type; }; }; My code doesn't compile because I haven't implemented the fusion::convert customization point. Where is that documented and what does it do? -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 22/08/2012 22:38, Eric Niebler wrote:
Say I have a valid fusion sequence. I want to use an mpl algorithm on it. I include boost/fusion/mpl.hpp and put a nested tag type in my sequence that is an alias for fusion::fusion_sequence_tag. So far, so good. Then I try to call mpl::pop_back on this sequence, and I find that, despite fulfilling all the documented requirements for Fusion ForwardSequence, the code fails to compile. That's because of this in boost/fusion/mpl/pop_back.hpp:
template <> struct pop_back_impl<fusion::fusion_sequence_tag> { template <typename Sequence> struct apply { typedef typename fusion::result_of::pop_back<Sequence>::type result;
typedef typename fusion::result_of::convert< typename fusion::detail::tag_of<Sequence>::type, result>::type type; }; };
My code doesn't compile because I haven't implemented the fusion::convert customization point. Where is that documented and what does it do?
As the name suggests, it converts a Fusion sequence type to a MPL sequence type. For example, it converts a fusion::vector to a mpl::vector, and a fusion::list to a mpl::list. Maybe pop_back shouldn't require that conversion though.
On 22/08/2012 22:58, Mathias Gaunard wrote:
As the name suggests, it converts a Fusion sequence type to a MPL sequence type.
For example, it converts a fusion::vector to a mpl::vector, and a fusion::list to a mpl::list.
From the rest of the thread, I suppose this is actually incorrect?
On 8/23/2012 8:05 PM, Mathias Gaunard wrote:
On 22/08/2012 22:58, Mathias Gaunard wrote:
As the name suggests, it converts a Fusion sequence type to a MPL sequence type.
For example, it converts a fusion::vector to a mpl::vector, and a fusion::list to a mpl::list.
From the rest of the thread, I suppose this is actually incorrect?
Yep. It converts any fusion sequence S (including MPL sequences) to a specific fusion sequence, by its tag. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On 8/23/2012 4:38 AM, Eric Niebler wrote:
Say I have a valid fusion sequence. I want to use an mpl algorithm on it. I include boost/fusion/mpl.hpp and put a nested tag type in my sequence that is an alias for fusion::fusion_sequence_tag. So far, so good. Then I try to call mpl::pop_back on this sequence, and I find that, despite fulfilling all the documented requirements for Fusion ForwardSequence, the code fails to compile. That's because of this in boost/fusion/mpl/pop_back.hpp:
template <> struct pop_back_impl<fusion::fusion_sequence_tag> { template <typename Sequence> struct apply { typedef typename fusion::result_of::pop_back<Sequence>::type result;
typedef typename fusion::result_of::convert< typename fusion::detail::tag_of<Sequence>::type, result>::type type; }; };
My code doesn't compile because I haven't implemented the fusion::convert customization point. Where is that documented and what does it do?
The basic problem is that MPL has this Extensible Sequence concept that Fusion does not have. Keep in mind that mpl::pop_back requires a Front Extensible Sequence and Fusion only guarantees adaptation of an MPL Forward, Bidirectional or Random Access Sequence (http://tinyurl.com/c7tmyfj). In this regard, we are actually going beyond what's guaranteed by Fusion. That being said, I am really considering adding extensible sequences into Fusion. The Deque, for example is a good example. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On 8/22/2012 7:45 PM, Joel de Guzman wrote:
On 8/23/2012 4:38 AM, Eric Niebler wrote:
Say I have a valid fusion sequence. I want to use an mpl algorithm on it. I include boost/fusion/mpl.hpp and put a nested tag type in my sequence that is an alias for fusion::fusion_sequence_tag. So far, so good. Then I try to call mpl::pop_back on this sequence, and I find that, despite fulfilling all the documented requirements for Fusion ForwardSequence, the code fails to compile. That's because of this in boost/fusion/mpl/pop_back.hpp:
template <> struct pop_back_impl<fusion::fusion_sequence_tag> { template <typename Sequence> struct apply { typedef typename fusion::result_of::pop_back<Sequence>::type result;
typedef typename fusion::result_of::convert< typename fusion::detail::tag_of<Sequence>::type, result>::type type; }; };
My code doesn't compile because I haven't implemented the fusion::convert customization point. Where is that documented and what does it do?
The basic problem is that MPL has this Extensible Sequence concept that Fusion does not have. Keep in mind that mpl::pop_back requires a Front Extensible Sequence and Fusion only guarantees adaptation of an MPL Forward, Bidirectional or Random Access Sequence (http://tinyurl.com/c7tmyfj). In this regard, we are actually going beyond what's guaranteed by Fusion.
That being said, I am really considering adding extensible sequences into Fusion. The Deque, for example is a good example.
The point remains: there is a required Fusion customization point that is not documented. (Or is it?) So, should convert generate an mpl sequence? Or a Fusion sequence? Is there any post-condition regarding the tag type (or fusion_tag type?) of the resulting sequence? TIA, -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 8/23/2012 11:04 AM, Eric Niebler wrote:
On 8/22/2012 7:45 PM, Joel de Guzman wrote:
On 8/23/2012 4:38 AM, Eric Niebler wrote:
Say I have a valid fusion sequence. I want to use an mpl algorithm on it. I include boost/fusion/mpl.hpp and put a nested tag type in my sequence that is an alias for fusion::fusion_sequence_tag. So far, so good. Then I try to call mpl::pop_back on this sequence, and I find that, despite fulfilling all the documented requirements for Fusion ForwardSequence, the code fails to compile. That's because of this in boost/fusion/mpl/pop_back.hpp:
template <> struct pop_back_impl<fusion::fusion_sequence_tag> { template <typename Sequence> struct apply { typedef typename fusion::result_of::pop_back<Sequence>::type result;
typedef typename fusion::result_of::convert< typename fusion::detail::tag_of<Sequence>::type, result>::type type; }; };
My code doesn't compile because I haven't implemented the fusion::convert customization point. Where is that documented and what does it do?
The basic problem is that MPL has this Extensible Sequence concept that Fusion does not have. Keep in mind that mpl::pop_back requires a Front Extensible Sequence and Fusion only guarantees adaptation of an MPL Forward, Bidirectional or Random Access Sequence (http://tinyurl.com/c7tmyfj). In this regard, we are actually going beyond what's guaranteed by Fusion.
That being said, I am really considering adding extensible sequences into Fusion. The Deque, for example is a good example.
The point remains: there is a required Fusion customization point that is not documented. (Or is it?)
No, it is not.
So, should convert generate an mpl sequence? Or a Fusion sequence?
A Fusion sequence.
Is there any post-condition regarding the tag type (or fusion_tag type?) of the resulting sequence?
The actual (undocumented) extension mechanism is: namespace extension { template <typename Tag> struct convert_impl; } which result_of::convert calls. Here's an example for vector: struct vector_tag; namespace extension { template <typename T> struct convert_impl; template <> struct convert_impl<vector_tag> { template <typename Sequence> struct apply { typedef typename detail::as_vector<result_of::size<Sequence>::value> gen; typedef typename gen:: template apply<typename result_of::begin<Sequence>::type>::type type; static type call(Sequence& seq) { return gen::call(fusion::begin(seq)); } }; }; } Pretty much straightforward. I think it is safe to use this extension mechanism. It has been stable for a long time now. Surely, that is not an excuse for it being undocumented though. At any rate, I added this in my TODO list. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On 23/08/2012 05:39, Joel de Guzman wrote:
which result_of::convert calls. Here's an example for vector:
struct vector_tag;
namespace extension { template <typename T> struct convert_impl;
template <> struct convert_impl<vector_tag> { template <typename Sequence> struct apply { typedef typename detail::as_vector<result_of::size<Sequence>::value> gen; typedef typename gen:: template apply<typename result_of::begin<Sequence>::type>::type type;
static type call(Sequence& seq) { return gen::call(fusion::begin(seq)); } }; }; }
Pretty much straightforward. I think it is safe to use this extension mechanism. It has been stable for a long time now. Surely, that is not an excuse for it being undocumented though. At any rate, I added this in my TODO list.
Any reason not to make the vector version the default?
On 8/23/2012 7:57 PM, Mathias Gaunard wrote:
On 23/08/2012 05:39, Joel de Guzman wrote:
which result_of::convert calls. Here's an example for vector:
struct vector_tag;
namespace extension { template <typename T> struct convert_impl;
template <> struct convert_impl<vector_tag> { template <typename Sequence> struct apply { typedef typename detail::as_vector<result_of::size<Sequence>::value> gen; typedef typename gen:: template apply<typename result_of::begin<Sequence>::type>::type type;
static type call(Sequence& seq) { return gen::call(fusion::begin(seq)); } }; }; }
Pretty much straightforward. I think it is safe to use this extension mechanism. It has been stable for a long time now. Surely, that is not an excuse for it being undocumented though. At any rate, I added this in my TODO list.
Any reason not to make the vector version the default?
No, the result should be the target sequence type identified by its tag. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On 8/23/2012 7:35 AM, Joel de Guzman wrote:
On 8/23/2012 7:57 PM, Mathias Gaunard wrote:
On 23/08/2012 05:39, Joel de Guzman wrote:
which result_of::convert calls. Here's an example for vector:
struct vector_tag;
namespace extension { template <typename T> struct convert_impl;
template <> struct convert_impl<vector_tag> { template <typename Sequence> struct apply { typedef typename detail::as_vector<result_of::size<Sequence>::value> gen; typedef typename gen:: template apply<typename result_of::begin<Sequence>::type>::type type;
static type call(Sequence& seq) { return gen::call(fusion::begin(seq)); } }; }; }
Pretty much straightforward. I think it is safe to use this extension mechanism. It has been stable for a long time now. Surely, that is not an excuse for it being undocumented though. At any rate, I added this in my TODO list.
Any reason not to make the vector version the default?
No, the result should be the target sequence type identified by its tag.
I managed to implement this customization point for proto, but it wasn't easy, and I hope we can find a better solution. The problem is that, although a proto expression is a fusion sequence of expressions, a fusion sequence of expressions can't be turned into a proto expression without additional information: the tag and the domain. I worked around the problem by making the fusion_tag of proto expressions a template and parameterizing it on the expression's tag and domain. That way, the customization point can be implemented like this: // Note: proto_expr must encode Tag and Domain for // convert_impl to be implementable for proto. template<typename Tag, typename Domain> struct convert_impl<proto::tag::proto_expr<Tag, Domain> > { template<typename Sequence> struct apply { typedef typename proto::result_of::unpack_expr< Tag , Domain , Sequence >::type type; static type call(Sequence& seq) { return proto::unpack_expr<Tag, Domain>(seq); } }; }; IMO, this is less than ideal. In this case, it so happens that tag and domain are pure compile-time information, and can be bundled with the fusion tag. Generally, that won't be the case. Any type that models a more refined concept than Fusion[xxx]Sequence can have additional requirements that would be effectively sliced off by round-tripping through this customization point. Joel, do you have thoughts about this? Would extensible Fusion sequences solve this? -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 8/28/2012 2:06 PM, Eric Niebler wrote:
On 8/23/2012 7:35 AM, Joel de Guzman wrote:
On 8/23/2012 7:57 PM, Mathias Gaunard wrote:
On 23/08/2012 05:39, Joel de Guzman wrote:
which result_of::convert calls. Here's an example for vector:
struct vector_tag;
namespace extension { template <typename T> struct convert_impl;
template <> struct convert_impl<vector_tag> { template <typename Sequence> struct apply { typedef typename detail::as_vector<result_of::size<Sequence>::value> gen; typedef typename gen:: template apply<typename result_of::begin<Sequence>::type>::type type;
static type call(Sequence& seq) { return gen::call(fusion::begin(seq)); } }; }; }
Pretty much straightforward. I think it is safe to use this extension mechanism. It has been stable for a long time now. Surely, that is not an excuse for it being undocumented though. At any rate, I added this in my TODO list.
Any reason not to make the vector version the default?
No, the result should be the target sequence type identified by its tag.
I managed to implement this customization point for proto, but it wasn't easy, and I hope we can find a better solution. The problem is that, although a proto expression is a fusion sequence of expressions, a fusion sequence of expressions can't be turned into a proto expression without additional information: the tag and the domain. I worked around the problem by making the fusion_tag of proto expressions a template and parameterizing it on the expression's tag and domain. That way, the customization point can be implemented like this:
// Note: proto_expr must encode Tag and Domain for // convert_impl to be implementable for proto. template<typename Tag, typename Domain> struct convert_impl<proto::tag::proto_expr<Tag, Domain> > { template<typename Sequence> struct apply { typedef typename proto::result_of::unpack_expr< Tag , Domain , Sequence >::type type;
static type call(Sequence& seq) { return proto::unpack_expr<Tag, Domain>(seq); } }; };
IMO, this is less than ideal. In this case, it so happens that tag and domain are pure compile-time information, and can be bundled with the fusion tag. Generally, that won't be the case. Any type that models a more refined concept than Fusion[xxx]Sequence can have additional requirements that would be effectively sliced off by round-tripping through this customization point.
Joel, do you have thoughts about this? Would extensible Fusion sequences solve this?
In my mind, if we have forward and backward extensible Fusion sequences (and we should), the proper CP will be fusion::pop_back_impl. Fusion's mpl extensions, e.g. mpl::pop_back_impl will simply call fusion's CP for known (traits) forward and backward extensible Fusion sequences and default to the original behavior for non-extensible Fusion sequences (as before). Convert is another story. And as you pointed out, it looks like it needs more info than is provided in the CP. But still I lack proper understanding of this particular use-case. In general, convert should be able to convert any fusion sequence T (including MPL sequences) to another sequence U. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
On 8/29/2012 12:58 AM, Joel de Guzman wrote:
On 8/28/2012 2:06 PM, Eric Niebler wrote:
IMO, this is less than ideal. In this case, it so happens that tag and domain are pure compile-time information, and can be bundled with the fusion tag. Generally, that won't be the case. Any type that models a more refined concept than Fusion[xxx]Sequence can have additional requirements that would be effectively sliced off by round-tripping through this customization point.
Joel, do you have thoughts about this? Would extensible Fusion sequences solve this?
In my mind, if we have forward and backward extensible Fusion sequences (and we should), the proper CP will be fusion::pop_back_impl. Fusion's mpl extensions, e.g. mpl::pop_back_impl will simply call fusion's CP for known (traits) forward and backward extensible Fusion sequences and default to the original behavior for non-extensible Fusion sequences (as before).
Makes sense.
Convert is another story. And as you pointed out, it looks like it needs more info than is provided in the CP. But still I lack proper understanding of this particular use-case. In general, convert should be able to convert any fusion sequence T (including MPL sequences) to another sequence U.
The problem was reported by Mathias Guarnard and reported here: <https://svn.boost.org/trac/boost/ticket/7254> He was calling mpl::pop_back on a Proto expression and it was failing. Here is the code, which fails on 1.51 but will succeed on trunk because of my ugly hack: #include <boost/proto/proto.hpp> #include <boost/fusion/mpl.hpp> #include <boost/mpl/pop_back.hpp> #include <boost/type_traits/is_same.hpp> #include <boost/static_assert.hpp> namespace mpl = boost::mpl; namespace proto = boost::proto; namespace fusion = boost::fusion; using proto::_; template<class E> struct my_expr; struct my_domain : proto::domain<proto::generator<my_expr> > {}; template<class E> struct my_expr : proto::extends<E, my_expr<E>, my_domain> { my_expr(E const &e = E()) : proto::extends<E, my_expr<E>, my_domain>(e) {} typedef fusion::fusion_sequence_tag tag; }; template<typename T> void test_impl(T const &) { typedef typename mpl::pop_back<T>::type result_type; BOOST_STATIC_ASSERT( (boost::is_same< result_type , my_expr<proto::basic_expr<proto::tag::plus, proto::list1<my_expr<proto::terminal<int>::type>&> > > >::value) ); } // Test that we can call mpl algorithms on proto expression types, and get proto expression types back void test_mpl() { my_expr<proto::terminal<int>::type> i; test_impl(i + i); } The MPL tag of the Proto expression is fusion_sequence_tag, so mpl::pop_back uses fusion's mpl hooks. The fusion implementation of mpl pop_pack first creates a fusion iterator_range of fusion pop_back iterators, both of which wrap a proto expression iterator. Then, fusion calls convert to turn this fusion sequence back into a proto expression before handing it back to mpl. Proto, in the convert_impl CP, just sees a fusion range of proto expressions coming in. It can't know in general what proto expression was initially used before fusion modified it, so it can't find out what that expression's tag type was. And it needs to know that if it's to turn the sequence of child expressions back into an expression. Does that help? -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 8/30/2012 12:59 AM, Eric Niebler wrote:
On 8/29/2012 12:58 AM, Joel de Guzman wrote:
On 8/28/2012 2:06 PM, Eric Niebler wrote:
IMO, this is less than ideal. In this case, it so happens that tag and domain are pure compile-time information, and can be bundled with the fusion tag. Generally, that won't be the case. Any type that models a more refined concept than Fusion[xxx]Sequence can have additional requirements that would be effectively sliced off by round-tripping through this customization point.
Joel, do you have thoughts about this? Would extensible Fusion sequences solve this?
In my mind, if we have forward and backward extensible Fusion sequences (and we should), the proper CP will be fusion::pop_back_impl. Fusion's mpl extensions, e.g. mpl::pop_back_impl will simply call fusion's CP for known (traits) forward and backward extensible Fusion sequences and default to the original behavior for non-extensible Fusion sequences (as before).
Makes sense.
Convert is another story. And as you pointed out, it looks like it needs more info than is provided in the CP. But still I lack proper understanding of this particular use-case. In general, convert should be able to convert any fusion sequence T (including MPL sequences) to another sequence U.
The problem was reported by Mathias Guarnard and reported here: <https://svn.boost.org/trac/boost/ticket/7254>
He was calling mpl::pop_back on a Proto expression and it was failing. Here is the code, which fails on 1.51 but will succeed on trunk because of my ugly hack:
#include <boost/proto/proto.hpp> #include <boost/fusion/mpl.hpp> #include <boost/mpl/pop_back.hpp> #include <boost/type_traits/is_same.hpp> #include <boost/static_assert.hpp> namespace mpl = boost::mpl; namespace proto = boost::proto; namespace fusion = boost::fusion; using proto::_;
template<class E> struct my_expr;
struct my_domain : proto::domain<proto::generator<my_expr> > {};
template<class E> struct my_expr : proto::extends<E, my_expr<E>, my_domain> { my_expr(E const &e = E()) : proto::extends<E, my_expr<E>, my_domain>(e) {}
typedef fusion::fusion_sequence_tag tag; };
template<typename T> void test_impl(T const &) { typedef typename mpl::pop_back<T>::type result_type; BOOST_STATIC_ASSERT( (boost::is_same< result_type , my_expr<proto::basic_expr<proto::tag::plus, proto::list1<my_expr<proto::terminal<int>::type>&> > > >::value) ); }
// Test that we can call mpl algorithms on proto expression types, and get proto expression types back void test_mpl() { my_expr<proto::terminal<int>::type> i; test_impl(i + i); }
The MPL tag of the Proto expression is fusion_sequence_tag, so mpl::pop_back uses fusion's mpl hooks. The fusion implementation of mpl pop_pack first creates a fusion iterator_range of fusion pop_back iterators, both of which wrap a proto expression iterator. Then, fusion calls convert to turn this fusion sequence back into a proto expression before handing it back to mpl.
Proto, in the convert_impl CP, just sees a fusion range of proto expressions coming in. It can't know in general what proto expression was initially used before fusion modified it, so it can't find out what that expression's tag type was. And it needs to know that if it's to turn the sequence of child expressions back into an expression.
Does that help?
Yes. My gut feeling is that the "convert" mechanism is either 1) broken or 2) we are asking it to do more than it is designed for. I am not sure which. The intent of the convert mechanism is to be able to generate a container C from any type of sequence S. Now, in general we don't know the exact type of C. E.g. We know that it is a vector, but what particular type --the number of elements and element types we don't care about. All we know is that, hey, I have a sequence S and I want to create a vector. Fill in the blanks (elements and element types) for me. So that is why it is tag based. So, for example, if I want to convert S to a fusion vector, I do: result_of::convert<tag-of-vector, S>::type This is the generalized mechanism behind: result_of::as_vector<S>::type As mentioned, S can be any sequence type. It can be an MPL sequence, it can be a vector, a list, etc. So, there is no way really to provide more information about S other than the requirements for a forward fusion sequence. So, if a container C is not generally not convertible from any arbitrary sequence S, then, it probably does not satisfy the convertible (inventing the name for the sake of discussion) concept. Yes, it's not really a formalized concept, but all the built-in fusion sequences have this ability. Take a peek at the docs (e.g. http://tinyurl.com/6xlnxd8) and notice that containers can take in any sequence S in its constructor. E.g.: V(s) // Copy constructs a vector from a Forward Sequence, s. ------------------ I think I am going to embark on some fusion-care. It's really not that difficult to add the extensible sequence concepts and CPs. Apart from that, there are other pressing matters that I need to take care of anyway; rvalue handling comes to mind. Regards, -- Joel de Guzman http://www.boostpro.com http://boost-spirit.com
participants (4)
-
Eric Niebler -
Joel de Guzman -
Joel de Guzman -
Mathias Gaunard