Requesting a review of Metaparse

Hi, I've created a version of Metaparse that depends on the Boost libraries only. I request this version of Metaparse be placed in the queue for formal review. The zipped source code can be found here: http://abel.web.elte.hu/metaparse/metaparse.zip The documentation can be found here: http://abel.web.elte.hu/metaparse/metaparse The library has been tested on the following compilers: - GCC 4.6 (with and without -std=c++0x) - GCC 3.4 - Clang 2.9 - Visual C++ 10 A quick summary: Metaparse is a library for constructing parsers parsing at compile-time based on template metaprogramming. The parsers built with the library take boost::mpl::strings as input and can produce - types - objects (types with public static members) - callable C++ functions (types with public static method) - template metafunction classes as output (based on the input being parsed). On compilers supporting constexpr the library provides the following syntactic sugar for writing the input of the parsers: BOOST_STRING("this is a string") The library can be used for implementing DSLs in C++, including DSLs making C++ template metaprogramming easier (see examples). Regards, Abel

On 28 Jan 2012, at 18:24, Abel Sinkovics wrote:
Hi,
I've created a version of Metaparse that depends on the Boost libraries only. I request this version of Metaparse be placed in the queue for formal review.
The zipped source code can be found here: http://abel.web.elte.hu/metaparse/metaparse.zip The documentation can be found here: http://abel.web.elte.hu/metaparse/metaparse
The library has been tested on the following compilers: - GCC 4.6 (with and without -std=c++0x) - GCC 3.4 - Clang 2.9 - Visual C++ 10
A quick summary:
Metaparse is a library for constructing parsers parsing at compile-time based on template metaprogramming. The parsers built with the library take boost::mpl::strings as input and can produce
- types - objects (types with public static members) - callable C++ functions (types with public static method) - template metafunction classes
as output (based on the input being parsed).
On compilers supporting constexpr the library provides the following syntactic sugar for writing the input of the parsers:
BOOST_STRING("this is a string")
Two questions: 1) What are the upper limits on the kinds of parsers you can build? 2) How does the library compare to boost::spirit? Is there some reason you did not build it on to of that existing library? Chris

Hi Chris,
Two questions:
1) What are the upper limits on the kinds of parsers you can build? 2) How does the library compare to boost::spirit? Is there some reason you did not build it on to of that existing library?
1) I haven't constructed parsers yet, that were too complex for the compilers I've been testing with, thus I don't know the limits yet. 2) Parsers built with Spirit parse at runtime, while parsers built with Metaparse parse at compile-time and can produce things that can affect the compilation process. I've added this to the documentation (index.html). I can't see how it could be built on top of Spirit. Regards, Abel

On 01/29/2012 11:51 AM, Abel Sinkovics wrote:
Hi Chris,
Two questions:
1) What are the upper limits on the kinds of parsers you can build? 2) How does the library compare to boost::spirit? Is there some reason you did not build it on to of that existing library?
1)
I haven't constructed parsers yet, that were too complex for the compilers I've been testing with, thus I don't know the limits yet.
Then how can this library be reviewed if there has been no use of it or application and the limits of the library are known to exist but we don't know what they are?

Hi Mathias,
Then how can this library be reviewed if there has been no use of it or application and the limits of the library are known to exist but we don't know what they are?
It has been used to construct parsers for printf (added as an example) and other DSLs (not made public) for compile-time verification. These use cases and the tests/examples were not too complex for the library/compilers. I'll measure how complex the parsers or how long the input can be for the library/compilers to handle it and add it to the documentation. Regards, Abel

In message <4F2524C4.9000904@elte.hu>, Abel Sinkovics <abel@elte.hu> writes
Hi Chris,
...
1) What are the upper limits on the kinds of parsers you can build?
...
1)
I haven't constructed parsers yet, that were too complex for the compilers I've been testing with, thus I don't know the limits yet.
I think that this can be/has been read differently from your intent, due to the first comma. (Since you have constructed parsers.) No? Alec -- Alec Ross

Hi,
In message <4F2524C4.9000904@elte.hu>, Abel Sinkovics <abel@elte.hu> writes
Hi Chris,
...
1) What are the upper limits on the kinds of parsers you can build?
...
1)
I haven't constructed parsers yet, that were too complex for the compilers I've been testing with, thus I don't know the limits yet.
I think that this can be/has been read differently from your intent, due to the first comma. (Since you have constructed parsers.) No?
Alec
Thank you for pointing that out Alec. What I intended to say was: I (and others) constructed parsers. None of those parsers was too complex. The library and the compilers could deal with all of them. As a result of this, I don't know the limits of the library yet. Sorry for any confusion I caused. Chris, I'm happy to measure limits for you. Could you be more specific on what limits you are interested in? Regards, Abel

Hi Abel, I prefer Sprout.Weed as a compile-time syntax analysis library. https://github.com/bolero-MURAKAMI/Sprout/tree/master/sprout/weed Sprout.Weed support Boost.Spirit.Qi like syntax. Genya Murakami is an author, seems to be willing to propose to Boost. http://thread.gmane.org/gmane.comp.lib.boost.devel/227096/focus=227134 If you think about this library? Thanks, Akira 2012/1/29 Abel Sinkovics <abel@elte.hu>:
Hi,
I've created a version of Metaparse that depends on the Boost libraries only. I request this version of Metaparse be placed in the queue for formal review.
The zipped source code can be found here: http://abel.web.elte.hu/metaparse/metaparse.zip The documentation can be found here: http://abel.web.elte.hu/metaparse/metaparse
The library has been tested on the following compilers: - GCC 4.6 (with and without -std=c++0x) - GCC 3.4 - Clang 2.9 - Visual C++ 10
A quick summary:
Metaparse is a library for constructing parsers parsing at compile-time based on template metaprogramming. The parsers built with the library take boost::mpl::strings as input and can produce
- types - objects (types with public static members) - callable C++ functions (types with public static method) - template metafunction classes
as output (based on the input being parsed).
On compilers supporting constexpr the library provides the following syntactic sugar for writing the input of the parsers:
BOOST_STRING("this is a string")
The library can be used for implementing DSLs in C++, including DSLs making C++ template metaprogramming easier (see examples).
Regards, Abel
======================== Akira Takahashi mailto:faithandbrave@gmail.com https://sites.google.com/site/faithandbrave/about/en

Hi Akira,
I prefer Sprout.Weed as a compile-time syntax analysis library. https://github.com/bolero-MURAKAMI/Sprout/tree/master/sprout/weed
Sprout.Weed support Boost.Spirit.Qi like syntax. Genya Murakami is an author, seems to be willing to propose to Boost. http://thread.gmane.org/gmane.comp.lib.boost.devel/227096/focus=227134
If you think about this library?
I've seen this library. It parses using constexpr, while Metaparse uses template metaprograms. I can't see how one could do the following using constexpr: - generate template metafunction classes as the result of parsing that are immediately callable (build a DSL for metaprograms) - printf argument verification (by parsing the format string) at compile-time - create types as the result of parsing One of the examples of Metaparse (constexpr_parser) demonstrates how to combine a parser based on constexpr functions with ones based on template metaprogramming. It is a parser for an "a* b* a*" grammar and parses the "a*" parts using metaprograms and the "b*" parts using constexpr functions. The documentation of Metaparse describes the difference as well (http://abel.web.elte.hu/metaparse/metaparse/manual.html#_parsing_based_on_tt...). Regards, Abel

2012/1/30 Mathias Gaunard <mathias.gaunard@ens-lyon.org>
On 01/30/2012 08:22 AM, Abel Sinkovics wrote:
I can't see how one could do the following using constexpr:
[...]
- create types as the result of parsing
You'd have to return objects, but then you can call decltype on them and get types.
The type of the object you return can't depend on the content of the string. Roman Perepelitsa.

On 01/30/2012 10:37 PM, Roman Perepelitsa wrote:
2012/1/30 Mathias Gaunard<mathias.gaunard@ens-lyon.org>
On 01/30/2012 08:22 AM, Abel Sinkovics wrote:
I can't see how one could do the following using constexpr:
[...]
- create types as the result of parsing
You'd have to return objects, but then you can call decltype on them and get types.
The type of the object you return can't depend on the content of the string.
The arguments themselves aren't constexpr indeed. (constexpr arguments would be something nice to add to the standard)

-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Abel Sinkovics Sent: Saturday, January 28, 2012 10:24 AM To: boost@lists.boost.org Subject: [boost] Requesting a review of Metaparse Hi, I've created a version of Metaparse that depends on the Boost libraries only. I request this version of Metaparse be placed in the queue for formal review. The zipped source code can be found here: http://abel.web.elte.hu/metaparse/metaparse.zip The documentation can be found here: http://abel.web.elte.hu/metaparse/metaparse The library has been tested on the following compilers: - GCC 4.6 (with and without -std=c++0x) - GCC 3.4 - Clang 2.9 - Visual C++ 10 A quick summary: Metaparse is a library for constructing parsers parsing at compile-time based on template metaprogramming. The parsers built with the library take boost::mpl::strings as input and can produce - types - objects (types with public static members) - callable C++ functions (types with public static method) - template metafunction classes as output (based on the input being parsed). On compilers supporting constexpr the library provides the following syntactic sugar for writing the input of the parsers: BOOST_STRING("this is a string") The library can be used for implementing DSLs in C++, including DSLs making C++ template metaprogramming easier (see examples). Regards, Abel _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

From: Abel Sinkovics
The library can be used for implementing DSLs in C++, including DSLs making C++ template metaprogramming easier (see examples).
I like your compile to native code example: " #define LAMBDA(exp) apply_wrap1<function_parser, BOOST_STRING(#exp)>::type LAMBDA(13) f1; LAMBDA(2 + 3) f2; LAMBDA(2 * 2) f3; LAMBDA( 1+ 2*4-6/2) f4; LAMBDA(2 * _) f5; #endif int main() { using std::cout; using std::endl; cout << f1(11) << endl << f2(11) << endl << f3(11) << endl << f4(11) << endl << f5(11) << endl << f5(1.1) << endl ; } " Right now it is hard coded that the function object generated by apply_wrap1 takes one argument. I'm pretty sure you could build compile time logic to search for the largest _2, _4 type token and select the specialization of the wrapper that matches the number of arguments used in the expression string. You have a comment: /* * The grammar * * expression ::= plus_exp * plus_exp ::= prod_exp ((plus_token | minus_token) prod_exp)* * prod_exp ::= value_exp ((mult_token | div_token) value_exp)* * value_exp ::= int_token | '_' */ And then implement the grammar using metaparse based metaprogramming, eventually declaring your compile time parser with this: typedef build_parser<entire_input<expression> > function_parser; I believe we could apply the metaparse library to itself to get a more natural syntax for declaring a metaparser: typedef build_parser<entire_input<grammar<rule<_S("expression ::= plus_exp")>, rule<_S("plus_exp ::= prod_exp ((plus_token | minus_token) prod_exp)*")>, rule<_S("value_exp ((mult_token | div_token) value_exp)*")>, rule<_S("value_exp ::= int_token | '_'")> >, semantic_action<_S("value_exp"), build_value>, semantic_action<_S("prod_exp"), build_mult>, semantic_action<_S("plus_exp"), build_plus> > > function_parser; This is essentially what I have in mind when I think of metaparse and constexpr based mpl string used to make metaprogramming easier. I'd be interested in seeing if you can get the above proposed syntax (or something similar) to compile to demonstrate the metapower of metaparse. A compile time parser for generating compile time parsers. Regards, Luke

Hi Luke,
Right now it is hard coded that the function object generated by apply_wrap1 takes one argument. I'm pretty sure you could build compile time logic to search for the largest _2, _4 type token and select the specialization of the wrapper that matches the number of arguments used in the expression string. Yes, it is possible. I used the "always take 1 argument" approach to make the example as simple as I could - the goal of this example is the demonstration of how to build callable functions.
You have a comment: /* * The grammar * * expression ::= plus_exp * plus_exp ::= prod_exp ((plus_token | minus_token) prod_exp)* * prod_exp ::= value_exp ((mult_token | div_token) value_exp)* * value_exp ::= int_token | '_' */
And then implement the grammar using metaparse based metaprogramming, eventually declaring your compile time parser with this:
typedef build_parser<entire_input<expression> > function_parser;
I believe we could apply the metaparse library to itself to get a more natural syntax for declaring a metaparser:
typedef build_parser<entire_input<grammar<rule<_S("expression ::= plus_exp")>, rule<_S("plus_exp ::= prod_exp ((plus_token | minus_token) prod_exp)*")>, rule<_S("value_exp ((mult_token | div_token) value_exp)*")>, rule<_S("value_exp ::= int_token | '_'")> >, semantic_action<_S("value_exp"), build_value>, semantic_action<_S("prod_exp"), build_mult>, semantic_action<_S("plus_exp"), build_plus> > > function_parser;
This is essentially what I have in mind when I think of metaparse and constexpr based mpl string used to make metaprogramming easier.
I'd be interested in seeing if you can get the above proposed syntax (or something similar) to compile to demonstrate the metapower of metaparse. A compile time parser for generating compile time parsers.
I've added a new example: meta_metaparse doing this. You can define the grammar the following way: typedef grammar<> ::rule_<_S("plus_token"), token<lit_c<'+'>>>::type ::rule_<_S("minus_token"), token<lit_c<'-'>>>::type ::rule_<_S("mult_token"), token<lit_c<'*'>>>::type ::rule_<_S("div_token"), token<lit_c<'/'>>>::type ::rule_<_S("arg_token"), token<lit_c<'_'>>>::type ::rule_<_S("int_token"), token<int_>>::type ::rule<_S("S"), _S("plus_exp")>::type ::rule<_S("plus_exp"), _S("prod_exp plus_exp_*")>::type ::rule<_S("plus_exp_"), _S("plus_op prod_exp")>::type ::rule<_S("plus_op"), _S("plus_token | minus_token")>::type ::rule<_S("prod_exp"), _S("value_exp prod_exp_*")>::type ::rule<_S("prod_exp_"), _S("prod_op value_exp")>::type ::rule<_S("prod_op"), _S("mult_token | div_token")>::type ::rule<_S("value_exp"), _S("int_token | arg_token")>::type ::semantic_action<_S("int_token"), build_value>::type ::semantic_action<_S("arg_token"), build_arg>::type ::semantic_action<_S("prod_exp"), build_mult>::type ::semantic_action<_S("plus_exp"), build_plus>::type g; typedef build_parser<entire_input<g>> function_parser; Regards, Abel
participants (7)
-
Abel Sinkovics
-
Akira Takahashi
-
Alec Ross
-
Christopher Jefferson
-
Mathias Gaunard
-
Roman Perepelitsa
-
Simonson, Lucanus J