[wave] limited extensibility
Hi, i am trying to use boost.wave as a preprocessor for OpenGL Shading Language shader files (GLSL). GLSL uses a limited C/C++ preprocessor (no included etc.). I want to use boost.wave to handle the includes correctly besides generally gaining a more powerful preprocessor. Now the problem is that there a special preprocessor directives in GLSL that the compiler itself has to handle. I am speaking of the #version and #extension directives. I hoped boost.wave would provide the possibility to add special directives that the client code then can handle properly. Currently I use the context hook as follows: class glsl_preprocessing_hook : public boost::wave::context_policies::default_preprocessing_hooks { public: template <typename ContextT> void throw_exception(ContextT const &ctx, std::exception const& e) { throw e; } template <typename ContextT> void throw_exception(ContextT const &ctx, boost::wave::preprocess_exception const& e) { // just ignore ill formed directives (version, exception...) and let the glsl compiler handle these if(e.get_errorcode() != boost::wave::preprocess_exception::ill_formed_directive) { throw e; } } }; This just ignores the ill formed directives, but may still let unwanted errors through. Would it be possible to extend boost.wave in a way to be able feed the lexer or context regular expressions of the new custom directives? These could be pushed into a special hook method with the appropriate context information. I think this functionality would greatly increase the usability for C++ like languages with custom extensions to the preprocessor. Regards -chris
i am trying to use boost.wave as a preprocessor for OpenGL Shading Language shader files (GLSL). GLSL uses a limited C/C++ preprocessor (no included etc.). I want to use boost.wave to handle the includes correctly besides generally gaining a more powerful preprocessor.
Now the problem is that there a special preprocessor directives in GLSL that the compiler itself has to handle. I am speaking of the #version and #extension directives. I hoped boost.wave would provide the possibility to add special directives that the client code then can handle properly.
Currently I use the context hook as follows:
class glsl_preprocessing_hook : public boost::wave::context_policies::default_preprocessing_hooks { public: template <typename ContextT> void throw_exception(ContextT const &ctx, std::exception const& e) { throw e; } template <typename ContextT> void throw_exception(ContextT const &ctx, boost::wave::preprocess_exception const& e) { // just ignore ill formed directives (version, exception...) and let the glsl compiler handle these if(e.get_errorcode() != boost::wave::preprocess_exception::ill_formed_directive) { throw e; } } };
This just ignores the ill formed directives, but may still let unwanted errors through.
Would it be possible to extend boost.wave in a way to be able feed the lexer or context regular expressions of the new custom directives? These could be pushed into a special hook method with the appropriate context information. I think this functionality would greatly increase the usability for C++ like languages with custom extensions to the preprocessor.
Extending Wave to recognize more preprocessor directives without involving changes to the lexer seems possible by introducing a on_illformed() preprocessing hook and pass it the whole line starting with the faulty token. This would allow you to do the parsing on your own, instructing Wave to continue preprocessing afterwards. The default behavior of on_illformed would be to throw the ill_formed_directive exception as before. Does this make sense to you? Regards Hartmut --------------- Meet me at BoostCon www.boostcon.com
Hi Hartmut, that sounds perfect. When passing the line, i think, possible comments should be stripped to prevent problems when this line is stripped from the output. The on_illformed hook should also have a boolean return value signaling to pass the line to the output or to strip it (similar to other hooks). Regards -chris On Sat, Feb 6, 2010 at 7:00 PM, Hartmut Kaiser <hartmut.kaiser@gmail.com> wrote:
Extending Wave to recognize more preprocessor directives without involving changes to the lexer seems possible by introducing a on_illformed() preprocessing hook and pass it the whole line starting with the faulty token. This would allow you to do the parsing on your own, instructing Wave to continue preprocessing afterwards. The default behavior of on_illformed would be to throw the ill_formed_directive exception as before.
Christopher,
that sounds perfect.
When passing the line, i think, possible comments should be stripped to prevent problems when this line is stripped from the output. The on_illformed hook should also have a boolean return value signaling to pass the line to the output or to strip it (similar to other hooks).
Ok, I added the following preprocessing hook to Wave: /////////////////////////////////////////////////////////////////////////// // // The function 'found_unknown_directive' is called whenever an unknown // preprocessor directive was encountered. // // The parameter 'ctx' is a reference to the context object used for // instantiating the preprocessing iterators by the user. // // The parameter 'line' holds the tokens of the entire source line // containing the unknown directive. // // The parameter 'pending' may be used to push tokens back into the input // stream, which are to be used as the replacement text for the whole // line containing the unknown directive. // // The return value defines whether the given expression has been // properly interpreted by the hook function or not. If this function // returns 'false', the library will raise an 'ill_formed_directive' // preprocess_exception. Otherwise the tokens pushed back into 'pending' // are passed on to the user program. // /////////////////////////////////////////////////////////////////////////// template <typename Context, typename Container> bool found_unknown_directive(Context const& ctx, Container const& line, Container& pending) { return false; // by default we never interpret unknown directives } Depending on the settings comments are either retained or stripped _before_ invoking this hook function. For instance for a line: #version 150 core // some comment the parameter 'line' will contain the following tokens: T_POUND "#" T_IDENTIFIER "version" T_SPACE " " T_PP_NUMBER "150" T_SPACE " " T_IDENTIFIER "core" T_NEWLINE "\n" (if comments are to be stripped), or: T_POUND "#" T_IDENTIFIER "version" T_SPACE " " T_PP_NUMBER "150" T_SPACE " " T_IDENTIFIER "core" T_CPPCOMMENT "// some comment\n" if those are to be retained. A new example (custom_directives.cpp) demonstrates its usage. HTH Regards Hartmut --------------- Meet me at BoostCon www.boostcon.com
Hi Hartmut, thank you! That was amazing fast. I tried the new hook and I am now able to get all the information I need. Thank you very much for the fast help. -chris On Sun, Feb 7, 2010 at 5:07 PM, Hartmut Kaiser <hartmut.kaiser@gmail.com> wrote:
Christopher,
that sounds perfect.
When passing the line, i think, possible comments should be stripped to prevent problems when this line is stripped from the output. The on_illformed hook should also have a boolean return value signaling to pass the line to the output or to strip it (similar to other hooks).
Ok, I added the following preprocessing hook to Wave:
/////////////////////////////////////////////////////////////////////////// // // The function 'found_unknown_directive' is called whenever an unknown // preprocessor directive was encountered. // // The parameter 'ctx' is a reference to the context object used for // instantiating the preprocessing iterators by the user. // // The parameter 'line' holds the tokens of the entire source line // containing the unknown directive. // // The parameter 'pending' may be used to push tokens back into the input // stream, which are to be used as the replacement text for the whole // line containing the unknown directive. // // The return value defines whether the given expression has been // properly interpreted by the hook function or not. If this function // returns 'false', the library will raise an 'ill_formed_directive' // preprocess_exception. Otherwise the tokens pushed back into 'pending' // are passed on to the user program. // /////////////////////////////////////////////////////////////////////////// template <typename Context, typename Container> bool found_unknown_directive(Context const& ctx, Container const& line, Container& pending) { return false; // by default we never interpret unknown directives }
Depending on the settings comments are either retained or stripped _before_ invoking this hook function. For instance for a line:
#version 150 core // some comment
the parameter 'line' will contain the following tokens:
T_POUND "#" T_IDENTIFIER "version" T_SPACE " " T_PP_NUMBER "150" T_SPACE " " T_IDENTIFIER "core" T_NEWLINE "\n"
(if comments are to be stripped), or:
T_POUND "#" T_IDENTIFIER "version" T_SPACE " " T_PP_NUMBER "150" T_SPACE " " T_IDENTIFIER "core" T_CPPCOMMENT "// some comment\n"
if those are to be retained.
A new example (custom_directives.cpp) demonstrates its usage.
HTH Regards Hartmut
--------------- Meet me at BoostCon www.boostcon.com
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (2)
-
Christopher Lux -
Hartmut Kaiser