Using program_options with other options methods
I have recently started using the program_options library, and I find it very useful. Unfortunately I can't seem to figure out how to allow it to work with other option parsers/handlers at the same time. For example, right now I am writing an application that uses a library that has its own set of common options that are process internally. So I end up needing to write code similar to the following: po::options_description desc("Allowed options"); desc.add_options() ("help", "produce help message") ("my_opt", po::value<int>(), "set my option") ; po::variables_map vm; po::store(po::parse_command_line(ac, av, desc), vm); po::notify(vm); custom_lib::options opts; opts.process(ac, av); Then when I run this application: ./app --my_opt=10 --liboption=custom The problem is that I get an exception from program_options: error: unknown option --liboption Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers? -Allen
On Tue, 23 Nov 2004 10:43:38 -0600 Allen Bierbaum <allenb@vrsource.org> wrote:
./app --my_opt=10 --liboption=custom
The problem is that I get an exception from program_options:
error: unknown option --liboption
Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers?
I think this is what you want... http://www.boost.org/doc/html/program_options/howto.html#id549217
Jody Hagins wrote:
On Tue, 23 Nov 2004 10:43:38 -0600 Allen Bierbaum <allenb@vrsource.org> wrote:
./app --my_opt=10 --liboption=custom
The problem is that I get an exception from program_options:
error: unknown option --liboption
Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers?
I think this is what you want... http://www.boost.org/doc/html/program_options/howto.html#id549217
From what I understand of the custom parsers, this just allows the developer to recognize addition options and options formats. This doesn't handle my case because what I want to do is ignore unknown options. (so they can be handled by a different method entirely). -Allen
On Tue, 23 Nov 2004 15:42:40 -0600 Allen Bierbaum <allenb@vrsource.org> wrote:
From what I understand of the custom parsers, this just allows the developer to recognize addition options and options formats. This doesn't handle my case because what I want to do is ignore unknown options. (so they can be handled by a different method entirely).
Right. Hmmm... I guess a <cough> "workaround" would be to return something like std::make_pair(std::string("Internal Unhandled Parameters"), s) and then you can have a vector of strings for "Internal Unhandled Parameters" and your user callback can then invoke your third party call to parse those options. Ideally, you would the ability to write a custom parser that "handles" the option, but does not return any information back to the library. I am not familiar enough with the library to know the "correct" method. The only thing I can think of is that the interface is lacking. I took a few minutes to look at the code in libs/program_options/src/cmdline.cpp and it looks like the only interpretation is empty or a valid option name. So, at this stage, I think it requires an interface change to support this option. I am, most likely, missing something in my very brief glance, so I guess we need to wait for Vladimir to chime in with the real answer...
Hi Jody, Hi Allen, Allen Bierbaum wrote:
Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers?
Jody wrote:
I am not familiar enough with the library to know the "correct" method. The only thing I can think of is that the interface is lacking. I took a few minutes to look at the code in libs/program_options/src/cmdline.cpp and it looks like the only interpretation is empty or a valid option name.
Ok, let me start with a caveat: ignoring unknown options can be ambiguous: program --something 10 --known_option Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form: --something=10 which may be problem, or may not, depending on your situation. What do you think?
So, at this stage, I think it requires an interface change to support this option.
The detail::cmdline constructor actually has 'allow_unregisted' parameter, which is intended to help with this. However, no non-detail interface allows to specify. Frankly, I don't know if I forgot about this or just decided to wait for a user to request this feature explicitly. Adding a new method to the 'common_command_line_parser' to set 'allow_unregistered' flag and then tweaking common_command_line_parser::run should do the trick? Opinions? - Volodya
On Wed, 24 Nov 2004 10:27:59 +0300 Vladimir Prus <ghost@cs.msu.su> wrote:
Hi Jody, Hi Allen,
Allen Bierbaum wrote:
Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers?
Jody wrote:
I am not familiar enough with the library to know the "correct" method. The only thing I can think of is that the interface is lacking. I took a few minutes to look at the code in libs/program_options/src/cmdline.cpp and it looks like the only interpretation is empty or a valid option name.
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do you think?
I think your example actually brings up several issues. First, I was actually a bit surprised to see in the docs that the library does not require positional parameters to follow all non-positional parameters. The normal unix use cases typically expect options first, then positional parameters. If the user wants to "end" the options early, then a "--" is inserted as an option that specifically says to end option processing and begin positional processing. The ambiguity in your example is actually caused by the interpretation of positional options. I guess if I were writing a custom parser to handle "--something 10" then I would give the custom parser some state, and it would have to know what to expect.
So, at this stage, I think it requires an interface change to support this option.
The detail::cmdline constructor actually has 'allow_unregisted' parameter, which is intended to help with this. However, no non-detail interface allows to specify. Frankly, I don't know if I forgot about this or just decided to wait for a user to request this feature explicitly.
Adding a new method to the 'common_command_line_parser' to set 'allow_unregistered' flag and then tweaking common_command_line_parser::run should do the trick? Opinions?
Maybe. I am trying to think in terms of bigger picture use case support. I think we need the following... 1. Ability to tell the parser to be in "strict" mode or something like that, where it assumes options come first, and "positional options" follow. This will remove all ambiguity in parsing options, and AFAIK, it is consistent with just about all conventions. 2. Currently, a custom parser can pass two kinds of information; either "I do not recognize this option" or "I recognize this option, and here is a name/value pair that I want you to use as a substitute for the option." It needs to be enhanced so it can also pass "I recognize this option, and you should ignore it." 3. Some way for the main cmdline parser to do all its parsing, and return all "unrecognized" stuff, without an error. This way, parsers can be chained. For example, parsing in such a manner... program --something 10 --known_option would resognize --known_option and I can then as for "parsed" options and get a vector of strings representing the parsed options. I should also be able to get a vector of strings of "unparsed" options. I think this will allow me to chain option processing in almost any manner...
Hi Jody,
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do you think?
I think your example actually brings up several issues.
First, I was actually a bit surprised to see in the docs that the library does not require positional parameters to follow all non-positional parameters. The normal unix use cases typically expect options first, then positional parameters. If the user wants to "end" the options early, then a "--" is inserted as an option that specifically says to end option processing and begin positional processing.
The ambiguity in your example is actually caused by the interpretation of positional options.
It's an explicit decision to allow positional options everywhere. I can't count the number of times when I run cvs ci foo it failed and I then runned cvs ci foo -F /tmp/commit_message_left_by_failed_invocation to find that options are not allowed after 'foo'.
So, at this stage, I think it requires an interface change to support this option.
The detail::cmdline constructor actually has 'allow_unregisted' parameter, which is intended to help with this. However, no non-detail interface allows to specify. Frankly, I don't know if I forgot about this or just decided to wait for a user to request this feature explicitly.
Adding a new method to the 'common_command_line_parser' to set 'allow_unregistered' flag and then tweaking common_command_line_parser::run should do the trick? Opinions?
Maybe. I am trying to think in terms of bigger picture use case support. I think we need the following...
1. Ability to tell the parser to be in "strict" mode or something like that, where it assumes options come first, and "positional options" follow. This will remove all ambiguity in parsing options, and AFAIK, it is consistent with just about all conventions.
As I mention above, I'm not sure this is right approach.
2. Currently, a custom parser can pass two kinds of information; either "I do not recognize this option" or "I recognize this option, and here is a name/value pair that I want you to use as a substitute for the option." It needs to be enhanced so it can also pass "I recognize this option, and you should ignore it."
I agree, and it will be implemented.
3. Some way for the main cmdline parser to do all its parsing, and return all "unrecognized" stuff, without an error. This way, parsers can be chained. For example, parsing in such a manner... program --something 10 --known_option would resognize --known_option and I can then as for "parsed" options and get a vector of strings representing the parsed options. I should also be able to get a vector of strings of "unparsed" options.
What about program --something 10 11 --known_option In this case, after parsing know options you'll get --something 10 11 which can be passed to another parser, which will eat --something 10 and will leave one token, which will be considered positional options. Seems OK. - Volodya
On Wed, 24 Nov 2004 19:01:54 +0300 Vladimir Prus <ghost@cs.msu.su> wrote:
1. Ability to tell the parser to be in "strict" mode or something like that, where it assumes options come first, and "positional options" follow. This will remove all ambiguity in parsing options, and AFAIK, it is consistent with just about all conventions.
As I mention above, I'm not sure this is right approach.
Even as an option that the user must set on the cmdline parser?
What about
program --something 10 11 --known_option
In this case, after parsing know options you'll get
--something 10 11
which can be passed to another parser, which will eat
--something 10
and will leave one token, which will be considered positional options. Seems OK.
Seems OK at first blush. Like everything though, someone will come along later with something we have not considered...
Jody Hagins wrote:
On Wed, 24 Nov 2004 19:01:54 +0300 Vladimir Prus <ghost@cs.msu.su> wrote:
1. Ability to tell the parser to be in "strict" mode or something like that, where it assumes options come first, and "positional options" follow. This will remove all ambiguity in parsing options, and AFAIK, it is consistent with just about all conventions.
As I mention above, I'm not sure this is right approach.
Even as an option that the user must set on the cmdline parser?
This particular option would cost nothing, so I might implement it.
What about
program --something 10 11 --known_option
In this case, after parsing know options you'll get
--something 10 11
which can be passed to another parser, which will eat
--something 10
and will leave one token, which will be considered positional options. Seems OK.
Seems OK at first blush. Like everything though, someone will come along later with something we have not considered...
Sure. I'll be waiting ;-) - Volodya
From: Jody Hagins <jody-boost-011304@atdesk.com>
Vladimir Prus <ghost@cs.msu.su> wrote:
Allen Bierbaum wrote:
Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers?
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do you think?
I think your example actually brings up several issues.
First, I was actually a bit surprised to see in the docs that the library does not require positional parameters to follow all non-positional parameters. The normal unix use cases typically expect options first, then positional parameters. If the user wants to "end" the options early, then a "--" is inserted as an option that specifically says to end option processing and begin positional processing.
Regularity is good.
The detail::cmdline constructor actually has 'allow_unregisted' parameter, which is intended to help with this. However, no non-detail interface allows to specify. Frankly, I don't know if I forgot about this or just decided to wait for a user to request this feature explicitly.
Adding a new method to the 'common_command_line_parser' to set 'allow_unregistered' flag and then tweaking common_command_line_parser::run should do the trick? Opinions?
Maybe. I am trying to think in terms of bigger picture use case support. I think we need the following...
1. Ability to tell the parser to be in "strict" mode or something like that, where it assumes options come first, and "positional options" follow. This will remove all ambiguity in parsing options, and AFAIK, it is consistent with just about all conventions.
Reasonable.
2. Currently, a custom parser can pass two kinds of information; either "I do not recognize this option" or "I recognize this option, and here is a name/value pair that I want you to use as a substitute for the option." It needs to be enhanced so it can also pass "I recognize this option, and you should ignore it."
You could have the parser, custom or otherwise, call an unrecognized_argument function that, by default, throws an exception, but could be customized. unrecognized_argument() could collect such arguments and maintain state, as desired, to act when it has enough information. You could even call unrecognized_argument() with an object from which it can extract arguments such that those arguments are no longer visible to the parser. Another approach is for the parser to use the decorator pattern such that passing a custom parser doesn't preclude owning and forwarding to a default parser. Then, the custom parser can let the default parser try its hand and, if the default parser fails to recognize the argument, the customer parser can give it a go. If both fail, then the framework can cry foul.
3. Some way for the main cmdline parser to do all its parsing, and return all "unrecognized" stuff, without an error. This way, parsers can be chained. For example, parsing in such a manner... program --something 10 --known_option would resognize --known_option and I can then as for "parsed" options and get a vector of strings representing the parsed options. I should also be able to get a vector of strings of "unparsed" options.
Providing a callback, like unrecognized_argument(), leaves to the client whether the right behavior is to collect them in some container. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;
On Wed, 24 Nov 2004 10:27:59 +0300, Vladimir Prus <ghost@cs.msu.su> wrote:
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do you think?
I think it is unreasonable to expect all command lines to be specified in this way. If --something expects an argument, then 10 is that argument. If --something is a flag then its either an unknown argument or (if you allow argument vector rearranging ala GNU getopt) it is the first positional parameter. Allowing unknown options to be ignored is a very desirable feature, though it should be disabled by default. -- Caleb Epstein caleb dot epstein at gmail dot com
Caleb Epstein wrote:
On Wed, 24 Nov 2004 10:27:59 +0300, Vladimir Prus <ghost@cs.msu.su> wrote:
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do you think?
I think it is unreasonable to expect all command lines to be specified in this way. If --something expects an argument, then 10 is that argument.
Well, I obvious don't know if an unknown option expects an argument ;-)
If --something is a flag then its either an unknown argument or (if you allow argument vector rearranging ala GNU getopt) it is the first positional parameter.
While it's possible to consider all unknown options to be positional options, this still leaves the question what to do with "10". Should it be passed to the application as position option?
Allowing unknown options to be ignored is a very desirable feature, though it should be disabled by default.
I just don't see any solution to make unknown options work without either -- requiring --something=10 format -- requiring that all positional options are at the end - Volodya
On Wed, 24 Nov 2004 17:14:03 +0300, Vladimir Prus <ghost@cs.msu.su> wrote:
Well, I obvious don't know if an unknown option expects an argument ;-)
While it's possible to consider all unknown options to be positional options, this still leaves the question what to do with "10". Should it be passed to the application as position option?
Allowing unknown options to be ignored is a very desirable feature, though it should be disabled by default.
I just don't see any solution to make unknown options work without either -- requiring --something=10 format -- requiring that all positional options are at the end
What about the combination of (if some user-settable switch is thrown, but not by default): * allowing unknown options -- these are considered positional parameters * rearranging the argument list such that all positional parameters are moved to the end This way: program --unknown 42 --known-flag --known-arg value is handled as if it were (in standard UNIX command-line-ese): program --known-flag --known-arg value -- --unknown 42 -- Caleb Epstein caleb dot epstein at gmail dot com
Caleb Epstein wrote:
I just don't see any solution to make unknown options work without either -- requiring --something=10 format -- requiring that all positional options are at the end
What about the combination of (if some user-settable switch is thrown, but not by default):
* allowing unknown options -- these are considered positional parameters * rearranging the argument list such that all positional parameters are moved to the end
This way:
program --unknown 42 --known-flag --known-arg value
is handled as if it were (in standard UNIX command-line-ese):
program --known-flag --known-arg value -- --unknown 42
Looks possible. I'll add this suggestion to my todo. - Volodya
On Wed, 24 Nov 2004 17:14:03 +0300 Vladimir Prus <ghost@cs.msu.su> wrote:
Caleb Epstein wrote:
On Wed, 24 Nov 2004 10:27:59 +0300, Vladimir Prus <ghost@cs.msu.su> wrote:
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove> the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do> you think?
I think it is unreasonable to expect all command lines to be specified in this way. If --something expects an argument, then 10 is that argument.
Well, I obvious don't know if an unknown option expects an argument ;-)
Right, but the custom parser better. It should hold state and know what is supposed to come next. However, it can only do this if the custom parser is allowed to eat up the option, leaving the "driver" with nothing to do at all for that option (like a Jedi mind trick, "This is not the option you are looking for.") Imagine a custom parser written like this... enum action_t { a_none, a_modified, a_comsumed }; action_t action = a_none; switch (state_) { case s_idle: if (option == "--something") { state_ = s_something; action = a_consumed; } break; case s_something: { action = a_consumed; char * p; long val = strtol(option.c_str(), &p, 0); if (/*something is valid*/) { state_ = s_idle; } else { // I expected a value for "something" // What do I do here? There is no way to return an error // to the driver caller. Should I throw an exception? } } break; ... } switch (action) { case a_none: // return something saying that I did nothing with this option break; case a_modified: // return a name/value tuple, similar to the current interface break; case a_consumed: // return something saying that this option has been // consumed, and the driver should pretend like it // never even existed. break; }
If --something is a flag then its either an unknown argument or (if you allow argument vector rearranging ala GNU getopt) it is the first positional parameter.
While it's possible to consider all unknown options to be positional options, this still leaves the question what to do with "10". Should it be passed to the application as position option?
If you want to allow positional options to be mixed with non-positional options, then yes, it would be treated as a positional option.
Allowing unknown options to be ignored is a very desirable feature, though it should be disabled by default.
I just don't see any solution to make unknown options work without either-- requiring --something=10 format -- requiring that all positional options are at the end
If the custom parser can actually "consume" options, then the "10" would not be left over at all... I am afraid we have a misunderstanding somewhere...
Hi Jody,
I think it is unreasonable to expect all command lines to be specified in this way. If --something expects an argument, then 10 is that argument.
Well, I obvious don't know if an unknown option expects an argument ;-)
Right, but the custom parser better.
Ok, so you'd like to call your existing parser from inside custom parser that program_options provide? That would indeed solve some of the problems.
It should hold state and know what is supposed to come next. However, it can only do this if the custom parser is allowed to eat up the option, leaving the "driver" with nothing to do at all for that option (like a Jedi mind trick, "This is not the option you are looking for.")
I agree. I already plan that a future version will allow custom parser to return a vector of options, so you can return 0, 1, or many options, while eating as many tokens as you like. I'm afraid I don't know when it will be implemented. "This year" is as accurate estimate as I can provide. - Volodya
Vladimir Prus wrote:
Hi Jody, Hi Allen,
Allen Bierbaum wrote:
Is there any way to tell the program options parser to ignore options that it does not recognize? If there is no option like this, how can program_options be used with other option parsers?
Jody wrote:
I am not familiar enough with the library to know the "correct" method. The only thing I can think of is that the interface is lacking. I took a few minutes to look at the code in libs/program_options/src/cmdline.cpp and it looks like the only interpretation is empty or a valid option name.
Ok, let me start with a caveat: ignoring unknown options can be ambiguous:
program --something 10 --known_option
Now, is "10" a value of unknown option or positional option? To remove the problems all unknown options must use this form:
--something=10
which may be problem, or may not, depending on your situation. What do you think?
I recognized that this could be a problem and I think I can work around it. One option I had thought of and have seen in other option parsing libraries is to have the parser method modify the arguments passed and remove the arguments it recognizes. This would be able to handle cases like "--something 10" above if the parser for that options was called first, but I don't really like the idea of modifying the argument list. If it was done it should definitely be optional.
So, at this stage, I think it requires an interface change to support this option.
The detail::cmdline constructor actually has 'allow_unregisted' parameter, which is intended to help with this. However, no non-detail interface allows to specify. Frankly, I don't know if I forgot about this or just decided to wait for a user to request this feature explicitly.
I ran across 'allow_unregistered' and wondered if it would do the trick. I just couldn't figure out how to use it since there is no non-detail interface for using it directly. How could I use this ability through the current interfaces (even if I have to poke into the detail interface a bit)? (I am restricted to boost 1.32 for this project) For example what would I change the following code to in order to make it use allow_unregistered? po::options_description desc("Allowed options"); desc.add_options() ("help", "produce help message") ("my_opt", po::value<int>(), "set my option") ; po::variables_map vm; po::store(po::parse_command_line(ac, av, desc), vm); po::notify(vm);
Adding a new method to the 'common_command_line_parser' to set 'allow_unregistered' flag and then tweaking common_command_line_parser::run should do the trick? Opinions?
That is where I expected to see it. That doesn't mean it is the right design, but it was the expected design. -Allen
- Volodya
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (5)
-
Allen Bierbaum -
Caleb Epstein -
Jody Hagins -
Rob Stewart -
Vladimir Prus