[Program Options] terminate parsing after first positional argument

Hello,
How can I tell Boost Program Options to stop parsing the arguments as soon as it finds a positional argument?
I'd like to be able to parse the argument list "-v chmod -R -v 600 foo" as having one argument for my program (-v) and the rest as a std::vector of std::string (including the -v).
In the attached program I have tried 2 different approaches. The first uses multitoken with the arguments "--command date -u -v", it fails because "-u" isn't known.
The second uses collect_unrecognized with "chmod -R -v 600", it doesn't work because -v is interpreted by boost and doesn't end up in the vector.
As someone said back in 2005, perhaps I'm trying to write xargs:
http://thread.gmane.org/gmane.comp.lib.boost.user/9529/focus=9641
I know that I can ask my users to write
a.out -v -- chmod -R -v 600 foo
but I don't want to do that. I want them to write "a.out -v chmod -R -v 600 foo". The first positional argument should should tell boost to stop processing.
Another slightly different question. If I have a multitoken option that takes doubles how can I get it to accept negative values? If I give my program "--floats 1 8 -4 6" where --floats starts a multitoken option that accepts double, I get an exception saying that "-4" is an unknown option.
thanks
Stuart
This is what my test program gives me with boost 1.50:
argv: dd --command date -u -v
unrecognised option '-u'
Allowed options:
-v [ --verbose ] verbose
--command arg command to run
2nd try
argv: dd chmod -R -v 600
I'm verbose
v: chmod -R 600
3rd try
argv: dd --floats 1 8 -4 6
unrecognised option '-4'
Allowed options:
-v [ --verbose ] verbose
-f [ --floats ] arg numbers
and my program (formatted to fit in 80 characters)
#include

On 01/-10/-28163 01:59 PM, Stuart Pook wrote:
Hello,
How can I tell Boost Program Options to stop parsing the arguments as soon as it finds a positional argument?
I asked this question a few weeks ago and filed a feature request: https://svn.boost.org/trac/boost/ticket/6991 There is a patch attached to the bug report which seems to work for me, however, you of course will have to use a modified version of Boost. (And it's a compiled library so more than just header changes as well.) I don't know the best way to get that option from the client code down to where it's needed, so right now choosing that behavior or the default one is done by changing the Boost code and recompiling. :-) (Incidentally, from my biased perspective I'd say that the fact that there were two independent requests for this feature inside a month, not to mention the old one linked in the quoted post which sites another a couple weeks prior to *that*, speaks to the importance of this feature. I'd like to reiterate my offer to help with any of documentation, tests (if someone can sketch out how to add one), and implementation (if someone sketches out a high-level implementation).) Evan

On 04.07.2012 20:37, Evan Driscoll wrote:
On 01/-10/-28163 01:59 PM, Stuart Pook wrote:
Hello,
How can I tell Boost Program Options to stop parsing the arguments as soon as it finds a positional argument?
I asked this question a few weeks ago and filed a feature request: https://svn.boost.org/trac/boost/ticket/6991
There is a patch attached to the bug report which seems to work for me, however, you of course will have to use a modified version of Boost. (And it's a compiled library so more than just header changes as well.)
I don't know the best way to get that option from the client code down to where it's needed, so right now choosing that behavior or the default one is done by changing the Boost code and recompiling. :-)
(Incidentally, from my biased perspective I'd say that the fact that there were two independent requests for this feature inside a month, not to mention the old one linked in the quoted post which sites another a couple weeks prior to *that*, speaks to the importance of this feature. I'd like to reiterate my offer to help with any of documentation, tests (if someone can sketch out how to add one), and implementation (if someone sketches out a high-level implementation).)
Evan, that patch of yours looks perfectly reasonable to me. It would seem that a style option, e.g. 'allow_intermixed' can be used to control this behaviour (with the current behaviour as default). As for testing, you can add a new function to tests/positional_options_test.cpp, and call it from main. The structure of existing tests should be straightfoward. For avoidance of doubt, if you come up with a patch along these lines, I'll be happy to commit it. Thanks, Volodya

On 01/-10/-28163 01:59 PM, Vladimir Prus wrote:
that patch of yours looks perfectly reasonable to me. It would seem that a style option, e.g. 'allow_intermixed' can be used to control this behaviour (with the current behaviour as default).
Can I just run the plan by you then? Actually looks even a little simpler than I was thinking. The docs don't discuss the style options really. Filenames are relative to the program_options directories: - cmdline.hpp Add a new value to 'enum style_t'. (I'll use 'allow_interspersed' if there's no objection) - src/cmdline.cpp and details/cmdline.hpp Take my patch from before and change "false /* or true for the behavior I want */" to "m_style & allow_interspersed" Then how do I use it? Do something like command_line_parser(argc, argv) .style(default_style | allow_interspersed) .run() ? Evan
As for testing, you can add a new function to tests/positional_options_test.cpp, and call it from main. The structure of existing tests should be straightfoward.
For avoidance of doubt, if you come up with a patch along these lines, I'll be happy to commit it.
Thanks, Volodya

On 07/05/2012 06:07 PM, Evan Driscoll wrote:
On 01/-10/-28163 01:59 PM, Vladimir Prus wrote:
that patch of yours looks perfectly reasonable to me. It would seem that a style option, e.g. 'allow_intermixed' can be used to control this behaviour (with the current behaviour as default).
Can I just run the plan by you then?
OK, this is not as clear as I was thinking. (Surprise surprise!) As phrased, this is a backwards-incompatible change. If a client of the library builds their own style_t value from scratch (as opposed to starting from default_style and setting/unsetting bits in that), they'll not set allow_interspersed, and lose interspersed arguments. This was illustrated by the fact that I had to modify three style_t declarations in cmdline_test.cpp to fix test cases that I broke. (Lines 368, 397, and 412 or thereabouts.) If I just pass a literal false as in my original patch, everything is peachy in the unmodified test cases. (I'm not sure I understand why the test cases were failing, but that's probably just because I don't have a holistic enough view of how the test cases are written.) How should I handle this? Cleanest way I can think of that doesn't break compatibility is to add a 'allow_interspersed' style option instead with the opposite sense, but everything else is 'allow_*'. Evan

On 07/05/2012 06:54 PM, Evan Driscoll wrote:
How should I handle this? Cleanest way I can think of that doesn't break compatibility is to add a 'allow_interspersed' style option instead with the opposite sense, but everything else is 'allow_*'.
Sorry, that should say "add a 'disallow_interspersed' style option"

One more post to this thread for now then I'm done for a bit. :-) I implemented both versions (allow_interspersed and disallow_interspersed) with a test. Patches for each are now up at the issue tracker. https://svn.boost.org/trac/boost/ticket/6991 Evan On 07/05/2012 06:07 PM, Evan Driscoll wrote:
On 01/-10/-28163 01:59 PM, Vladimir Prus wrote:
that patch of yours looks perfectly reasonable to me. It would seem that a style option, e.g. 'allow_intermixed' can be used to control this behaviour (with the current behaviour as default).
Can I just run the plan by you then? Actually looks even a little simpler than I was thinking. The docs don't discuss the style options really. Filenames are relative to the program_options directories:
- cmdline.hpp
Add a new value to 'enum style_t'. (I'll use 'allow_interspersed' if there's no objection)
- src/cmdline.cpp and details/cmdline.hpp
Take my patch from before and change "false /* or true for the behavior I want */" to "m_style & allow_interspersed"
Then how do I use it? Do something like
command_line_parser(argc, argv) .style(default_style | allow_interspersed) .run() ?
Evan
As for testing, you can add a new function to tests/positional_options_test.cpp, and call it from main. The structure of existing tests should be straightfoward.
For avoidance of doubt, if you come up with a patch along these lines, I'll be happy to commit it.
Thanks, Volodya

On 06.07.2012 04:25, Evan Driscoll wrote:
One more post to this thread for now then I'm done for a bit. :-)
I implemented both versions (allow_interspersed and disallow_interspersed) with a test. Patches for each are now up at the issue tracker. https://svn.boost.org/trac/boost/ticket/6991
Evan, I was thinking about option called allow_interspersed with a default value of '1' -- so that you'd have to remove it from default style if you don't want it. Does this seem reasonable? I really would not like backward-incompatible changes. - Volodya

On 06.07.2012 04:25, Evan Driscoll wrote:
One more post to this thread for now then I'm done for a bit. :-)
I implemented both versions (allow_interspersed and disallow_interspersed) with a test. Patches for each are now up at the issue tracker. https://svn.boost.org/trac/boost/ticket/6991
Evan,
I was thinking about option called allow_interspersed with a default value of '1' -- so that you'd have to remove it from default style if you don't want it. Does this seem reasonable?
Yes, but I don't see how to get it in a consistent way. (Well, not *entirely* true: see #4 below.) When you said "style" in your last email, it reminded me that there's a style_t enum defined in cmdline.hpp [1]. This enum is of the "define values with one bit set, so that people can or them together as a bitmask" variety, and has things like "allow long options" and "use Windows-style /options" and "-abc is like -a -b -c". What I thought you were suggesting, and what seems reasonable, was to add a new style flag for interspersed arguments, with the next bit set. But the problem is that I can't make it default to 1 if I do that, because people can | together flags to form their own style and ignore the new one, and of course then the allow_interspersed bit wouldn't be set. (This is what the test cases do.) If someone says style_t style = allow_short | allow_slash_for_short I can't automatically go and set allow_interspersed for them as well. This means that, unless I'm missing something, it seems like we have these options (#2 is my favorite): 1. Do it anyway and break compatibility (probably only bad for a very very small number of people -- but it will be *very* bad for them) 2. Implement the reverse flag, disallow_interspersed, which works out fine but is opposite all the other style_t flags (not great but not awful) 3. Implement this via a different mechanism. There *is* a bit of precedence for this with the parser's allow_unregistered() function [2], which sets a flag in the parser somewhere and eventually it works its way down where it's checked from the cmdline::finish_option function [3]. However, we still have to decide on the API. It shouldn't just be cmdline::allow_interspersed() like allow_unregistered(), because interspersed arguments will already be allowed. So it could either take a boolean, and then I'd call it like 'command_line_parser(argc, argv).allow_interspersed(false).run()', or we could call it disabled_interspersed(). (A reasonable choice but even more inconsistent with the other style options than #2 was, and I like that one more. allow_interspersed has more in common with the other style_t flags than it does with allow_unregistered(), and the new API wouldn't even exactly mirror allow_unregistered() anyway unless that were changed as well.) 4. (The "a bit of work" choice.) Stop using an enum. Create (or use?) a class that well let you | together individual values like people are doing now, but which will start with the allow_interspersed option set (because now we get a constructor). (This would be a fair bit of work, probably a little error-prone, and to provide full-backwards compatibility would probably have to provide conversions from 'int' and other ugly stuff like that. In some ways it's the nicest option, but I'm still not in favor.) 5. (Ugly hack.) Redefine all the other style options so that the allow_interspersed bit is set as well. E.g. 'allow_long = 1 | allow_interspersed'. (This is offensively bad and I'm kind of ashamed I even thought of it. But it is *technically* an option.) [1] http://www.boost.org/doc/libs/1_50_0/boost/program_options/cmdline.hpp [2] http://www.boost.org/doc/libs/1_50_0/doc/html/program_options/howto.html#id2... [3] http://svn.boost.org/svn/boost/trunk/libs/program_options/src/cmdline.cpp about 2/3 of the way down Evan

I know I keep spamming the list, but the last email was pretty long and I think the key point was buried, which was this:
If someone says style_t style = allow_short | allow_slash_for_short I can't automatically go and set allow_interspersed for them as well.
And this is why I suggest going with the inverse flag: disallow_interspersed, because then it shouldn't be set from the start. (That's my option #2, and one of the patches on the bug report.) Evan

[I was away for a couple weeks shortly after sending the message below so I didn't push very hard at the time, but now that I'm back I figured I'd try to revive this discussion.] To try to reset the context, I and others want the ability to stop argument parsing at the first positional argument, so that it's possible to write a program with a syntax like xargs/valgrind/many other tools. The ideal would be to have an 'allow_interspersed' style option which would be set by default (to give the now-usual behavior of "cmd arg --opt" being the same as "cmd --opt arg") but which a client program could unset if they wanted the xargs-like behavior. However, I don't see how to do this, because
If someone says style_t style = allow_short | allow_slash_for_short I can't automatically go and set allow_interspersed for them as well.
And so without some ugly and not-worth-it coding tricks (changing style_t to a class), unless I'm missing something you can't pick this ideal without breaking backwards compatibility. So I instead suggested a new 'disallow_interspersed' style option, which would be off by default (keeping backwards compatibility) but you could set it to get xargs-like behavior, but this is inconsistent with all the other style flags which are allow_xxx. So what do you want to do regarding this issue? References: Trac issue: https://svn.boost.org/trac/boost/ticket/6991 (includes a patch for both of those options) The previous messages in this discussion: http://lists.boost.org/boost-users/2012/07/75100.php Evan

Following with interest from the side lines ... driscoll@cs.wisc.edu writes:
3. Implement this via a different mechanism. There *is* a bit of precedence for this with the parser's allow_unregistered() function [2], which sets a flag in the parser somewhere and eventually it works its way down where it's checked from the cmdline::finish_option function [3].
However, we still have to decide on the API. It shouldn't just be cmdline::allow_interspersed() like allow_unregistered(), because interspersed arguments will already be allowed. So it could either take a boolean, and then I'd call it like 'command_line_parser(argc, argv).allow_interspersed(false).run()', or we could call it disabled_interspersed().
(A reasonable choice but even more inconsistent with the other style options than #2 was, and I like that one more. allow_interspersed has more in common with the other style_t flags than it does with allow_unregistered(), and the new API wouldn't even exactly mirror allow_unregistered() anyway unless that were changed as well.)
Interspersed options/arguments are unregistered, aren't they? Unless I misunderstood, the only thing you want to change is what happens when you encounter the first unregistered option/argument. Why not let that member function accept a flag (defaulting to false for backward compatibility) that controls that? What allow_unregistered() does now is in effect move every registered option before the first unregistered one. AFAIU, this is what you don't want, right? When the parser sees the first unregistered option it can just stop parsing. Right now, it seems to be going out of its way to find more recognizable options. Hope this helps, -- Olaf Meeuwissen, LPIC-2 FLOSS Engineer -- AVASYS CORPORATION FSF Associate Member #1962 Help support software freedom http://www.fsf.org/jf?referrer=1962

Olaf Meeuwissen writes:
Interspersed options/arguments are unregistered, aren't they?
No, not necessarily. Consider foo --help foo target_program --help or (going full inception) foo foo --any-flag which is why
When the parser sees the first unregistered option it can just stop parsing.
isn't a good solution (though much better than the current state): parsing really needs to stop at the first *positional* argument, not at the first unrecognized option. Evan

On 06/07/2012 11:36, driscoll@cs.wisc.edu wrote:
parsing really needs to stop at the first *positional* argument, not at the first unrecognized option.
yes, my_program target_program --help must show the help from target_program not from my_program Stuart

driscoll@cs.wisc.edu writes:
Olaf Meeuwissen writes:
Interspersed options/arguments are unregistered, aren't they?
No, not necessarily. Consider foo --help foo target_program --help or (going full inception) foo foo --any-flag
which is why
When the parser sees the first unregistered option it can just stop parsing.
isn't a good solution (though much better than the current state): parsing really needs to stop at the first *positional* argument, not at the first unrecognized option.
Point taken, but I think there may be a need for *both* behaviours. My use case would definitely become easier to implement if the command line parser could be instructed to stop at the first occurence of either a positional argument or an unrecognized option. So far I've been getting by with not registering the positional argument and reverting state for all arguments recognized after the first one that was not recognized. Hope this helps, -- Olaf Meeuwissen, LPIC-2 FLOSS Engineer -- AVASYS CORPORATION FSF Associate Member #1962 Help support software freedom http://www.fsf.org/jf?referrer=1962

On 04/07/2012 18:37, Evan Driscoll wrote:
On 01/-10/-28163 01:59 PM, Stuart Pook wrote:
that's a weird date!
How can I tell Boost Program Options to stop parsing the arguments as soon as it finds a positional argument? I asked this question a few weeks ago and filed a feature request: https://svn.boost.org/trac/boost/ticket/6991
I guess that the short answer is that you cannot. You cannot write env(1), sudo(1), xargs(1) ... using boost::program_options Thanks for your answer. I'm sorry that I missed your question and spent so long looking for a solution :-( I rewrote my program using getopts. I lost the long option names but I got the behaviour I need: the parsing stops as soon it finds a non-option argument. I can now write: gaingroup -g w-navicell-gp date -u instead of gaingroup -g w-navicell-gp -- date -u for (int opt; (opt = getopt(argc, argv, "+hqivg:")) != -1; ) { switch (opt) { case 'h': std::cout << desc; exit(0); break; case 'v': *verbose = true; break; case 'i': *ignore = true; break; case 'q': *quiet = true; break; case 'g': *group = optarg; break; default: /* '?' */ std::cerr << desc; exit(exit_codes::usage); } } *command = argv + optind; Stuart

Hello everyone, Stuart Pook wrote
Hello,
How can I tell Boost Program Options to stop parsing the arguments as soon as it finds a positional argument?
I was struggling with the same issue, and in the end I found a solution that does not involve any patch to program_options. My solution uses the (undocumented but public) command_line_parser::extra_style_parser() method. You have to define the following style parser: and then register it to your instance of command_line_parser with extra_style_parser. In my code, this yields: Note that the solution requires you to define the positional arguments with .positional(), as usual. In fact, the function I showed above mimics the built-in one that takes care of the '--', with some tweaks. Cheers, Sébastien -- View this message in context: http://boost.2283326.n4.nabble.com/Program-Options-terminate-parsing-after-f... Sent from the Boost - Users mailing list archive at Nabble.com.
participants (6)
-
driscoll@cs.wisc.edu
-
Evan Driscoll
-
Olaf Meeuwissen
-
sjrd
-
Stuart Pook
-
Vladimir Prus