program_options: Support for open-end options

Hi, It would be great to implement support for open-end options, something like that: item_1 = 23 item_2 = 45 ... item_N = 465 The idea is to to be able to provide following input: configOptions.add_options() ("item_", value<int>(), "items"); now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this? Thank you, Simone

On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
Thank you, Simone
I have a definite use case for the type of open-ended options you're proposing here. It seems you want to get into contact with Vladimir Prus. He's an active member of the mailing list, so you should have no issues getting a response.

On Thu, Dec 27, 2012 at 8:43 AM, Joshua Boyce <raptorfactor@raptorfactor.com
wrote:
On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
Thank you, Simone
I have a definite use case for the type of open-ended options you're proposing here.
It seems you want to get into contact with Vladimir Prus. He's an active member of the mailing list, so you should have no issues getting a response.
I'll echo that. I have had a need for this support (although I worked around it). Sometimes, it isn't good enough to do something like: variable_name = itemA|itemB|itemC|itemD maybe because 'itemA' is actually a string of characters over 80 characters long or requires special characters that make it non-obvious how to separate the items in the list. Although I'd probably prefer a signature that used a sort of tag instead of relying upon the appending '_' character to trigger the list, and I'd prefer to use a regular expression instead of relying upon the ending. For example: namespace po = boost::program_options; configOptions.add_options() ( "item_.*", po::value<int>(), "items", po::regex ); That would let you use some fairly interesting variable names while ensuring they have the right type. You shouldn't be able to use the following signature, though: configOptions.add_options() ( "item_.*", po::value<int>( &my_var ), "items", po::regex ); And I should think it would get very odd to try to put the variables in a single container for those cases where someone skipped a number or something, e.g.: configOptions.add_options() ( "item_.*", po::value< std::vector< int > >( &my_var ), "items", po::regex ); - Trey

On Thu, 2012-12-27 at 09:35 -0500, Joseph Van Riper wrote:
On Thu, Dec 27, 2012 at 8:43 AM, Joshua Boyce <raptorfactor@raptorfactor.com
wrote:
On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
Thank you, Simone
I have a definite use case for the type of open-ended options you're proposing here.
It seems you want to get into contact with Vladimir Prus. He's an active member of the mailing list, so you should have no issues getting a response.
I'll echo that. I have had a need for this support (although I worked around it).
Sometimes, it isn't good enough to do something like:
variable_name = itemA|itemB|itemC|itemD
maybe because 'itemA' is actually a string of characters over 80 characters long or requires special characters that make it non-obvious how to separate the items in the list.
Although I'd probably prefer a signature that used a sort of tag instead of relying upon the appending '_' character to trigger the list, and I'd prefer to use a regular expression instead of relying upon the ending. For example:
namespace po = boost::program_options; configOptions.add_options() ( "item_.*", po::value<int>(), "items", po::regex );
That would let you use some fairly interesting variable names while ensuring they have the right type.
I thought about using RegEx but came to the conclusion that it is unnecessary - you will introduce an extra dependency and implementing it is less easy. Using '_' gives you enough flexibility.
You shouldn't be able to use the following signature, though:
configOptions.add_options() ( "item_.*", po::value<int>( &my_var ), "items", po::regex );
what if my_var is a map?
And I should think it would get very odd to try to put the variables in a single container for those cases where someone skipped a number or something, e.g.:
configOptions.add_options() ( "item_.*", po::value< std::vector< int > >( &my_var ), "items", po::regex );
BTW. I have a case where numbers intentionally can be left out. Vector is not good but map can work. Simone

On 27.12.2012 17:43, Joshua Boyce wrote:
On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
Thank you, Simone
I have a definite use case for the type of open-ended options you're proposing here.
It seems you want to get into contact with Vladimir Prus. He's an active member of the mailing list, so you should have no issues getting a response.
Hi, just a couple of points: - It is generally advisable to CC me personally. I do check this list, but not as often as before - I'll be on public holiday starting really soon and returning on Jan 9, so expect some delay :-( Thanks, Volodya

On Sat, Dec 29, 2012 at 7:50 AM, Vladimir Prus <ghost@cs.msu.su> wrote:
Hi,
just a couple of points:
- It is generally advisable to CC me personally. I do check this list, but not as often as before - I'll be on public holiday starting really soon and returning on Jan 9, so expect some delay :-(
Thanks, Volodya
No worries, thanks for responding. I was going to CC you on my original reply, but I wasn't sure if that was considered rude by some.

On Fri, 2012-12-28 at 00:43 +1100, Joshua Boyce wrote:
On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
I looked into source code and noticed that there is already something like I was suggesting just the special character is not "_" but "*". See (version 1.52): options_description.cpp Line 87 config_file.cpp Line 43 This feature is not documented!? I didn't try it yet, but seemingly the only thing that lacks is the ability to provide a map to such an option that will be auto-populated by PO. Or is it also supported already, just I don't see it? For now I have to itereate through the variables_map in order to pick all those options, right? Volodya - please, tell us what is already there! ST

On Fri, 2012-12-28 at 00:43 +1100, Joshua Boyce wrote:
On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
I checked this and indeed everything worked just fine. Following declaration worked as expected configOptions.add_options() ("item_*", value<int>(), "items"); and recognized item_1, item_3, etc, while rejecting unknown options. Its really sad, that this feature is not mentioned in the documentation - I saw numerous people asking for it on different forums over last years. And also here there were people who need it. Well now the question is, how can I write a custion validate function that will populate a map, like this: map<string, int> itemsMap; options_description generic("Generic options"); generic.add_options() ("item_*",value<map<string, int>>(&itemsMap)->multitoken(), "items") ; My question is - how do I get the key of the option from within validate() function? Thanks in advance, Simone

On Fri, 2012-12-28 at 00:43 +1100, Joshua Boyce wrote:
On Thu, Dec 27, 2012 at 4:56 AM, ST <smntov@gmail.com> wrote:
Hi,
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int . Nesting should also be possible - like this: "item_.subitem_.subsubitem_" (item_3.subitem_FOO.subsubitem_34 should be a valid option key). I can try to implement it, however only if it will have a chance to be merged into the boost library. Whom should I contact regarding this?
Thank you, Simone
I have a definite use case for the type of open-ended options you're proposing here.
It seems you want to get into contact with Vladimir Prus. He's an active member of the mailing list, so you should have no issues getting a response.
He doesn't seem to be that active nowdays. I tried to get in touch with him during last month but without success. I wrote a workaround that works fine for me. So I decided to open 2 new tickets and close the case. Should there be any progress I'll be glad to hear about it. https://svn.boost.org/trac/boost/ticket/7932 https://svn.boost.org/trac/boost/ticket/7933 Regards, Simone

On Dec 26, 2012, at 12:56 PM, ST <smntov@gmail.com> wrote:
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int .
I think that's really odd, though I understand that you want something to allow a great number of options without defining them ahead of time. My first thought was a command line delimiter to introduce a list of such options: ... --foo 1=23 2=45 ... N=465 You'd specify "foo" as the delimiter and PO would then parse the remaining values into pairs to populate a map. Since the set of name/value pairs is open ended, they would always go last on the command line. Of course, you could also specify a delimiter to signal the end of the pairs. (If you want validation that the option names match a pattern, you can add that, too.) To avoid the name/value pairs, the following could be used to populate a vector: ... --foo 23 45 ... 465 ___ Rob

Hi,
just a couple of points:
- It is generally advisable to CC me personally. I do check this list, but not as often as before - I'll be on public holiday starting really soon and returning on Jan 9, so expect some delay :-(
To Volodya: Is there a chance you express your opinion on this issue before 31.12 starts? :) Thank you On Thu, 2012-12-27 at 10:28 -0500, Rob Stewart wrote:
On Dec 26, 2012, at 12:56 PM, ST <smntov@gmail.com> wrote:
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int .
I think that's really odd, though I understand that you want something to allow a great number of options without defining them ahead of time.
Why is it odd? I find it pretty intuitive/flexible and seems to be not that hard to implement.
My first thought was a command line delimiter to introduce a list of such options:
... --foo 1=23 2=45 ... N=465
My way this would be: --foo_1 23 --foo_2 45 ... --foo_N 465
You'd specify "foo" as the delimiter and PO would then parse the remaining values into pairs to populate a map. Since the set of name/value pairs is open ended, they would always go last on the command line.
With my aproach there is no need for this limitation and you can have as many open-ended options as you like.
Of course, you could also specify a delimiter to signal the end of the pairs. (If you want validation that the option names match a pattern, you can add that, too.)
To avoid the name/value pairs, the following could be used to populate a vector:
... --foo 23 45 ... 465
Sometimes one wants to use strings to distinguish between such open-ended options and not numbers(indices of a vector in that case), for example: --os_linux 435 --os_solaris 234 --os_mac 123 In my specific case I need to have the ability to "see" options that were ommitied (on purpose), like: # visible points and their current coordinates [second_123] point_1 = 12 45 1 point_5 = 65 23 23 [second_124] point_1 = 12 47 3 point_4 = 212 565 12 point_7 = 122 453 157 So map is the right option here. Thank you, Simone

On Dec 30, 2012, at 7:07 AM, ST <smntov@gmail.com> wrote:
On Thu, 2012-12-27 at 10:28 -0500, Rob Stewart wrote:
On Dec 26, 2012, at 12:56 PM, ST <smntov@gmail.com> wrote:
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int .
I think that's really odd, though I understand that you want something to allow a great number of options without defining them ahead of time.
Why is it odd?
Well, "odd" is a subjective word, but I'll try anyway. Your approach leads to duplication ("item_") for each option. I also generally like open-ended, or special-cased, lists of things to come last, not unlike C's ellipsis.
I find it pretty intuitive/flexible and seems to be not that hard to implement.
Yours would be somewhat harder to document/specify in my estimation. Implementation difficulty isn't particularly relevant (other than affecting the likelihood of its acceptance).
My first thought was a command line delimiter to introduce a list of such options:
... --foo 1=23 2=45 ... N=465
My way this would be: --foo_1 23 --foo_2 45 ... --foo_N 465
I understood full well. Notice how mine makes the name/value pairs manifest, whereas the names are substrings in yours.
You'd specify "foo" as the delimiter and PO would then parse the remaining values into pairs to populate a map. Since the set of name/value pairs is open ended, they would always go last on the command line.
With my aproach there is no need for this limitation and you can have as many open-ended options as you like.
The part below allows for such a list to not be last, but my expectation is such should normally be last, so the end delimiter won't be needed normally.
Of course, you could also specify a delimiter to signal the end of the pairs. (If you want validation that the option names match a pattern, you can add that, too.)
To avoid the name/value pairs, the following could be used to populate a vector:
... --foo 23 45 ... 465
Sometimes one wants to use strings to distinguish between such open-ended options and not numbers(indices of a vector in that case), for example:
--os_linux 435 --os_solaris 234 --os_mac 123
I understand, but I was illustrating how my approach could be modified slightly (by populating a vector rather than a map) to access the options by index when desired.
In my specific case I need to have the ability to "see" options that were ommitied (on purpose), like:
# visible points and their current coordinates [second_123] point_1 = 12 45 1 point_5 = 65 23 23
[second_124] point_1 = 12 47 3 point_4 = 212 565 12 point_7 = 122 453 157
So map is the right option here.
Of course. I wasn't suggesting to not use a map. However, another option in all of this is to just arrange for PO to collect all unrecognized options in a map keyed by command line position, leaving special case parsing to applications with unusual requirements. IOW, PO would be most extensible with such an ability, regardless of its ever gaining support for anything like you or I have mentioned herein. ___ Rob

On Dec 30, 2012, at 7:07 AM, ST <smntov@gmail.com> wrote:
On Thu, 2012-12-27 at 10:28 -0500, Rob Stewart wrote:
On Dec 26, 2012, at 12:56 PM, ST <smntov@gmail.com> wrote:
It would be great to implement support for open-end options, something like that:
item_1 = 23 item_2 = 45 ... item_N = 465
The idea is to to be able to provide following input:
configOptions.add_options() ("item_", value<int>(), "items");
now if add_options() sees a key that ends with a "_" it accepts all options with keys that start with item_, no matter what comes after it, and treat all of them as int .
I think that's really odd, though I understand that you want something to allow a great number of options without defining them ahead of time.
Why is it odd?
Well, "odd" is a subjective word, but I'll try anyway. Your approach leads to duplication ("item_") for each option. I also generally like open-ended, or special-cased, lists of things to come last, not unlike C's ellipsis.
I find it pretty intuitive/flexible and seems to be not that hard to implement.
Yours would be somewhat harder to document/specify in my estimation.
Implementation difficulty isn't particularly relevant (other than affecting the likelihood of its acceptance).
My first thought was a command line delimiter to introduce a list of such options:
... --foo 1=23 2=45 ... N=465
My way this would be: --foo_1 23 --foo_2 45 ... --foo_N 465
I understood full well. Notice how mine makes the name/value pairs manifest, whereas the names are substrings in yours.
You'd specify "foo" as the delimiter and PO would then parse the remaining values into pairs to populate a map. Since the set of name/value pairs is open ended, they would always go last on the command line.
With my aproach there is no need for this limitation and you can have as many open-ended options as you like.
The part below allows for such a list to not be last, but my expectation is such should normally be last, so the end delimiter won't be needed normally.
Of course, you could also specify a delimiter to signal the end of the pairs. (If you want validation that the option names match a pattern, you can add that, too.)
To avoid the name/value pairs, the following could be used to populate a vector:
... --foo 23 45 ... 465
Sometimes one wants to use strings to distinguish between such open-ended options and not numbers(indices of a vector in that case), for example:
--os_linux 435 --os_solaris 234 --os_mac 123
I understand, but I was illustrating how my approach could be modified slightly (by populating a vector rather than a map) to access the options by index when desired.
In my specific case I need to have the ability to "see" options that were ommitied (on purpose), like:
# visible points and their current coordinates [second_123] point_1 = 12 45 1 point_5 = 65 23 23
[second_124] point_1 = 12 47 3 point_4 = 212 565 12 point_7 = 122 453 157
So map is the right option here.
Of course. I wasn't suggesting to not use a map. However, another option in all of this is to just arrange for PO to collect all unrecognized options in a map keyed by command line position, leaving special case parsing to applications with unusual requirements. IOW, PO would be most extensible with such an ability, regardless of its ever gaining support for anything like you or I have mentioned herein. I thought about it, but the problem with this approach is, that you'll have to parse through the whole map for each group of open-ended
On Sun, 2012-12-30 at 12:37 -0500, Rob Stewart wrote: options. Abilitiy to provide seperate map for each such group, is better IMHO.

On Jan 13, 2013, at 10:39 AM, ST <smntov@gmail.com> wrote:
On Sun, 2012-12-30 at 12:37 -0500, Rob Stewart wrote:
[Please don't overquote.]
However, another option in all of this is to just arrange for PO to collect all unrecognized options in a map keyed by command line position, leaving special case parsing to applications with unusual requirements. IOW, PO would be most extensible with such an ability, regardless of its ever gaining support for anything like you or I have mentioned herein. I thought about it, but the problem with this approach is, that you'll have to parse through the whole map for each group of open-ended options. Abilitiy to provide seperate map for each such group, is better IMHO.
Since command line parsing is done once, performance isn't critical, so your argument cannot be against the overhead of iterating the map. That leaves convenience, which can be addressed by a free function which provides the behavior you want built atop the unrecognized argument map I've suggested. The difference is that numerous schemes can be implemented similarly without further change to PO's implementation. ___ Rob
participants (5)
-
Joseph Van Riper
-
Joshua Boyce
-
Rob Stewart
-
ST
-
Vladimir Prus