Do really property_tree and program_options overlap?
Hi Everybody, I see there is some confusion about possible overlap of program_options and property_tree libraries. First of all, my opinion is there is very little overlap. The biggest difference between the libraries is that property_tree is hierarchical and program_options is linear - i.e. options are accessible through a map-like interface. Structure of file formats, like XML, JSON etc. is inherently hierarchical. The presence of hierarchy is their greatest strength, and most important feature. Large amount of information is conveyed by structure of the tree alone, not only by values it contains. The same value in two different branches of the tree can have completely different meaning. Contrary to that, in program_options library there is only at best a notion of "positional options", which is completely different from hierarchy. Briefly, PO library operates on a flat strings of options, property_tree is a DOM. Think of property_tree like a replacement for Microsoft XML parser, or JSON API, or Windows INI API, or registry API. Do you imagine using program_options to manipulate XML trees, like the ones you get from XML parser? It is not suited for that, its problem domain lies in completely different place, namely parsing a linear string of (command-line) options, possibly supplied as a config file or files. In addition to that, PO library does not support writing the structures back to config files. It is not a flaw with the library, it just does not need to do it, because options for programs are meant to be "one shot", or "read and forget", like command line arguments. There's not need to store them back where they came from. On the other hand, PO library supports many things that do not belong to property_tree. These include, for example, options descriptions and notify mechanisms. Taking all that into account I do not see why anybody insists there might exist 1:1 correspondence between the libraries. I agree both can parse command lines, but that's about it. Other functionality is very different. One has to distinguish between reading command-line options (or other simple configuration) and reading structured DOM data. For example, take a program like visual XML editor. It could use both PO and property_tree to read its startup config (keyboard shortcuts, font etc.). But only property_tree would be suitable to handle XML files to be parsed, displayed, edited and saved by the editor. It would be rather ridiculous to use program_options to do that, although not impossible, if one tried really hard. Best regards, Marcin
Marcin Kalicinski wrote:
Briefly, PO library operates on a flat strings of options, property_tree is a DOM. Think of property_tree like a replacement for Microsoft XML parser, or JSON API, or Windows INI API, or registry API. Do you imagine using program_options to manipulate XML trees, like the ones you get from XML parser? It is not suited for that, its problem domain lies in completely different place, namely parsing a linear string of (command-line) options, possibly supplied as a config file or files.
Well, this is generally wrong. The program_options goal is to obtain named runtime options from whatever source you can store them. I hope to add Win32 Registry or LDAP support in future. Note also that options *are* hierarchical, you can use "your_app.window1.button15.width" as option name, if you're so inclined.
In addition to that, PO library does not support writing the structures back to config files. It is not a flaw with the library, it just does not need to do it, because options for programs are meant to be "one shot", or "read and forget", like command line arguments. There's not need to store them back where they came from.
This *is* a flaw with the library, and this will hopefully be fixed.
Taking all that into account I do not see why anybody insists there might exist 1:1 correspondence between the libraries. I agree both can parse command lines, but that's about it. Other functionality is very different.
I don't know how suggested 1:1 correspondence, but it's crucial for both library to work together in a sane fashion. Let me describe the primary use case. At the moment, boost::program_options is well suited for reading simple values from command line and config files. In future, it will also be able to write config files and read/write other config sources (again, LDAP comes to mind). However, the hierarchical naming of options is not scalable. Right at the moment, I'm working on debugger component of an IDE, and there's breakpoint window. It has a number of breakpoints, and each breakpoint has a bunch of other data. Narually, the state of the window should be saved in a config file. It's technically possible to write it using option names like: kdevelop.debugger.breakpoints.breakpoint1.tracing.expression1=i but generating such structure is pretty cumbersome, because there's no interface to work with hierarchical options. DOM (or property_tree!) is such an interface. To put it other way, I think it would be worthwhile to either: - replace the variables_map class from program_options with property_tree - make variables_map::operator[] return property_tree Of course, both solution require that property_tree make it possible to: - Access any node with a fully-qualifed name - Allow to store values of any type as nodes. The first requirement is already met. The second seems not -- the property_tree docs say that operator>> is used by the 'get' methods, while program_options used strong typing. I would say the second requirements is hard one for program_options, so would hope property_tree is adjusted to make this possible.
One has to distinguish between reading command-line options (or other simple configuration) and reading structured DOM data. For example, take a program like visual XML editor. It could use both PO and property_tree to read its startup config (keyboard shortcuts, font etc.). But only property_tree would be suitable to handle XML files to be parsed, displayed, edited and saved by the editor. It would be rather ridiculous to use program_options to do that, although not impossible, if one tried really hard.
The question is where to draw the line. Reading XML with program_options is not what I'd like to do. Ideally, if somebody wants XML config files he should do this: options_description desc.........; property_tree options; store(options, parse_command_line(desc, .....)); store(options, property_tree::parse_xml_config_file(.....)); So, parsing of XML files is done by property_tree, and merging that data with command line options is done by program_options. I think it's much more reasonable to use program_options for parsing command line, and classic config files. I'm not sure about registry values. - Volodya
Vladimir Prus wrote:
Marcin Kalicinski wrote:
Briefly, PO library operates on a flat strings of options, property_tree is a DOM. Think of property_tree like a replacement for Microsoft XML parser, or JSON API, or Windows INI API, or registry API. Do you imagine using program_options to manipulate XML trees, like the ones you get from XML parser? It is not suited for that, its problem domain lies in completely different place, namely parsing a linear string of (command-line) options, possibly supplied as a config file or files.
Well, this is generally wrong. The program_options goal is to obtain named runtime options from whatever source you can store them. I hope to add Win32 Registry or LDAP support in future. Note also that options *are* hierarchical, you can use "your_app.window1.button15.width" as option name, if you're so inclined.
I haven't used program_options yet. But if I understand correctly both libraries seem to support storing and accessing data with strings that might describe some kind of hierarchy. This seems to be the core idea of both libraries - is this correct? Then it wouldn't matter much what container is used. However a generic tree which can store data hierarchically probably makes most sense. If I understand correctly both libraries could make use of such a class? While a generic tree could store any data we would still need something to read data from a XML file, a INI file, the Windows registry, program options etc. to populate a generic tree. How about a hierarchical iterator? Just like stream iterators they could be coupled with a data source. Source code would look somewhat like this: boost::tree t; copy(ihierarchical_xml_iterator<string>("config.xml"), ihierarchical_xml_iterator<string>(), inserter(t)); copy(ihierarchical_ini_iterator<string>("config.ini"), ihierarchical_ini_iterator<string>(), inserter(t)); copy(ihierarchical_registry_iterator<string>("<registry>"), ihierarchical_registry_iterator<string>(), inserter(t)); copy(ihierarchical_options_iterator<string>(argv), ihierarchical_options_iterator<string>(argv + argc), inserter(t)); This should be flexible, extensible and based on existing practices? Boris
[...]
I haven't used program_options yet. But if I understand correctly both libraries seem to support storing and accessing data with strings that might describe some kind of hierarchy. This seems to be the core idea of both libraries - is this correct?
Then it wouldn't matter much what container is used. However a generic tree which can store data hierarchically probably makes most sense. If I understand correctly both libraries could make use of such a class?
I think generic tree container is material for another library. Whether property_tree should be based on it or not is a matter of internal implementation, and generally of little interest to users. The biggest value of property_tree is in its easy to use interface, that should not be compromised, if at all possible. I have been already reassured in this view by quite many people who took their time to review the library. Best regards, Marcin
Marcin Kalicinski wrote:
[...] The biggest value of property_tree is in its easy to use interface, that should not be compromised, if at all possible. I have been already reassured in this view by quite many people who took their time to review the library.
I was trying to see the big picture: I rather prefer a C++ standard based on a few well-known concepts like containers, iterators, algorithms etc. instead of having a C++ standard with hundreds of components which are tailored for specific needs, collaborate with only a handful of other components and think they provide an easy-to-use interface while all the easy-to-use interfaces make the whole standard less easy-to-use. That said I have used your property tree library myself to read and write a configuration file. It was indeed very easy to use. However it would have been even easier if it was something I had known before like eg. an iterator. For now I will definitely use your property tree library but would appreciate if existing concepts were reused many C++ developers are familiar with. My opinion is that your library should be a part of Boost but should be more generalized in the future. Boris
That said I have used your property tree library myself to read and write a configuration file. It was indeed very easy to use. However it would have been even easier if it was something I had known before like eg. an iterator.
It has iterators. Its interface is as close to std container as possible. It actually (almost) is a standard container - I think the only thing it misses at the moment is max_size() (correct me if I'm wrong). You have find, insert, push_back, the lot. You can use BOOST_FOREACH on it. You can use algorithms on it (just keep in mind it is not sorted by key unless you do that explicitly). Best regards, Marcin
Marcin Kalicinski wrote:
It has iterators. Its interface is as close to std container as possible. It actually (almost) is a standard container - I think the only thing it misses at the moment is max_size() (correct me if I'm wrong). You have find, insert, push_back, the lot. You can use BOOST_FOREACH on it. You can use algorithms on it (just keep in mind it is not sorted by key unless you do that explicitly).
Hi Marcin, I had now a look at the source code: If I understand correctly it is possible to use the read/write functions (like read_xml, read_ini, read_json etc.) with other containers? So there is no tight coupling? Boris
I had now a look at the source code: If I understand correctly it is possible to use the read/write functions (like read_xml, read_ini, read_json etc.) with other containers? So there is no tight coupling?
No, it is not normally possible. read/write functions expect a recursive container, because they need tree-like structure. I made no attempt to generalize the library, so that read/write functions can work with general recursive containers (btw. the whole idea of recursive container is actually quite fuzzy). They require basic_ptree, so there definitely is tight coupling. Best regards, Marcin
Boris wrote:
Marcin Kalicinski wrote:
[...] The biggest value of property_tree is in its easy to use interface, that should not be compromised, if at all possible. I have been already reassured in this view by quite many people who took their time to review the library.
I was trying to see the big picture: I rather prefer a C++ standard based on a few well-known concepts like containers, iterators, algorithms etc. instead of having a C++ standard with hundreds of components which are tailored for specific needs,
Well, I think we need both. Boost.MultiIndex is a great library and can do all kinds of wonderful things. But I would still like to see a bidirectiobal map (boost::bimap) written as a wrapper around it to get an easy and specialized interface. -Thorsten
"Thorsten Ottosen"wrote:
Well, I think we need both. Boost.MultiIndex is a great library and can do all kinds of wonderful things. But I would still like to see a bidirectiobal map (boost::bimap) written as a wrapper around it to get an easy and specialized interface.
bimap is available in libs/multi-index/examples/bimap.cpp. /Pavel
Pavel Vozenilek wrote:
"Thorsten Ottosen"wrote:
Well, I think we need both. Boost.MultiIndex is a great library and can do all kinds of wonderful things. But I would still like to see a bidirectiobal map (boost::bimap) written as a wrapper around it to get an easy and specialized interface.
bimap is available in libs/multi-index/examples/bimap.cpp.
Right, but the real value comes when somebody designs a nice STL-like interface and write docs etc, at least that was my point. -Thorsten
Thorsten Ottosen <thorsten.ottosen@dezide.com> writes:
Pavel Vozenilek wrote:
"Thorsten Ottosen"wrote:
Well, I think we need both. Boost.MultiIndex is a great library and can do all kinds of wonderful things. But I would still like to see a bidirectiobal map (boost::bimap) written as a wrapper around it to get an easy and specialized interface.
bimap is available in libs/multi-index/examples/bimap.cpp.
Right, but the real value comes when somebody designs a nice STL-like interface and write docs etc, at least that was my point.
IMO Thorsten is exactly right. This is precisely the sort of thing that could be added to the library as part of its ongoing maintenance and development (without review, of course). -- Dave Abrahams Boost Consulting www.boost-consulting.com
Hi Vladimir,
Briefly, PO library operates on a flat strings of options, property_tree is a DOM. [...]
Well, this is generally wrong. The program_options goal is to obtain named runtime options from whatever source you can store them. I hope to add Win32 Registry or LDAP support in future. Note also that options *are* hierarchical, you can use "your_app.window1.button15.width" as option name, if you're so inclined.
I will repeat the same argument I already used in discussion with Gennadyi. The fact that PO can handle options in form "a.b.c.d" does not make it hierarchical. How do you manipulate branches in PO, for example how can you take a branch, make a copy of it and attach somewhere else in hierarchy? How do you take one branch and make a separate data structure of it?
[...] require that property_tree make it possible to:
- Access any node with a fully-qualifed name - Allow to store values of any type as nodes.
The first requirement is already met. The second seems not -- the property_tree docs say that operator>> is used by the 'get' methods, while program_options used strong typing.
It's the fault of my docs that you think that operator >> is hardcoded. Of course you can customize it, as well as the data type. Part of the traits is extractor and inserter function objects, which are used to handle the conversions. Library contains an example (custom_data_type.cpp) where std::string is replaced with boost::any, and operator << is replaced with any_cast. You can even use parsers with this combination, as long as you do not put anything else than a string in boost::any (or provide better extractor than just any_cast). And last of all, I do not want to make impression that I try to prove property_tree makes PO library obsolete, or is generally "better" or "worse". I only think they cover different problem domain. And as you pointed out, they could work nicely together. Best regards, Marcin
participants (6)
-
Boris -
David Abrahams -
Marcin Kalicinski -
Pavel Vozenilek -
Thorsten Ottosen -
Vladimir Prus