[property_tree] Parsing JSON
Hi all,
I am currently trying to parse a JSON file, but cannot figure how to achieve
this task (see for instance
http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa/wa/wsSearch?term=sudoku&entity=software&limit=1
).
Here is a piece of code:
#ifndef ITUNES_JSON_STRUCTURE_HPP
#define ITUNES_JSON_STRUCTURE_HPP
#include
Hi -
I don't have the code in front of me, but we had to modify the grammar for
the JSON parser in boost property_tree in order to read top-level groups.
If you're familiar with boost spirit, I think it's not too much work.
Brian
On Mon, Dec 27, 2010 at 11:14 AM, Olivier Tournaire
Hi all,
I am currently trying to parse a JSON file, but cannot figure how to achieve this task (see for instance http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa/wa/wsSearch?term=sudoku&entity=software&limit=1 ). Here is a piece of code:
#ifndef ITUNES_JSON_STRUCTURE_HPP
#define ITUNES_JSON_STRUCTURE_HPP
#include
#include <string>
class itunes_json_structure
{
public:
void load(const std::string &filename);
boost::property_tree::ptree m_ptree;
};
#endif // ITUNES_JASON_STRUCTURE_HPP
#include "itunes_jason_structure.hpp"
#include
#include <iostream>
using namespace std;
void itunes_json_structure::load(const string &filename)
{
using boost::property_tree::ptree;
try
{
// Load the JSON file into the property tree. If reading fails
// (cannot open file, parse error), an exception is thrown.
read_json(filename, m_ptree);
cout << "resultCount = " << m_ptree.get<string>("resultCount") << endl;
ptree all_results = m_ptree.get_child("results");
// HERE!!! How can I read the content of the "results" node????
}
catch(exception &e)
{
throw e;
}
}
I am unable to read the content of the "results" node. Could you please give me some help?
Regards,
Olivier
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Ok, thank you. I am however not familiar with Spirit. I was expected that thé parser works out of thé box. If you hAve a pièce of code it Will be really helpful
Regards
Olivier
Envoyé de mon iPhone
Le 29 déc. 2010 à 02:08, Brian Budge
Hi -
I don't have the code in front of me, but we had to modify the grammar for the JSON parser in boost property_tree in order to read top-level groups. If you're familiar with boost spirit, I think it's not too much work.
Brian
On Mon, Dec 27, 2010 at 11:14 AM, Olivier Tournaire
wrote: Hi all, I am currently trying to parse a JSON file, but cannot figure how to achieve this task (see for instance http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa/wa/wsSearch?term=sudoku&entity=software&limit=1). Here is a piece of code:
#ifndef ITUNES_JSON_STRUCTURE_HPP #define ITUNES_JSON_STRUCTURE_HPP #include
#include <string> class itunes_json_structure { public: void load(const std::string &filename); boost::property_tree::ptree m_ptree; }; #endif // ITUNES_JASON_STRUCTURE_HPP #include "itunes_jason_structure.hpp"
#include
#include <iostream>
using namespace std;
void itunes_json_structure::load(const string &filename)
{
using boost::property_tree::ptree;
try
{
// Load the JSON file into the property tree. If reading fails
// (cannot open file, parse error), an exception is thrown.
read_json(filename, m_ptree);
cout << "resultCount = " << m_ptree.get<string>("resultCount") << endl;
ptree all_results = m_ptree.get_child("results");
// HERE!!! How can I read the content of the "results" node????
}
catch(exception &e)
{
throw e;
}
}
I am unable to read the content of the "results" node. Could you please give me some help?
Regards,
Olivier
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Hi, On Fri, 2010-12-31 at 13:30 +0200, Olivier Tournaire wrote:
Ok, thank you. I am however not familiar with Spirit. I was expected that thé parser works out of thé box. If you hAve a pièce of code it Will be really helpful
The first match is a viable alternative: http://www.google.fi/search?q=json+spirit Native UTF8 support would be a great improvement though. --> Mika Heiskanen
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Native UTF8 support would be a great improvement though.
https://github.com/brycelelbach/prana/tree/master/boost/spirit/home/prana/in... https://github.com/brycelelbach/prana/tree/master/boost/spirit/home/prana/ou... Includes native UTF8 support. Tests are also available in that repository. I have a compiler that compiles the JSON objects in the utree AST into TST lookups. The utree AST can be treated as a boost optional, a boost variant, a boost any, a std::string, an integer, a double, an std::list (with random access, slow, though) and other things. The interface is documented in Spirit from the top of trunk. I will commit the AST json object -> ternary search tree compiler sometime today. P.S. Tests are here: https://github.com/brycelelbach/prana/tree/master/libs/spirit/test/prana - -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0eCrsACgkQO/fqqIuE2t7SygCg3MmHVl0WI4HrpO/wSZugNJOH m/MAoMnlqfgxGSqGvOABLX3tRNUm3Jgq =pgpd -----END PGP SIGNATURE-----
On Fri, 2010-12-31 at 18:54 +0200, Bryce Lelbach wrote:
Native UTF8 support would be a great improvement though.
https://github.com/brycelelbach/prana/tree/master/boost/spirit/home/prana/in...
https://github.com/brycelelbach/prana/tree/master/boost/spirit/home/prana/ou...
I was referring to the json_spirit library, not spirit itself. Looks like I won't be needing json_spirit in the future though, thank you for the information. --> Mika Heiskanen
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Prana isn't a part of Spirit (just an informal project of mine). I keep it in the Spirit namespace for ease of use.
Looks like I won't be needing json_spirit in the future though, thank you for the information.
You're welcome, hope you've found a suitable solution. - -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0eFX4ACgkQO/fqqIuE2t51dwCeNHJGA9E1NrlpIXoLESAV+LE8 1mUAoKKetZ7vF+bJxas0SRXxZhKSKZQ9 =GROl -----END PGP SIGNATURE-----
2010/12/31 Bryce Lelbach
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Native UTF8 support would be a great improvement though.
https://github.com/brycelelbach/prana/tree/master/boost/spirit/home/prana/in...
https://github.com/brycelelbach/prana/tree/master/boost/spirit/home/prana/ou...
Includes native UTF8 support. Tests are also available in that repository. I have a compiler that compiles the JSON objects in the utree AST into TST lookups. The utree AST can be treated as a boost optional, a boost variant, a boost any, a std::string, an integer, a double, an std::list (with random access, slow, though) and other things. The interface is documented in Spirit from the top of trunk.
Thank you Bryce, I am sure this will really help. Hoewever, it seems that prana (focusing on the JSON parser) uses additional spirit headers that I cannot find (boost/spirit/include/support_utree.hpp, boost/spirit/include/support_line_pos_iterator.hpp, ...). Could you please help? Regards, Olivier
I will commit the AST json object -> ternary search tree compiler sometime today.
P.S. Tests are here:
https://github.com/brycelelbach/prana/tree/master/libs/spirit/test/prana
- -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAk0eCrsACgkQO/fqqIuE2t7SygCg3MmHVl0WI4HrpO/wSZugNJOH m/MAoMnlqfgxGSqGvOABLX3tRNUm3Jgq =pgpd -----END PGP SIGNATURE----- _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Thank you Bryce, I am sure this will really help. Hoewever, it seems that prana (focusing on the JSON parser) uses additional spirit >headers that I cannot find (boost/spirit/include/support_utree.hpp, boost/spirit/include/support_line_pos_iterator.hpp, ...). Could you >please help?
I believe prana is based on the trunk version of boost spirit.
2011/1/2 Mathieu -
Thank you Bryce, I am sure this will really help. Hoewever, it seems that prana (focusing on the JSON parser) uses additional spirit >headers that I cannot find (boost/spirit/include/support_utree.hpp, boost/spirit/include/support_line_pos_iterator.hpp, ...). Could you >please help?
I believe prana is based on the trunk version of boost spirit.
yes, missing files seems to be in the trunk. can I easily replace spirit files in my boost 1.44 version or should I recompile all libs from trunk?
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'd advise checking out the trunk from SVN. Prana depends only on header-only
libraries.
If you'd like, I can get you a list of the exact dependencies.
Once Boost 1.46 is released, I'll start a release branch of Prana, too.
On Sun, 2 Jan 2011 16:44:26 +0100
Olivier Tournaire
2011/1/2 Mathieu -
Thank you Bryce, I am sure this will really help. Hoewever, it seems that prana (focusing on the JSON parser) uses additional spirit >headers that I cannot find (boost/spirit/include/support_utree.hpp, boost/spirit/include/support_line_pos_iterator.hpp, ...). Could you >please help?
I believe prana is based on the trunk version of boost spirit.
yes, missing files seems to be in the trunk. can I easily replace spirit files in my boost 1.44 version or should I recompile all libs from trunk?
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
- -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0gqJQACgkQO/fqqIuE2t46xgCdGLtcgPwY0nWZT+jltYSxpP1h 0Z0AoOQrg0WFs57T5JaWqBBxvXYuolYE =evGn -----END PGP SIGNATURE-----
Thank you for your help, it works now. However, I think I really need some
help on how to access the data in my JSON structure (see
http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa/wa/wsSearch?term=sudoku&entity=software&limit=1
).
For instance, how can I access the value associated with the node "
artworkUrl100"?
Best regards,
Olivier
2011/1/2 Bryce Lelbach
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'd advise checking out the trunk from SVN. Prana depends only on header-only libraries.
If you'd like, I can get you a list of the exact dependencies.
Once Boost 1.46 is released, I'll start a release branch of Prana, too.
On Sun, 2 Jan 2011 16:44:26 +0100 Olivier Tournaire
wrote: 2011/1/2 Mathieu -
Thank you Bryce, I am sure this will really help. Hoewever, it seems that prana (focusing on the JSON parser) uses additional spirit >headers that I cannot find (boost/spirit/include/support_utree.hpp, boost/spirit/include/support_line_pos_iterator.hpp, ...). Could you please help?
I believe prana is based on the trunk version of boost spirit.
yes, missing files seems to be in the trunk. can I easily replace spirit files in my boost 1.44 version or should I recompile all libs from trunk?
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
- -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAk0gqJQACgkQO/fqqIuE2t46xgCdGLtcgPwY0nWZT+jltYSxpP1h 0Z0AoOQrg0WFs57T5JaWqBBxvXYuolYE =evGn -----END PGP SIGNATURE----- _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Currently there's one piece missing from my JSON parser in prana, the compiler that turns the utree ast into the JSON object that the user would access. ATM the low level `parse_xxx(in, out)' interface for JSON is done, but this just produces a utree AST. The utree AST is useful for translating to other languages, or for applying generic utree algorithms to the JSON data, but it's not the ideal interface for less involved access. For that, we need a JSON specific data structure (as mentioned above) and compiler that can build said data structure from a utree AST. I've written most of the compiler (it's not lightning fast, but respectable for now, and it can be optimized later). If you're seriously interested in using Prana's JSON parser, perhaps you could help me come up with the syntax and semantics for the interface that would be exposed? How would you like to access that member? A few ideas, (not mutually exclusive): std::ifstream ifs("foo.json", ifstream::in); prana::json_object js; prana::compile_json(ifs, js); // a number of types, such as a std::string, would // work for the first parameter. std::cout << js["results"]["artworkUrl100"] << std::endl; // no range check std::cout << js.at("results").at("artworkUrl100") << std::endl; // if out of range, throws std::cout << js.find_first("artworkUrl100") << std::endl; // null if not found // uri stuff isn't integrated yet, but in a few days you'd also be able to do: prana::uri_parts uri; prana::parse_uri(js["results"]["artworkUrl100"], uri); std::cout << uri.host << std::endl; I'm hoping to finish up the JSON and URI stuff in the next day or so. I'd love to hear some suggestions :). - -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0gu24ACgkQO/fqqIuE2t5S6QCgma4zlA/wp7sy7A6db/IF7VCC 0EkAn12l5qL57yESwcTGQSTu6FPpYzGS =ept9 -----END PGP SIGNATURE-----
Bryce,
2011/1/2 Bryce Lelbach
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Currently there's one piece missing from my JSON parser in prana, the compiler that turns the utree ast into the JSON object that the user would access.
ATM the low level `parse_xxx(in, out)' interface for JSON is done, but this just produces a utree AST. The utree AST is useful for translating to other languages, or for applying generic utree algorithms to the JSON data, but it's not the ideal interface for less involved access. For that, we need a JSON specific data structure (as mentioned above) and compiler that can build said data structure from a utree AST.
OK. So, right now, nothing to acceess the parsed JSON structure. Am I right?
I've written most of the compiler (it's not lightning fast, but respectable for now, and it can be optimized later).
For my particular application, fast running code is not really a requirement.
If you're seriously interested in using Prana's JSON parser, perhaps you could help me come up with the syntax and semantics for the interface that would be exposed? How would you like to access that member? A few ideas, (not mutually exclusive):
std::ifstream ifs("foo.json", ifstream::in);
prana::json_object js;
prana::compile_json(ifs, js); // a number of types, such as a std::string, would // work for the first parameter.
std::cout << js["results"]["artworkUrl100"] << std::endl; // no range check std::cout << js.at("results").at("artworkUrl100") << std::endl; // if out of range, throws std::cout << js.find_first("artworkUrl100") << std::endl; // null if not found
Yes, I like this kind of STL-like access. What about the property_tree
interface:
std::cout << js.get
// uri stuff isn't integrated yet, but in a few days you'd also be able to do:
prana::uri_parts uri;
prana::parse_uri(js["results"]["artworkUrl100"], uri);
std::cout << uri.host << std::endl;
I'm hoping to finish up the JSON and URI stuff in the next day or so. I'd love to hear some suggestions :).
OK. I will thus, impatiently, wait for the next few days. Would love to help you, but as I really have no background with spirit, it is a little bit hard. Regards, Olivier
- -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAk0gu24ACgkQO/fqqIuE2t5S6QCgma4zlA/wp7sy7A6db/IF7VCC 0EkAn12l5qL57yESwcTGQSTu6FPpYzGS =ept9 -----END PGP SIGNATURE----- _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Sun, 2 Jan 2011 20:05:32 +0100
Olivier Tournaire
OK. So, right now, nothing to acceess the parsed JSON structure. Am I right?
I'm almost done with read-only access. ATM a mutable accessor would be a bit more complex to implement.
Yes, I like this kind of STL-like access.
On second thought, I've decided I don't like the STL-like access, because it's dangerous, and/or requires boilerplate code to make nested calls (such as js["results"]["artworkUrl100"]) safe. Instead, I've written a simple Boost.Proto DSEL for access, so accessing the node in question would look like: std::cout << (js/"results"/"artworkUrl100") << std::endl; std::cout << (js/1/4) << std::endl; The json_ir operator/ takes a proto expression and returns a proto terminal. The proto DSEL's expression wrapper has an operator utree and streamable ops. Instead of throwing or seg faulting on a failed chain access (for example, if there was no "results" member), this method of access will always return a utree; if a failure occurs, an invalid_type utree node is returned.
std::cout << js.get
("results.artworkUrl100") << std::endl;
There is no need to do this; as mentioned above, code that requires checking (aka code that code fail or throw) is undesirable. I have no intention of emulating what I view as property tree's failures. The js object's accessor would return a utree node, and the utree class does have a get method, but it is not advisable to use it. It is safer to write a visitor class, because such a class will fail to compile if you do not have a default case (aka `template<class T> result_type operator() (T) const { ... }') or handle every possibility in the visitor. Also, utree is streamable, and has a number of sane operator overloads (for example, arithmetic operators are overloaded so that you can do math with utree arithmetic nodes. Probably, we should overload operator+ for strings or something).
And what about an access via index:
As I mentioned in my follow up email (and in the above example), this will be possible. I suspect to have a complete prototype by the end of the day. Major dependencies: Spirit, MPL, Fusion, Proto, Multi-Index, Variant. - -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0jKFkACgkQ9cB/V3/s9EyjngCeM1eBUhaJTwzBfdLiggWnAp+l wZEAnj66yv93w7n+QhUQaIntf+FiFW66 =k2jk -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I have finished a prototype of this. I haven't tested accessing objects by
zero index yet, but I suspect it will work fine.
I made an example for you, that takes this as an input:
http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa/wa/wsSearch?term=sudoku&entity=software&limit=1
and accesses the results[0].artworkUrl100.
Here it is:
#include
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Forgot to mention, index by position, e.g. js[1][4] for the node you mentioned. JSON objects are actually compiled into a TST, so that particular index is taken from the original utree AST (the compiled map holds reference utree nodes into the actual ast) - -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0gvJAACgkQO/fqqIuE2t5cYQCcCXedtHKW+2bPJAinzetTk4Vv Uz8AoIeZFPF+nIYtJsNbewn5WABK7KqB =yDBj -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 P.S. All of Prana's data format parsers, including the JSON parser, annotate the utree AST with source code location references, albeit primitive ones. The line of any array, object or member_pair can be retrieved with `ast.tag()'; examples of this are demonstrated throughout Prana. - -- Bryce Lelbach aka wash boost-spirit.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAk0eC98ACgkQO/fqqIuE2t5S5ACeMeclQn47DUReNKCxdHaTeLCb AMYAoMhtoXtQ/j+M8gO+yc+UKPqDXT6Y =9uru -----END PGP SIGNATURE-----
participants (6)
-
Brian Budge
-
bruno romano
-
Bryce Lelbach
-
Mathieu -
-
Mika Heiskanen
-
Olivier Tournaire