- Boost - lists.preview.boost.org

[nowide] cxx11_moveable_fstreams target
by Sean Farrow 24 Sep '20

24 Sep '20

Hi All, By way of context to the following query, I am trying to update the VCPKG boost port to v1.74. I noticed that the nowide library now has a jam file that has the following import statement: import ../../config/checks/config : requires ; Due to the way VCPKG works, I'm copying the config directory to where it is expected and then changing the import statement. But I then get a message that it can't find the cxx11_moveable_fstreams target. Does the nowide build process rely on any other parts of boost config? If not, where is the cxx11_moveable_fstreams defined? I should say that I'm new to the boost build system, so any help is apreciated. Thanks, Sean.

5 9

[review][JSON] The library name...
by Andrzej Krzemienski 24 Sep '20

24 Sep '20

Hi Everyone, Looking at the reviews of Boost.JSON, a number of them fall into a distinctive pattern: reviewers are saying "you did not implement important parts of JSON parsing/serializing", and the authors' reply is "this was not the design goal". My hypothesis is that this is caused by the library name. Simple "Boost.JSON" makes a promise (at least I read it this way) "this is Boost's response to JSON-related problems". And the library does not fulfil that promise. In a sense, the name misguides the potential users who need an answer, sometimes in less than 10 seconds, to the question, "is this library going to address my need?". Maybe the library would encounter a wider consensus if it was called Boost.JSON_Value, or Boost.JSON_DOM. And if the first introductory sentence in the documentation read, "this library provides a type with Regular interface that can represent JSON contents, and provides a convenient way to serialize and then deserialize it fast and possibly in chunks." Alternatively, maybe we can consider the reviewed library as a candidate for a sub-library in a more general module of JSON. Much like Boost.Math aggregates a number of math-related components, we could treat this library as one of a number of components in a more general Boost.JSON_Toolkit library. Regards, &rzej;

1 0

Proposal for adding C++ level to the meta/libraries.json
by Edward Diener 24 Sep '20

24 Sep '20

Since we are reviewing Vinnie Falco's JSON library I thought that this would be a good time to present a proposal to add information to each Boost library's meta/libraries.json file regarding the level of C++ standard compliance for that library. This could then be added to each library's visual documentation so that end-users would instantly know the C++ standard level they would need to use in order to use a Boost library. My proposal is to add 3 fields whose data would be the same as the cxxstd allowed values from Boost Build, using the first value of each choice ( currently 03, 11, 14, 17, 20 ). standard = minimum C++ level for the library extended = same functionality in the library as the minimum C++ level but with extended use given higher C++ levels required = new functionality in the library above the minimum C++ level which requires given higher C++ levels The 'standard' field would be a single value, while the 'extended' or 'required' fields could be more than one comma separated value. If you don't like the names for 'standard', 'extended', or 'required' you can bikeshed the name, although I think the 'standard' name is pretty well apparent. The idea is to provide this information in the meta/libraries.json field for each library, so that the end-user of the library can immediately know the usability of the library with appropriate C++ standard levels of compilation. I realize that the line between the 'extended' and 'required' entries, as I have described them, could be blurred. The 'standard' field remains the most important. I have long felt that the end-user should not have to do any investigation, in code or in documentation, just to determine if a library is usable for his C++ standard level of compilation.

12 27

[review][JSON] Ruthgers' review
by Ruthger Dijt 23 Sep '20

23 Sep '20

> Do you think the library should be accepted as a Boost library? It will be a good addition to Boost if this library gets ACCEPTed > What is your evaluation of the design? It works, is straightforward and the interface feels good to use, the only pitfall is getting the right type you want; number_cast took me a second to realize it existed and the custom string class is something the will most likely be converted alot to std::string in an average users' code. > What is your evaluation of the implementation? As a casual user I cannot comment on this, but as I used it in stand alone / header only mode without problem I feel that i should be alright under the hood > What is your evaluation of the documentation? Could be better, especially in the examples department; it could use how-to basics > What is your evaluation of the potential usefulness of the library? Very useful > Did you try to use the library? With what compiler? Did you have any > problems? I have implemented it in a real project where RapidJSON was used before. No real problems found aside from getting exceptions because .as_double() is not able to get a technically integer from the json (which happens in user configs, example: { "test": 20 }) but that quickly resolved itself with number_cast, Ideally any number should be able to become a double in my opinion. Clang 10.0.1 was used. > How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? As stated above implemented in one project which took about an hour. > Are you knowledgeable about the problem domain? No, this review is coming from a pure casual user kindly, Ruthger Dijt

2 1

[json][review] JSON review
by Harshada Wighe 23 Sep '20

23 Sep '20

We vote to reject the JSON library from Boost and our review of the library and reason for rejection are below: DOCUMENTATION The documentation is very good. The usage and examples and reference are easy to understand and contain all the information we are looking for. IMPLEMENTATION The organization of source code is very messy: 1. Some files do not include all headers at the very start of the file but have some include statements at the end. For example the json/object.hpp file has 10 include statements at the start of the file before the opening namespace and then 1 include statement at the end of the file after the closing namespace. This is very confusing and not a good practice. 2. Splitting the files into HPP and IPP is also not good because it spreads the class into more files than necessary and causes compilation to be slower. Using the library in a project has a complication: 3. The library is complicated to use in a project because you cannot simply include headers in any file you want. You have to include a src.hpp file but this file must be included only once. Normal header libraries in Boost just let you include the headers without this strange requirement. The library adds a lot of size to your executable: 4. Unlike other JSON libraries which are smaller, this adds several KB to the exe size. This is possibly because of use of templates or because it contains very big lookup tables. The performance is not as fast for all use cases: 5. Compared to our in house JSON library, this library was slightly slower. We have very simple but very large JSON data. We use objects and arrays, but only integer numbers with no fractional part and very large ASCII strings, and no booleans and no nulls. In one of our standalone conversion tools we tried using this library and found it had a 4% longer runtime on a small 170 MB file. The parser allocates too much memory: 6. I think this is because the design is not optimized for the case when all the JSON content is already stored in memory. In our parser we point to the strings in the original buffer. The library duplicate code that is in the C++ compiler libraries: 7. In json/config.hpp there are implementations of move and remove_cv from the C++ standard. This introduces more duplicate of this functionality into programs. Besides these points the implementation is well written. DESIGN The allocator design is not standard: 1. The library has a different allocation system that is not the C++ standard system for allocators. We did not understand why storage_ptr was needed at first. It would be better for the library to just template on allocator just like the vector class in the C++ standard. The allocator system also uses virtual calls. The C++ standard vector class also does not have this problem when it uses the standard allocator system. Other than these points the design is ok. OVERALL We would not use this library because it does not offer anything special for JSON that we need. It also uses some designs that are not standard practice. We use JSON heavily in our historical market quote service and most of our tools deal with JSON files and we use many Boost libraries so we are hoping for a JSON library in Boost that we could use instead of having our own.

3 2

[json][review] Scott Determan review
by Scott Determan 23 Sep '20

23 Sep '20

I recommend ACCEPTING the proposed json library. - What is your evaluation of the design? Design is very good. The parser interfaces and object building interfaces meet my needs. I especially want to call out the pmr support and it's support for either sharing or transferring ownership of resources. It's unfortunate that this required a non-standard string class, but I think supporting shared ownership is worth it. I also like the use of tag_invoke for customization points. Apart from libunifex, this is the first library I've seen that uses this mechanism. I think this is the right way to do customization points. - What is your evaluation of the implementation? I looked at the value, storage_ptr, and object classes. It appears to be a very high quality implementation. However, I did this only as a sanity check, I did not spend much time looking at the implementation. The benchmarking numbers also point to a high quality implementation. - What is your evaluation of the documentation? Documentation is very good. Message for Vinnie: One minor nit caught my eye: My understanding is when implementing a tag_invoke customization point it should be a hidden friend because it improves compile times by keeping overload sets smaller. If that is best practice (and it may not be; I'm not a tag_invoke expert), then the examples in the documentation should follow this. - What is your evaluation of the potential usefulness of the library? It covers a widely useful range of json functionality and I expect this library to be widely used. If accepted I intend on using it immediately. - Did you try to use the library? With which compiler(s)? Did you have any problems? Yes: gcc 10.2 and clang 10.0.1; Linux; No problems. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? Read the documentation, browsed the code, wrote some toy programs. Spent about half a day on it. - Are you knowledgeable about the problem domain? As a library consumer, I am familiar with nlohmann_json as well as the custom json library used in my current work project. I am not familiar with the problem domain from a library writer point of view.

2 1

[review][json] Boost.JSON review
by Ruben Perez 23 Sep '20

23 Sep '20

Hi all, My recommendation is to ACCEPT Boost.JSON into Boost. Parsing and serializing JSON using a DOM representation is a common enough task to have a specialized library for it in Boost. Most other languages provide built-in support for this task. I see Boost as the repository where I look at when the standard doesn't provide me something it should, so having a DOM JSON library in Boost is a must for me. - What is your evaluation of the design? Generally sound. I especially value its simplicity. Concretely: 1. The memory management model is extremely simple. I wonder if storage_ptr could be moved somewhere else long term so other libraries can use it. 2. Using concrete types for numbers (uint64_t/int64_t/double) may not fit everyone's needs but will definitely fit most and makes things easy. I miss some way of accessing json::value from generic code, but I understand visitation is currently in progress. As a user I would also be satisfied with generic versions of get_xxx, is_xxx and so on: ```cpp json::value val = /* ... */ val.get<json::object>() // or json::get<json::object>(val) ``` I also miss some way of determining the location (line and column) of parse errors. I don't know how this feature would interact with performance though. Nice to have but not required for me. I would have also liked `json::kind` be streamable, as it makes logging/debugging easier in some contexts. Nothing I can't live without, though. Finally, I would also appreciate `json::string` have a straightforward way to be converted to `std::string` (e.g. by having a `to_std_string()` member function). - What is your evaluation of the implementation? I have just looked at the interface. - What is your evaluation of the documentation? The documentation is very good and detailed, nice work. I would say the main point of improvement would be `basic_parser`, as I had a hard time understanding when some of the callbacks get called, especially the `on_xxx_part` ones. It would be also good to know if accessing the `basic_parser` state (e.g. `depth()`) is allowed from within the handler. And if it is, maybe passing the parser instance at the beginning of the parse could be useful (e.g. in the `on_document_begin`, or in a separate hook). Ideally I would like to see an example of `basic_parser` used to parse a custom struct, if it's not excessively complex. Also note that `basic_parser` docs say that it's defined in `<boost/json/detail/basic_parser.hpp>` and that the convenience header is `<boost/json.hpp>`. Instead, it would be good to note that the user should include `<boost/json/basic_parser.hpp>` and that `<boost/json.hpp>` is not enough (I understand for build speed reasons). I've noted that streaming a `json::string` outputs the string within double quotes. It surprised me at first, as I was working with the 'json::string equals std::string' mentality. I guess this behavior makes sense, but I would make a note in the 'Using strings' section. 'See also' section for https://master.json.cpp.al/json/ref/boost__json__value_from_tag.html renders functions without separation, looks strange. I would make a note in the Value conversion mechanism referencing P1895 on `tag_invoke` so users who don't know this paper don't stare at the name wondering 'why?' As a final note, googling 'Boost json' renders this link to an old version of the docs: http://vinniefalco.github.io/doc/json/index.html Consider removing it so users don't get confused (I did at the very beginning). - What is your evaluation of the potential usefulness of the library? I think it will be incredibly useful. Parsing to DOM and serializing from DOM are very widespread tasks, and having a simple, well designed `json::value` container makes things much better. I would have used it for my use cases if it had been available. - Did you try to use the library? With which compiler(s)? Did you have any problems? I used the library in the Boost version and separate compilation model. I built and installed the library using CMake 3.16.3 and g++ 9.3.0 on Ubuntu 20.04, with no problems. I then built several toy examples with the library's CMake `find_package` interface. No problems. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I spent around 10h reading the documentation and building examples. - Are you knowledgeable about the problem domain? I have used JSON as a data exchange format in the embedded, financial and web worlds, in C++ and other languages (JavaScript, Python and PHP). In C++ I have mainly used rapidJSON, and I would replace it with Boost.JSON for future developments. I have no experience writing JSON libraries myself. Thanks, Ruben.

3 3

Re: [boost] [review][json] My review
by Gavin Lambert 23 Sep '20

23 Sep '20

At 00:22 24/09/2020, Vinnie Falco wrote: >On Wed, Sep 23, 2020 at 1:40 AM Gavin Lambert wrote: >> I've walked back on that one a little bit; I now think that it's >ok for >> basic_parser to do that (to retain a zero-allocation guarantee) >but in >> that case it should treat numbers the same way (which it >currently >> doesn't), and ideally there should be a standard >slightly-higher-level >> parser that has a similar interface but does preassemble the >values, >> eliminating the _part callbacks (for people who prefer ease of >use over >> raw performance). > >I had that originally, it was called number_parser, it was a >public >interface, and it was one of the first things to go during our >3-month >journey of optimization. You can fish it out of the commit log if >you >care to see it. I didn't fish out the code, but given the class name, that sounds like the complete opposite of what I was suggesting. To restate again, my suggestion is: 1. Remove on_int64/uint64/double from basic_parser's handler type. 2. Add on_number(string_view, size_t) to basic_parser's handler type -- having exactly the same semantics as on_key/string and their _part companions. 3. Add custom_parser, which wraps a basic_parser but has internal (dynamically allocated if needed) storage for partial keys/strings/numbers so it never calls _part methods on its own handler type. It still uses on_number(string_view) but note that this (and on_key/string) have no reason to pass a size_t any more, because it can never be any different from the size() of the string_view. 4. The int64/uint64/double parsing code that used to be in basic_parser should be moved to a dedicated helper class, in turn used by parser and *optionally* used by the user's handler implementation for basic_parser/custom_parser. This design has the proper separation of concerns between the parser layers (not dictating a numeric storage format, and keeping number parsing separate from JSON structural parsing). It should not negatively impact performance of parser, and should positively affect performance of basic/custom_parser consumers who didn't want the default conversion anyway.

2 1

[review][JSON] RJanson's review
by Janson R. 22 Sep '20

22 Sep '20

This is a review from a library user standpoint. Thanks to Vinnie, Krystian, and the other contributors for their work. > Do you think the library should be accepted as a Boost library? I recommend ACCEPTING this library into Boost. > - What is your evaluation of the design? The design takes some time to get used to, but it fits well to the problem being tackled (a JSON-DOM library). Switching my codebases using RapidJSON to this library made the code more readable and it will also allow me to get rid of a significant amount of boilerplate. > - What is your evaluation of the implementation? Looks solid. Boost-quality I find. > - What is your evaluation of the documentation? Solid. A month ago I still had to read the source code to find some details, but now all I need for my use-cases is in the docs, well explained. > - What is your evaluation of the potential usefulness of the library? Very useful; to me it takes the same place as Nlohmann.JSON and RapidJSON, both very popular and widely used libraries, but brings better performance and a better API. Regardless of the outcome of this review, I will use this library for my future JSON-DOM needs in C++. > - Did you try to use the library? With which compiler(s)? All in standalone mode on Debian: GCC 9, 10 Clang 9, 10, 11 [trunk of 03 Jul 2020] > - How much effort did you put into your evaluation? Converted two of my codebases (which have different use-cases for a JSON-DOM), read through the implementation, read the docs at multiple stages of its maturation over the last couple months. Amounts to about 40h. > - Are you knowledgeable about the problem domain? I do not have much experience designing nor writing libraries, however I do use a significant number of them, both Boost and non-Boost. JSON is my primary format for configuration files and web communication; In the past I have used Nlohmann.JSON and then primarily RapidJSON for its speed, despite its (in my eyes) terribly user-unfriendly interface. I have also used JSON a lot in other languages (TypeScript, PureScript, Java). -- RJanson

2 1

[JSON[review] Andrzej's review
by Andrzej Krzemienski 22 Sep '20

22 Sep '20

Hi Everyone, I recommend to *ACCEPT* the library into Boost. It meets the standard for BI am definitely going to use it. I tried to use the library in two configurations in header-only mode: g++ 9.2.0 with -std=c++2a on MinGW on Windows, g++ 10.2.1 on Fedora with -DBOOST_JSON_STANDALONE and -std=c++20 All worked fine. I studied library docs and played with examples for about 30 hours. I am not particularly knowledgeable about the problem domain. But my programs have to consume and produce JSON files. DOCUMENTATION It is very good, and meets the highest Boost standards. I mean both the tutorial and reference sections. While doing this review I had to look at the documentation of the competing libraries, and they are really bad. This is why I can appreciate the big effort you put in making these docs. There is a section "Comparison" that compares Boost.JSON to other JSON libraries. I personally think it should also mention the documentation. Some observations: 1. Add reference to RFC 7159 (https://www.rfc-editor.org/info/rfc7159) It allows implementations to set limits on the range and precision of numbers accepted. 2. The documentation for array object and index mention "Satisfies the requirements of ContiguousContainer, ReversibleContainer, and SequenceContainer" These requirements are never defined or referenced. The user will not even know if you defined them or if they are "commonly known". Maybe add link to cppreference.com: https://en.cppreference.com/w/cpp/named_req/ReversibleContainer https://en.cppreference.com/w/cpp/named_req/ContiguousContainer https://en.cppreference.com/w/cpp/named_req/SequenceContainer 3. String has two sections "Member Functions": https://master.json.cpp.al/json/ref/boost__json__string.html 4. Documentation for value::operator= seems corrupt: https://master.json.cpp.al/json/ref/boost__json__value/operator_eq_/overloa… * it has sections with no content * It is an unconstrained template, it look like it can accept any type 5. Examples of input JSON-s contain a lot of escape characters in string literals, like in https://master.json.cpp.al/json/usage/conversion.html This makes it difficult to read. Consider using raw string literals. Instead of assert( serialize( jv ) == "{\"x\":4,\"y\":1,\"z\":4}" ); We would have assert( serialize( jv ) == R"({"x":4,"y":1,"z":4})" ); 6. Some functions (like operator== or swap(x, y)) are naturally part of class interface, even though they are not member functions. It is annoying and confusing that rather than being listed along with the class they belong to they are instead listed separately as "free functions" The docs are not uniform about this issue. For `json::value` operator== is listed as friends along with other member functions, but not 2-argument swap(). 7. In https://master.json.cpp.al/json/usage/conversion.html we read: " Given the following definition of customer::customer( const value& ) [...] Objects of type customer can be converted *to* and from value". This "to" is surprising here. How can a constructor converting from `value` to `customer` could be used to convert `customer` to `value`? 8. As discussed in another thread, documentation would benefit from the info on implementation-limits-related design choices. a. Any json::value that you can build can be serialized and then deserialized, and you are guaranteed that the resulting json::value will be equal to the original. b. JSON inputs where number values cannot be represented losslessly in uint64_t, int64_t and double, may render different values when parsed and then serialized back, and for extremely big number values can even fail to parse. c. Whatever JSON output you can produce with this library, we guarantee it can be passed by any common JSON implementation (probably also based on uint64_t+int64_t+double implementation. DESIGN Clear, sound and practical. The design decisions correspond with my expectations of a library: fast, easy to use and learn, with value semantics. The interface is very intuitive, I practically do not have to consult the documentation: every functionality (serialization, value alteration) has the interface that I would expect. I appreciate that it targets embedded environments. Some observations: 1. The handling of infinities and NaN's needs to be addressed. Currently, I can do: `json::serialize(std::numeric_limits<double>::infinity())` and t produces an invalid JSON. You cannot easily make a precondition on json::value assignment/construction because you expose the double as a mutable member: void fun(double x) // precondition x is not infinity or nan { json::value v = x; // ok v.as_double() *= 2.0; // this may produce infinity } So maybe you need a precondition in the serializer. 2. Boost.JSON provides its own string type. This begs a lot of design questions for `json::string`. Is it a better replacement for std::string? std::string provides the following capability: ``` void set(std::string& s, char v, char x, char z, char k) { if (cond) s = {v, k}; else s = {v, x, z, k}; } ``` json::string does not. How do I do a similar thing with json::string? 3. string::swap() has a strange precondition: you cannot swap an object with itself https://master.json.cpp.al/json/ref/boost__json__string/swap.html This is a really unexpected and potentially dangerous, and at minimum requires a very strong rationale. This would be the first swap function in std or Boost that has a precondition. While discussing a somewhat related issue (on the guarantees of the moved-from state) in the LEWG, someone remarked that there was at least one STL implementation that did a self-swap for convenience of implementation. Is it really necessary? IMPLEMENTATION I didn't look at it much. Just some quick points I observed. 1. Values of type `double` representing Infinity and NaN can be stored in json::value and when later serialized, they render a malformed JSON output. json::value should either validate special values of `double` or indicate as precondition that they should not be assigned. Alternatively, the serializer should either validate this or indicate as precondition. 2. tag_invoke() takes tag arguments by reference, does it have to? Passing by reference requires the constexpr variables to be instantiated and their address assigned. 3. Customization through ADL-discovered overloads can cause ODR violations. There is no superior alternative though. But maybe put a warning in the docs that the user must make sure that all translation units must see the same set of overloads. 4. I do not like the 2-state-switch mechanics of BOOST_JSON_STANDALONE. It assumes that a user either wants a no-Boost version or that she wants a package: all Boost components. But what if I am using C++14 and I want to use std::error_code and Boost-emulated memoty_resource and string_view? What If I use Boost version and C++17 and want to use std::string_view? Maybe you can additionally provide macros for turning Boost-emulation libraries one-by-one? I mean, one for string_view, one for error_code, and one for memory_resource. 5. The header only mode -- I really appreciate it. It is good for initial experiments. Finally, I would like to thank Vinnie and Krystian for writing and sharing this library. Regards, &rzej;

3 4