- Boost - lists.preview.boost.org

[JSON][review] a small review of boost.json
by Julien Blanc 19 Sep '20

19 Sep '20

Hi, Here's my small review of boost json. ## Are you knowledgeable about the problem domain? I've written a json library (for reference: https://github.com/Julien-Blanc-tgcm/jbc-json ) which i used as playground / experiments. I guess this makes me knowledgeable enough of the domain. ## What is your evaluation of the design? I think the library is well designed, although it has several shortcomings. These shortcomings may not alter general usage of the library (with main target seeming to be REST api over http), but render it useless in specific cases. For me, a json library consists of three independant parts and some glue: * a parser * a type for representing values (usually a variant) * a printer (serializer) ### The parser In boost.json, the separation between the parser and seems neat enough (ie, you can use the parser without the json::value type). However, it resides in a <detail> header, which would indicate that it is not an expected use case. I believe it should be. The callbacks of the parser (push parser) are not documented. I'm a bit puzzled by what the on_number_part does, and how it fits with the on_int64/on_double callbacks, for example. The parser is an incremental push parser. However, it seems to expect to be feeded with partial json documents, but not consecutive json documents. I could not find a way to make : auto v = p.write(R"json({"foo":1}{"bar":2})json"); return 9. I may have missed something, but this use case is IMHO important to address. Since the only way to know that a json document has ended is to parse it, the parser should provide an option to parse a stream containing multiple consecutive json documents (ie, stopping without error at the end of each document). A common use case for this is JSON notifications received via raw sockets, so no envelope may be used to isolate JSON documents. ### The value type The value type is a variant, the kind that you would expect from a json library. Most of the interface uses string_view, which is a very good thing (the fact that boost.json uses another string type internally has not been an issue in my tests, due to the usage of string_view). It lacks visitation, though (IIRC there’s an open issue for this). The fact that this value type has been highly optimized is the great strength of this library. #### objects The object interface relies on the [] operator, which incurs the same shortcomings as with std::map : auto v = foo["bar"]; // will silently create bar if it does not exists Typos in keys had always been a source of hard to find bugs, i wished there was an option to disable operator[] and force the use of find. The library has no support for object with duplicate keys, which also are valid json. This, however, can be worked around by using the parser callbacks and not the variant (but that makes it difficult to produce such documents, cf later). Whether the order of the keys is kept by the object structure or not does not appear in the documentation. From my small testing, it seems that it is the case, but the api does not offer any interface to add elements at specific position, so i would assume it is not guaranted. The json specification, AFAIR, does not forbid to rely on a specific order of the keys. Some serialization libraries do expect the keys to be in a specific order. #### numbers Having two number types can lead to unexpected results : auto v = boost::json::parse("{\"foo\":1,\"bar\":2.0}"); std::cout << v.as_object()["bar"].as_double() << std::endl; // ok std::cout << " " << v.as_object()["foo"].as_double(); // asserts It is unexpected, because several json libraries will never output “2E0” or “2.0” for the value “2”, but a plain “2”. It means that whenever you expect a double value, you also have to check for the possibility that it has been parsed as an integer. The to_double function should silently convert integers to their double equivalent whenever it is possible. * as has already been noted by others, the number precision is hardcoded to IEE754 or 64bits integer. This is seldom an issue, since it is the case with most implementations. #### The printer The printer is not usable without the value item. I would expect it to be (producing json without resorting to an intermediate representation). Apart from this, it supports incremental printing, which is what is expected. It produces a strange representation for numbers, but it is conforming to the JSON specification. Because it is tied to the json::value, it cannot produce anything that cannot be represented by a json::value. This is an issue that should be addressed. ## What is your evaluation of the implementation? I did not have a thorough look into the implementation details. From what i have seen, the implementation looks fine. A parser is usually a landmine for security exposures, and would need a very careful review which i cannot do. IIRC the authors plan on having an independant audit on the library, which is a good thing. There is also fuzzing and tests, so we can expect that this point is correctly being taken care of. The rest of the code looks fine, easy to navigate into. ## What is your evaluation of the documentation? Very good. Easy to use, complete. There is some missing information, but they all seems related to design decision that have been taken implicitly (already discussed in design section). An example on using the parser on standalone mode would be a very welcome addition. Another thing that the doc should state clearly : boost.json is **not** a JSON (ECMA-404) library, but rather an I-JSON (RFC 7493) library with some extensions. ## What is your evaluation of the potential usefulness of the library? Very useful. While it does not address all use cases (and does not try to), it addresses what is probable the *most common* case. As JSON is very widespread, i expect this library to find a large audience, because it combines ease of use and performance. ## Did you try to use the library? With what compiler? Did you have any problems? I tried to use it on a small toy application, on a debian stable system. The main target seems to be to use the library with boost. I ran into several issues when trying to use the standalone version. The first one was easy to address, and due to gcc 8.3 lacking c++17 support. A patch was submitted, and merged quickly. The second one was incompatibilites between standalone asio bringing boost headers (because boost is installed on my machine) and json redefining them (because of standalone mode), resulting in multiple defenitions of throw_exception. I'm not sure yet if i should file this to boost.json, asio library, or asio debian package, or if it is just a missing define on my part. At that point, i switched to boost json, which fixed the issues i had. ## How much effort did you put into your evaluation? I'd say around 10 hours reading, testing and playing, including writing of the review. ## Do you think the library should be accepted as a Boost library? I think the library should be ACCEPTED. There are shortcomings in the library, it does not address every use case in the world, but it does a very good job at what it is designed to do. It is easy to use, performant, and it addresses a very widespread use case, so the pros clearly outweighs the cons. Regards, Julien

3 3

[preprocessor] Dropping support for compilers that do not implement variadic macros
by Edward Diener 18 Sep '20

18 Sep '20

In lieu with Boost's decision regarding dropping support for C++03 compilation, I would like to drop support in the preprocessor library for preprocessors which do not implement variadic macros. Variadic macros were added in the C++11 standard, although a number of compilers supported them even before the official C++ standard. I will of course keep the same macro APIs, although deprecating non-variadic macro forms when alternate variadic macro forms exist. This will simplify Boost PP. Ideally, once Boost PP supports only preprocessors which implement variadic macros, Boost VMD can be added to Boost PP, for anyone who wants to use its functionality, and Boost VMD can then be retired.

6 25

[JSON][review] Design: what is expected of a "JSON library"?
by Andrzej Krzemienski 18 Sep '20

18 Sep '20

Hi Everyone, This is not a complete Boost Review yet. In this email I wanted to discuss the design goals of the library. The point that I would like to raise is the tension between having relatively small types (int64_t, uint64_t, double) represent numbers in json::value on the one hand, and ECMA specification allowing arbitrarily big/precise numbers in JSON format on the other. Can I expect of a JSON library that when it converts a JSON contents into internal representation and then back to JSON contents, I should get the same contents (moduo white space)? This would be possible if the internal representation of JSON numbers was an arbitrary-precision decimal type or a string. But when we need to squeeze any number into 24 bits, we will soon get to the point when integer number `100000000000000000000001` after the experiment gets changed to `1E23`. Is this acceptable for a JSON library? But maybe it is not a valid goal? Maybe the goal of the JSON format is to have objects already created in internal representation converted to text and then back to objects? (assuming the recipient program is run on the same environment as the sender (no differences in word size or maximum int representation.) That is, as long as you agree to the constraints of json::value, whatever you manage to put inside, we guarantee that you get the same value when you serialize it and then parse it back. Boost.JSON does this nicely. In that other view, if I have objects of types `boost::multiprecision::cpp_int` my only option is to pass them as strings in JSON protocol. But I can pass any number as string anyway, so what is the use of numbers in JSON format? Uness it is just practical: you can choose to use numbers and then internal representations of JSON may be smaller. Do you get the concern that I am seeing? I mean I have used JSON libraries before, and this has never been a practical problem. But Boost Review puts the bar high for the libraries, so I guess this question should be answered: what guarantees can a JSON library give us with respect to accuracy of numbers? Regards, &rzej;

6 11

Re: [boost] [JSON[review] Andrzej's review
by Peter Dimov 17 Sep '20

17 Sep '20

> Vinnie Falco wrote: > > > Just open an issue and explain how you think self-swap, self-move, and > > self-assignment should work and I will make the changes. > > They should be no-ops. More specifically, self-swap should be a no-op, and assignments should be equivalent to: T& operator=( T const& rhs ) { T(rhs).swap(*this); return *this; } T& operator=( T&& rhs ) { T(std::move(rhs)).swap(*this); return *this; } This is important for classes such as json::value, object, array, where *this may own rhs, so it's important not to destroy rhs too early. E.g. v = v.at(0); v = std::move(v.at(0)); a = a.at(0).as_array(); a = std::move(a.at(0)).as_array(); o = o.at("key").as_object(); o = std::move(o.at("key").as_object()); This is not self-assignment, but it has the same lifetime pitfalls - if you destroy the old contents of *this, rhs is invalidated, so you can no longer copy (or move) it.

2 1

Interactive mail
by SIDDHARTH PANDEY 17 Sep '20

17 Sep '20

Hello everyone, myself Siddharth Pandey, a sophomore at Indian Institute of Information Technology Sri City, India. I love to do Competitive Programming and have a keen interest in Open Source Contribution. I have a strong foundation on C,C++ and some about Python & Java. Also I am trying to explore the field of Data Science & Machine learning. For more information do visit my Portfolio website https://siddharth25pandey.github.io/ I want to contribute to the Boost Organisation but I am quite confused how to start with. It will be really helpful if any mentor can guide me how to start Contributing to get a great learning from such a highly reputed organization. Thanks & Regards Siddharth Pandey

4 3

Boost Log 1.65.1 Linking Issue
by Karmethia Thompson 16 Sep '20

16 Sep '20

Good morning, I am trying to create a logger that uses Boost Log. I was successful getting my code to compile with 1.63.0, so I know that I am not too far off of the mark. cmake/make throws a linking issues whenever I attempt to create a sink: typedef sinks::synchronous_sink< sinks::text_ostream_backend > text_sink; boost::shared_ptr< text_sink > sink = boost::make_shared< text_sink >(); or create file sink logging::add_file_log ( keywords::file_name = log_specs.log_file, keywords::rotation_size = 10 * 1024 * 1024, keywords::time_based_rotation = sinks::file::rotation_at_time_point(0, 0, 0), keywords::filter = min_severity || severity >= file_severity, keywords::format = ( expr::stream << expr::format_date_time<boost::posix_time::ptime>("TimeStamp", "%Y-%m-%d_%H:%M:%S.%f") << ": [" << severity << "]" << expr::smessage ), keywords::auto_flush = true //flush to file.. ); My code will compile 95% and then produce the following: CMakeFiles/main.dir/src/Logger.cpp.o: In function `boost::enable_if_c<(boost::log::v2_mt_posix::aux::is_character_type<wchar_t>::value&&boost::log::v2_mt_posix::aux::is_character_type<char>::value)&&((sizeof (wchar_t))!=(sizeof (char))), bool>::type boost::log::v2_mt_posix::aux::code_convert<wchar_t, char, std::char_traits<char>, std::allocator<char> >(wchar_t const*, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::locale const&)': Logger.cpp:(.text._ZN5boost3log11v2_mt_posix3aux12code_convertIwcSt11char_traitsIcESaIcEEENS_11enable_if_cIXaaaasrNS2_17is_character_typeIT_EE5valuesrNS8_IT0_EE5valuenestS9_stSB_EbE4typeEPKS9_mRNSt7__cxx1112basic_stringISB_T1_T2_EEmRKSt6locale[_ZN5boost3log11v2_mt_posix3aux12code_convertIwcSt11char_traitsIcESaIcEEENS_11enable_if_cIXaaaasrNS2_17is_character_typeIT_EE5valuesrNS8_IT0_EE5valuenestS9_stSB_EbE4typeEPKS9_mRNSt7__cxx1112basic_stringISB_T1_T2_EEmRKSt6locale]+0x37): undefined reference to `boost::log::v2_mt_posix::aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::locale const&)' CMakeFiles/main.dir/src/Logger.cpp.o: In function `void boost::log::v2_mt_posix::sinks::basic_formatting_sink_frontend<char>::feed_record<boost::recursive_mutex, boost::log::v2_mt_posix::sinks::basic_text_ostream_backend<char> >(boost::log::v2_mt_posix::record_view const&, boost::recursive_mutex&, boost::log::v2_mt_posix::sinks::basic_text_ostream_backend<char>&)': Logger.cpp:(.text._ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS_15recursive_mutexENS2_26basic_text_ostream_backendIcEEEEvRKNS1_11record_viewERT_RT0_[_ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS_15recursive_mutexENS2_26basic_text_ostream_backendIcEEEEvRKNS1_11record_viewERT_RT0_]+0x141): undefined reference to `boost::log::v2_mt_posix::sinks::basic_text_ostream_backend<char>::consume(boost::log::v2_mt_posix::record_view const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' CMakeFiles/main.dir/src/Logger.cpp.o: In function `void boost::log::v2_mt_posix::sinks::basic_formatting_sink_frontend<char>::feed_record<boost::recursive_mutex, boost::log::v2_mt_posix::sinks::text_file_backend>(boost::log::v2_mt_posix::record_view const&, boost::recursive_mutex&, boost::log::v2_mt_posix::sinks::text_file_backend&)': Logger.cpp:(.text._ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS_15recursive_mutexENS2_17text_file_backendEEEvRKNS1_11record_viewERT_RT0_[_ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS_15recursive_mutexENS2_17text_file_backendEEEvRKNS1_11record_viewERT_RT0_]+0x141): undefined reference to `boost::log::v2_mt_posix::sinks::text_file_backend::consume(boost::log::v2_mt_posix::record_view const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' CMakeFiles/main.dir/src/Logger.cpp.o: In function `void boost::log::v2_mt_posix::sinks::basic_formatting_sink_frontend<char>::feed_record<boost::log::v2_mt_posix::aux::fake_mutex, boost::log::v2_mt_posix::sinks::basic_text_ostream_backend<char> >(boost::log::v2_mt_posix::record_view const&, boost::log::v2_mt_posix::aux::fake_mutex&, boost::log::v2_mt_posix::sinks::basic_text_ostream_backend<char>&)': Logger.cpp:(.text._ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS1_3aux10fake_mutexENS2_26basic_text_ostream_backendIcEEEEvRKNS1_11record_viewERT_RT0_[_ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS1_3aux10fake_mutexENS2_26basic_text_ostream_backendIcEEEEvRKNS1_11record_viewERT_RT0_]+0x141): undefined reference to `boost::log::v2_mt_posix::sinks::basic_text_ostream_backend<char>::consume(boost::log::v2_mt_posix::record_view const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' CMakeFiles/main.dir/src/Logger.cpp.o: In function `void boost::log::v2_mt_posix::sinks::basic_formatting_sink_frontend<char>::feed_record<boost::log::v2_mt_posix::aux::fake_mutex, boost::log::v2_mt_posix::sinks::text_file_backend>(boost::log::v2_mt_posix::record_view const&, boost::log::v2_mt_posix::aux::fake_mutex&, boost::log::v2_mt_posix::sinks::text_file_backend&)': Logger.cpp:(.text._ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS1_3aux10fake_mutexENS2_17text_file_backendEEEvRKNS1_11record_viewERT_RT0_[_ZN5boost3log11v2_mt_posix5sinks30basic_formatting_sink_frontendIcE11feed_recordINS1_3aux10fake_mutexENS2_17text_file_backendEEEvRKNS1_11record_viewERT_RT0_]+0x141): undefined reference to `boost::log::v2_mt_posix::sinks::text_file_backend::consume(boost::log::v2_mt_posix::record_view const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' collect2: error: ld returned 1 exit status make[2]: *** [main] Error 1 make[1]: *** [CMakeFiles/main.dir/all] Error 2 Can someone please help? I have tried everything from trying to dynamically and statically link. I've added both "log" and "log_setup" to the find_packages in cmake and have also explicitly specified the locations of those libraries as well. Please let me know if there is anything else to try. Thanks -- Karmethia Thompson Applied Mathematics, PhD

4 6

Re: [boost] Boost Log 1.65.1 Linking Issue
by Peter Dimov 15 Sep '20

15 Sep '20

Although, now that I think of it, the shown CMakeLists.txt file didn't use imported targets, so perhaps the order of libraries is literal and putting log_setup before log may indeed solve things. -----Original Message----- From: Peter Dimov Sent: Tuesday, September 15, 2020 20:28 To: boost(a)lists.boost.org Subject: Re: [boost] Boost Log 1.65.1 Linking Issue Andrey Semashev wrote: > I'm not sure, but I think the above would generate incorrect order of > libraries in the linker command line. You need to ensure that boost_log > comes after boost_log_setup, and their dependencies come after still. CMake automatically orders libraries so that the dependencies come after. Unfortunately, FindBoost has a bug and declares log as depending on log_setup, rather than vice versa. https://github.com/Kitware/CMake/blob/bdcde7762c71cd27fcba6dc1a3efe12fa7a09… No matter how you order the libraries in target_link_libraries, the order between these two will never be correct. Not sure if this is the problem here though.

1 0

[Math] Adding prime sieves - design discussion
by John Maddock 15 Sep '20

15 Sep '20

Just a heads up, that Boost.Math has an open PR for adding prime sieves, and there is an ongoing design discussion on interface etc. If anyone has any strong feelings on the matter, usage requirements etc, then comments would be most welcome. See https://github.com/boostorg/math/pull/400. Many thanks, John. -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

1 0

[JSON][review] Some early feedback
by Andrzej Krzemienski 15 Sep '20

15 Sep '20

Hi Everyone, The formal review of Boost.JSON has not started yet, but I thought I could offer some feedback from my playing with the library earlier in the process. Documentation --------------- Very well written. It makes it easy to grasp quickly the spirit and the potential of the library, and the reference section makes it easy to navigate through the features and their interfaces. Some minor issues: 1. Section "Using Numbers" is missing content. 2. Description of storage_ptr's destructor and move constructor is missing noexcept. 3. Docs in a couple of places call `storage_ptr` a "smart pointer container". Is it intentional? "Container" has a very specific meaning in C++. Design. ----------- It is generally sound, and well explained in the rationale. Here are a couple of minor remarks. 1. The usage of `nullptr` for null values. `nullptr` has been introduced to mean "a pointer with null value" and is only used in the standard library for smart pointers, with an ignoble exception of `std::function<>`. Using it for `json::value` is not an obvious choice for me. On the other hand, we do not have a universal type representing a missing value. A library could provide its own `json::null`, although I am not sure if this is an ideal solution either. 2. Why is json::value Semiregular rather than Regular? Equality comparison has a natural interpretation and application for checking if the produced JSON looks exactly as expected, or for checking if some two subtrees are the same. Is it because of efficiency issues with json::object comparison? 3. The library is using a custom string type. We get a rationale for this, and I accept it. But there are still some concerns about it that I would like to mention. 3.1. It is not the first library reviewed his year to have a custom type with std::string-compatible interface. It looks like the idea of having a "vocabulary" string type is unrealistic. If this should be the case, maybe we need a concept specifying what a string-compatible type should provide. It is obvious that json::string does not literally provide the same interface as std::string. Nor does anyone expect 100% compatibility. 3.2. Docs say that json::string is like std::string *except for* a number of things. For instance it does not provide the controversial assignment from `char`. Given that you are "fixing" the shortcomings of std::string, maybe it is an opportunity to fix other things in it also. For instance, std::string has been criticised for having too big an interface. That it mixes an iterator-based interface with an indexed-based one. Myb functions like `find_first_not_of()` are not necessary, given that users can use all sorts of iterator-based algorithms? 3.3. The rationale for json::string says that inisializer_list constructor is unnecessary, because string literal can always be used instead. This does not take into account that initializer_list does not necessarily operate on constant values. There is no concise substitute for: ``` std::string arrange(char a, char b, char c) { return {c, a, b, c}; } ``` I would not miss this construction much, I guess, but I would expect this to be mentioned in the rationale. 4. One thing one often expects of a JSON library is to be able to pretty-print a JSON document. The docs give an example how to do it, but the library does not provide this feature out of the box. Is pretty-printing not compatible with the design goals of the library? Implementation -------------- I didn't look much at it yet, but here is something I found. 1. `json::string_view` is convertible from `std::stirng`, but it is not convertible from `std::string_view`. I think there is no harm in enabling this conversion, and it seems useful. 2. The following piece of code ``` json::value v = 50.0; std::cout << json::serialize(v); ``` oututs: 5E1 which is technically compliant with JSON specification, but quite surprising. Is there a reason for this? Can serialization be controlled to output 50.0 or 50? 3. It is confusing for me to see member function serializer::read. I would think that the action that a serializer does is *write* rather than read. Am I missing something? 4. I really appreciate the ability to use the library in header-only mode with <boost/json/src.hpp>. This makes library testing much simpler. Regards, &rzej;

7 10

[json][review] A review from an embedded perspective
by Brooks Prumo 15 Sep '20

15 Sep '20

Hi everyone, this is my first time mailing the Boost developer list, and also submitting a review. Thank you in advance for understanding any issues it may have! I'm writing to review Boost JSON. I've been using it very lightly for the last week or so. My usage has been exclusively on embedded MCUs (ARM Cortex M4, compiling with ARM GCC 9 with C++17), which I'm guessing hasn't been the common case so far. Here's a list of things that matter a lot to me: - Compile from source. Boost JSON passes. - No exceptions. C++ exceptions are turned off, so all libraries I use must support not throwing. Boost JSON passes. - No iostreams/locale. If a library uses iostreams, my binary jumps ~220 KB which is a deal breaker. Boost JSON passes. - Binary size matters a lot, since onboard Flash is very limited. I am using a "big" MCU, and that means my whole binary must be under ~512 KB. Right now Boost JSON is not great. In doing a simple serialization and parse, the library itself adds ~58 KB to my .text section. This is OK for now, but surprisingly big. I know the team is aware of this and has some ideas to reduce the binary size. I will likely need those fixes to actually port my app to Boost JSON. - Control over the sizes of any "large" stack buffers. Boost JSON passes. - No external dependencies that do not meet the above requirements. Boost JSON passes. With this, Boost JSON is an ACCEPT from me. THe biggest reason for me is that the API is nice to work with. I started off with Nlohmann, but the binary size was too big due to iostreams. I'm currently using RapidJSON, and it's definitely not easy to use. In my app, JSON is not in the hot path. I want a library that is easy to use and meets my requirements above. I greatly appreciate being able to use a modern C++ library. And answers to the other questions: - What is your evaluation of the design? So far so good. I do not have complicated JSON uses though. - What is your evaluation of the implementation? I've only looked at the implementation when it comes to binary size. - What is your evaluation of the documentation? Documentation is nice. I personally would enjoy more examples, as that is how I commonly learn new libraries. - What is your evaluation of the potential usefulness of the library? HUGE! I haven't come across another modern C++ JSON library that'll work in the embedded space I live in. It's really exciting! - Did you try to use the library? With which compiler(s)? Did you have any problems? Yep! I used ARM GCC 9.2.1 with C++17. No issues with compilation. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I'd say my evaluation is "initial". I've used it in a way that mimics my real usage, but haven't used many aspects of the whole library. - Are you knowledgeable about the problem domain? Minimally. Just basic JSON knowledge. Thank you! Brooks

3 2