Re: [boost] New JSON library from the maker of Beast!

newer
Re: [boost] New JSON library from...

older
[release] Is everyone ready for...

Vinnie Falco

22 Oct 2019 22 Oct '19

4:25 p.m.

On Tue, Sep 24, 2019 at 7:32 AM Dominique Devienne wrote:

...

...why conflate the array and object APIs in the value type, rather than just having as_object() and as_array() as you already?

I agree, and these additional APIs have been pruned. I have also pruned other unnecessary APIs (thanks to advice from Peter Dimov) such as the local iterator and bucket interfaces in `json::object` (which needlessly constrain the implementation).

...

So I'm curious why it should be different here. Does the doc goes into that already?

`json::value` is not `variant<...>` for the same reason that `string_view` is not `std::pair`.

...

On the "compact" side, how compact is your json::value? What are the different sizeof on 32 and 64-bit arch?

I've done some work on this. I gave up on using std::string to represent strings for a few reasons related to performance and API. I have reimplemented string to have a small buffer optimization and to work natively with the `json::storage_ptr` type which gives some space efficiencies (and eliminates some unnecessary atomic operations caused by std::string's Allocator interface). The new sizes look like this: 32-bit sizeof(value) == 40 sizeof(object) == 8 sizeof(array) == 8 sizeof(string) == 32 sizeof(number) == 24 64-bit sizeof(value) == 48 sizeof(object) == 16 sizeof(array) == 16 sizeof(string) == 40 sizeof(number) == 24 The string type has a 20-byte small buffer which is pretty generous Thanks

Show replies by date

Gavin Lambert

22 Oct 22 Oct

9:52 p.m.

New subject: New JSON library from the maker of Beast!

On 23/10/2019 05:25, Vinnie Falco wrote:

...

...
So I'm curious why it should be different here. Does the doc goes into that already?

`json::value` is not `variant<...>` for the same reason that `string_view` is not `std::pair`.

That's a good reason for not inheriting from it. It's not a good reason for not using it as a private member, to avoid reimplementing the variant-storage concept. (Though there may be other good reasons for that.)

Emil Dotchevski

10 p.m.

New subject: New JSON library from the maker of Beast!

On Tue, Oct 22, 2019 at 2:53 PM Gavin Lambert via Boost < boost@lists.boost.org> wrote:

...

...
...
So I'm curious why it should be different here. Does the doc goes into

On 23/10/2019 05:25, Vinnie Falco wrote: that already?

...
`json::value` is not `variant<...>` for the same reason that `string_view` is not `std::pair`.

That's a good reason for not inheriting from it.

It's not a good reason for not using it as a private member, to avoid reimplementing the variant-storage concept. (Though there may be other good reasons for that.)

There are always two kinds of dependencies, logical and physical. Logically, you use something as a private member, you're ok, but physically you're still dependent on it, and that may be a problem. Further, std::variant may be problematic for portability, and it may or may not be as efficient as something hand-written specifically for the purpose. Consider that if performance is critical, you don't want to deal with "I changed my compiler and now your JSON parser runs 2x slower".

Daniel Frey

10:13 p.m.

New subject: New JSON library from the maker of Beast!

...

On 23. Oct 2019, at 00:00, Emil Dotchevski via Boost wrote:

On Tue, Oct 22, 2019 at 2:53 PM Gavin Lambert via Boost < boost@lists.boost.org> wrote:

...
...
...
So I'm curious why it should be different here. Does the doc goes into

On 23/10/2019 05:25, Vinnie Falco wrote: that already?

...
`json::value` is not `variant<...>` for the same reason that `string_view` is not `std::pair`.

That's a good reason for not inheriting from it.

It's not a good reason for not using it as a private member, to avoid reimplementing the variant-storage concept. (Though there may be other good reasons for that.)

There are always two kinds of dependencies, logical and physical. Logically, you use something as a private member, you're ok, but physically you're still dependent on it, and that may be a problem. Further, std::variant may be problematic for portability, and it may or may not be as efficient as something hand-written specifically for the purpose. Consider that if performance is critical, you don't want to deal with "I changed my compiler and now your JSON parser runs 2x slower".

That sounds like you should stop using the STL altogether. What if std::string is suddenly 2x slower? So... no, I don't buy that argument. I just switched our JSON library https://github.com/taocpp/json from our own classic enum+union implementation (back from the C++11 days) over to std::variant (as we are now C++17 anyways). It saves us about 800-1000 LOCs and I ran benchmarks with GCC and Clang as well as libstdc++ and libc++. They were all equally fast or even faster. Admittedly I only used the newest versions of those and I haven't tested MSVC, but given the quality of MSVC in the last years I am not worried about the performance at all. If something is slow, more people using it will only create more incentives for the STL maintainers to improve their implementations further.

Vinnie Falco

10:40 p.m.

New subject: New JSON library from the maker of Beast!

On Tue, Oct 22, 2019 at 3:13 PM Daniel Frey via Boost wrote:

...

That sounds like you should stop using the STL altogether. What if std::string is suddenly 2x slower? So... no, I don't buy that argument.

I just switched our JSON library https://github.com/taocpp/json from our own classic enum+union implementation (back from the C++11 days) over to std::variant (as we are now C++17 anyways).

Any idea why taocpp compiled with optimizations is 5 to 10 times slower than the other libraries at parsing the same inputs? Boost.JSON parse 55952763 bytes in 234ms Boost.JSON parse 55952763 bytes in 232ms Boost.JSON parse 55952763 bytes in 231ms rapidjson parse 55952763 bytes in 239ms rapidjson parse 55952763 bytes in 240ms rapidjson parse 55952763 bytes in 239ms nlohmann parse 55952763 bytes in 495ms nlohmann parse 55952763 bytes in 491ms nlohmann parse 55952763 bytes in 494ms tao-json parse 55952763 bytes in 2178ms tao-json parse 55952763 bytes in 2069ms tao-json parse 55952763 bytes in 2056ms Boost.JSON parse 488889121 bytes in 1820ms Boost.JSON parse 488889121 bytes in 1831ms Boost.JSON parse 488889121 bytes in 1835ms rapidjson parse 488889121 bytes in 1860ms rapidjson parse 488889121 bytes in 1886ms rapidjson parse 488889121 bytes in 1828ms nlohmann parse 488889121 bytes in 4195ms nlohmann parse 488889121 bytes in 4241ms nlohmann parse 488889121 bytes in 4320ms tao-json parse 488889121 bytes in 22952ms tao-json parse 488889121 bytes in 23265ms tao-json parse 488889121 bytes in 23117ms Here's a profile of an optimized build calling just tao-json for parsing: https://i.imgur.com/I7QLPkn.png It seems like there's this thing called PEGTL which has some large number of functions showing up as taking up time in the profile? What is PEGTL? Thanks

Vinnie Falco

10:14 p.m.

New subject: New JSON library from the maker of Beast!

On Tue, Oct 22, 2019 at 2:53 PM Gavin Lambert via Boost wrote:

...

It's not a good reason for not using it as a private member, to avoid reimplementing the variant-storage concept. (Though there may be other good reasons for that.)

I didn't reimplement the variant-storage concept, I used a union. This does away with all the unnecessary baggage of having to support arbitrary types with arbitrary exception specifications. In my case the notional "variant" is always instantiated with the same types, thus avoiding bulk in the headers. A goal of this JSON library is to have very fast compilations, I have avoided templates wherever possible (especially in the vocabulary types) and when there are member function templates I have used techniques to reduce the number and size of instantiations. Thanks

1846

Age (days ago)

1846

Last active (days ago)

List overview

Download

5 comments

4 participants

participants (4)

Daniel Frey
Emil Dotchevski
Gavin Lambert
Vinnie Falco