Re: [Boost-users] [Spirit] parsing double precision issues

On Wed, Nov 14, 2018 at 6:35 PM Michael Powell wrote:

On Wed, Nov 14, 2018 at 5:05 PM Michael Powell wrote:

...

On Wed, Nov 14, 2018 at 1:58 PM Michael Powell wrote:

...

Hello,

I am continuing to work through my Protobuf parsing with Constant/Floating-Point tests. Double precision is rearing its ugly head and I am curious what's the best way to handle potentially determining a healthy epsilon given the expected parsed value. For the most part, I think it's going to work, but for the precision issues. Catch:

No response? Does anyone have any insights as to the real parser policies? The documented grammar would lead me to believe that "fixed format", literally, (*digits >> '.' >> +digits) | (+digits >> '.' >> *digits), in Qi phraseology, might be possible. However, I am finding potentially this is not the case.

...

After doing some background reading on floating point comparison ... it's not so simple as a==b, but I kind of knew that already. Even the simple absolute difference epsilon check is not so robust. I did a combination of nearly, relatively, essentially, etc, checks, and I think I have gotten past that immediate issue.

Now I actually have a parsing issue... According to the Protobuf v2 spec, this should parse, I think. However, it is failing to parse.

CHECK( parse_tried == expected_tried ) with expansion: false == true with message: Source: syntax = 'proto2';option a = 933252761176835091419019014852578134454945897134240121039056292642745934626544776679309792643426912681734775655456099849022472225507009671635655261635494179939591695082743337844489696175391517474159955183289891072214770215979051082304343590361421008538352849939802258131368544983961387362613078430270357504.000000; REQUIRE( tried ) with expansion: false with message: Source: syntax = 'proto2';option a = 933252761176835091419019014852578134454945897134240121039056292642745934626544776679309792643426912681734775655456099849022472225507009671635655261635494179939591695082743337844489696175391517474159955183289891072214770215979051082304343590361421008538352849939802258131368544983961387362613078430270357504.000000;

This much I can say at the moment, it does not appear to be a max double issue, literally:

179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000000000000000

That would stand to reason, though, since my test data is randomly generated within max double boundaries.

I am scratching my head on the strict real policies, however. If the docs are accurate, I do not see why that would not accept the fixed double format...

But it could be a concern that parsing is done apart from Qi rules. It seems like some rigid logic embedded there.

https://www.boost.org/doc/libs/1_68_0/libs/spirit/doc/html/spirit/qi/referen...

...

...

REQUIRE( actual == expected ) with expansion: struct my::protobuf::ast::floating_point_t { 'val': 1.16624e+306, 'opt_sign': null } == struct my::protobuf::ast::floating_point_t { 'val': 1.16624e+306, 'opt_sign': null } with messages: Source: syntax = 'proto2';option L = 1.16624e+306; Delta was: 2.35309e+300

There was clearly a delta involved, most like a precision issue encountered during the parse. Without going into a great deal of depth, if possible:

namespace my { namespace protobuf { namespace ast { using float_t = double; enum num_sign_t { sign_none /// '-' , sign_minus /// '+' , sign_plus };

// Which includes struct floating_point_t : numeric_t { /* ... */ }; } } }

// Without sign, dot required. // >>> TODO: TBD: Precision issue? Or an actual double parsing issue? <<< float_lit %= real_parserast::float_t>{}; floating_point_lit %= -num_lit_sign >> float_lit;

qi::rule num_lit_sign;

qi::rule float_lit; qi::rule floating_point_lit;

https://developers.google.com/protocol-buffers/docs/reference/proto2-spec#fl... https://developers.google.com/protocol-buffers/docs/reference/proto2-spec#co... https://www.boost.org/doc/libs/1_68_0/libs/spirit/doc/html/spirit/qi/referen...

Cheers,

Michael Powell