Sergey Voropaev wrote:

Beman Dawes <bdawes <at> acm.org> writes:

...

If anyone know of any showstoppers, please let us know what the problem is.

I think boost::serialization library has very serious flaw. It is lack of backward compatibility for binary archive. I discovered this problem when I did upgrade from 1.34 to 1.43 and 1.44 versions of BOOST. 1.44 library can not work with old binary archive. The reason - format of binary header of archive has been changed. I think 1.45 version should be corrected for this problem and/or we should place some announcement about recommendation not to use the 1.44 version to work with binary archive.

See: https://svn.boost.org/trac/boost/ticket/4660 http://thread.gmane.org/gmane.comp.lib.boost.devel/208588

I"m aware of this, it's just that I haven't the the time to deal with it right now. I've had a lot of trouble with this and I'm warying of trying to fix it without having the time to do it right. Robert Ramey

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Vladimir Prus wrote:

Robert Ramey wrote: Uhm, so we don't have a fix checked in yet? Maybe, the right solution, for 1.45, is to revert completely whatever patch has broken the backward compatibility in the first place? Folks are complaining real loud about 1.44 already -- I'd rather not see same complains about 1.45

It's not that easy. The problem isn't compatibility of the library itself but rather compatibility with binary archives created with different past versions. Rolling back to 1.42 would leave all binary archives created since as unreadable. I'll try to get this addressed this weekend. Robert Ramey

- Volodya

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Eric Niebler wrote:

On 10/14/2010 11:55 PM, Robert Ramey wrote:

...

Sergey Voropaev wrote:

...

Beman Dawes <bdawes <at> acm.org> writes:

I think boost::serialization library has very serious flaw. It is lack of backward compatibility for binary archive. I discovered this problem when I did upgrade from 1.34 to 1.43 and 1.44 versions of BOOST. 1.44 library can not work with old binary archive. The reason - format of binary header of archive has been changed. I think 1.45 version should be corrected for this problem and/or we should place some announcement about recommendation not to use the 1.44 version to work with binary archive.

See: https://svn.boost.org/trac/boost/ticket/4660 http://thread.gmane.org/gmane.comp.lib.boost.devel/208588

I"m aware of this, it's just that I haven't the the time to deal with it right now. I've had a lot of trouble with this and I'm warying of trying to fix it without having the time to do it right.

I just read through the bug report. Oh god. <sigh>

I'm open to hearing suggestions. Robert, it sounds like you're not going to be able to fix this any time soon, is that right? Then it sounds to me like binary serialization is just broken, full stop. If we decide to push ahead with 1.45, my opinion is that we put a #error in the binary serialization headers so that anybody who tries to use it gets a hard error. If you care about binary serialization, the answer is just to not use 1.45.

And not use 1.44 either, but that ship has sailed ... without even so much as a release note. I have to say, as a release manager, I'm seriously pissed off. Can anybody comment on why there isn't any note AT ALL about serialization on http://www.boost.org/users/news/version_1_44_0?

I presume because if we knew about this breakage, we would not ship 1.44 with it? - Volodya

Bryce Lelbach wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

I have some free time today, so I will start working on a patch for this. I've given this particular problem consideration in the past and I'm pretty confident that I can have a fix done today or tomorrow.

For now, please don't spend time on this. If it were a simple fix it would have been done already. Everyone who has looked at this so far (including myself) has failed to consider one or other aspect of this situation and this has actually made things worse. I will make another pass at this so everyone please by patient.

I have already wrote one (semi-extensive) patch for Serialization (http://thread.gmane.org/gmane.comp.lib.boost.devel/208082) which has been signed off on by Ramey but has still not been applied because my requests for SVN access have gone unanswered. So, before I do any work on fixing this I would like some assurance that the time and effort I invest will not go to waste.

Bryce has re-implemented the xml_loader in terms of the the current version of spirit. This is something that people have asked for and he has stepped up and done it. He has coordinated with me and has run all the serialization tests with his new code on his own machine. I've endorsed trunk access privileges for him (weeks ago) but as far as I know nothing has happened. I would have hoped to see his code tested in the test matrix by now. This is orthongonal to the current more urgent problem, but interesting in any case. Robert Ramey .

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 15 Oct 2010 21:17:29 +0400 Vladimir Prus <vladimir@codesourcery.com> wrote:

Also, it is my experience that boost-owner@lists.boost.org is pretty quick at giving SVN access.

- Volodya

I sent two seperate emails to boost-owner@lists.boost.org weeks ago Robert, okay. Good luck! - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky4jusACgkQO/fqqIuE2t71BgCfc/9BTJuCoGsuv4spZW/ZTIdr ZXcAn0u5LPnGpcdlpI/pcrnhKdL13vp0 =GffQ -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have committed the changes. How do I check the build reports from the Boost build bots? (the new Serialization XML grammar passes all tests on Linux). On Fri, 15 Oct 2010 18:54:49 +0100 John Maddock <boost.regex@virgin.net> wrote:

...

...

Also, it is my experience that boost-owner@lists.boost.org is pretty quick at giving SVN access.

- Volodya

I sent two seperate emails to boost-owner@lists.boost.org weeks ago

Hmmm, I don't remember seeing those, if you contact me directly I'll set up SVN access for you.

HTH, John.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

- -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky6L9wACgkQO/fqqIuE2t4QoACgn3+PCwrv7LMrqp/yPdqN8H4/ 47AAnA51lHDxQS8Z5XBcRozacgqd7Run =grNj -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mhm, thanks. Seems an extra ; from another serialization change broke things on older gcc variants (I fixed it). Ramey, it looks like the new XML grammar breaks on Intel's compiler; according to the build logs the compile timed out, so I'm guessing icc chokes on the expression templates. I don't have access to a machine with icc, but I'll work on getting that fixed. (I've built and tested already on MSVC 10 and Linux; I'm pretty confident the only other new failures will be borland, sun and maybe MSVC 6) - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky6t+YACgkQO/fqqIuE2t5y3QCguXZ17+1itizc5eZamVP4gS73 oJYAoO1maxAb4qHrDCm0WQf+Q+aKcGtI =EJ6b -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 17 Oct 2010 09:25:35 -0800 "Robert Ramey" <ramey@rrsd.com> wrote:

Pending issues for the serialization library that hope to see fixed soon.

a) screw up that prevents loading binary archives created by some previous versions of the library.

Just to confirm, you're looking into this?

b) an auto linking issue which shows up as a warning when linking with msvc, and failure to build the library with MINGW.

Want me to look into this?

c) The sun compiler in the test matrix fails to build because an usage of locale inhibits the stream build. I don't know if this is an issue the the standard library implemention being used or what. I'm guessing it would be too hard to fix for someone who has that compiler and an interest in addressing it.

People use suncc? I'm registering for a free student copy of icc, hopefully I'll be able to debug the issue on icc soonish. - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky7QNkACgkQO/fqqIuE2t6Q2QCeLtVZ9oQO1Ra1WyVd0Mk5dIHv M1MAn3s/vvEUw6Ktzf0Nd/qDJVVk+9vR =wgtZ -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ramey, just to make you aware: The new XML grammar adds another fatal bug to serialization when building with mingw (there's already a set of issues with mingw) - mingw causes a win32 exception (a stack overflow) when compiling the Qi grammar. I did my best to try and come up with a fix for both this and the existing mingw issues, but I've had no luck. Documentation on debugging mingw is lacking, so I doubt I am going to be able to get serialization to compile with mingw I've got a copy of Intel's compiler and MSVC 7.1 now (both of which choke with my patch). Hartmut Kaiser indicated that the issue I'm having with MSVC 7.1 is something that has been encountered before with Qi, though he thought it was fixed. So it sounds like this is something that is fixable. Intel's compiler seems to be freezing when compiling the grammar; this can probably be dealt with by refactoring the grammar. Other than those issues, my changes seem to be working fine. - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky72iMACgkQO/fqqIuE2t60dwCgxhnZDUe1r9GdzqxpZw3Gz7yb H5sAniUYYSoLEBYI8UPmBuoa5mXNVtEO =0fsm -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 18 Oct 2010 09:15:14 +0200 Christopher Schmidt <mr.chr.schmidt@online.de> wrote:

Am 18.10.2010 07:24, schrieb Bryce Lelbach:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Ramey, just to make you aware:

The new XML grammar adds another fatal bug to serialization when building with mingw (there's already a set of issues with mingw) - mingw causes a win32 exception (a stack overflow) when compiling the Qi grammar. I did my best to try and come up with a fix for both this and the existing mingw issues, but I've had no luck. Documentation on debugging mingw is lacking, so I doubt I am going to be able to get serialization to compile with mingw

I have pretty much the same problem when compiling my projects with gcc 4.5.1 i686-pc-mingw32. So far just splitting the source files or manually increasing the size of the reserved stack of the c++ driver helped me.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45201

-Christopher

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Thanks Christopher! This is very helpful. I can instantiate the rules in seperate source files, perhaps. :) - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky8OAEACgkQO/fqqIuE2t6ZRQCbBPmfHf98u4Mz0J6EQ46pe8+4 YO0AmwWF+rTV2UHDl4UjhTut2ORXxNGQ =GvjE -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 18 Oct 2010 11:19:01 +0400 Vladimir Prus <vladimir@codesourcery.com> wrote:

Bryce Lelbach wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Ramey, just to make you aware:

The new XML grammar adds another fatal bug to serialization when building with mingw (there's already a set of issues with mingw) - mingw causes a win32 exception (a stack overflow) when compiling the Qi grammar. I did my best to try and come up with a fix for both this and the existing mingw issues, but I've had no luck. Documentation on debugging mingw is lacking, so I doubt I am going to be able to get serialization to compile with mingw

This sounds really disturbing. XML is generally not rocket science, and serialization does not use very much of XML even, so if gcc (even if running on windows) is unable to compile an XML parser, maybe something is wrong with the parser?

Volodya, The fact is, while XML might not be rocket science to the end user, the implementation of a proper XML parser -is- rocket science. Serialization does not feature anything near a full, validating XML parser. However, consider this. * A well-formed XML parser (e.g. excluding semantic analysis) must implement 84 production rules (5 are deprecated). * The average XML production rule is an expression with 4 (appx 4.2145 to be exact) terminals or non-terminals. To give you an idea of what an expression template of that size looks like to the compiler, you might want to take a look at http://tinyurl.com/37s6qqp. * When I talk about the new XML grammar, I am not referring to a newly designed grammar. While my initial instinct was to rewrite Ramey's grammar entirely, in the end I decided to simply port the grammar to Qi. With only a few minor exceptions, the expression templates in the new XML grammar are almost identical to the expression templates in the old Spirit Classic grammar * Existing C++ compilers were not designed to handle C++ TMP gracefully. To elegantly compile code built with expression templates (e.g. code built on top of the Proto library), one would need a C++ compiler that has been designed with specific consideration given to advanced C++ programming techniques such as template metaprogramming. Until I write that (check back in with me in a few months), the best we have is GCC or clang. * GCC does not have any problems with this code on Linux whatsoever; I have compiled Spirit projects with GCC in the past that pushed the compiler to it's limits. This is not one of them. Relatively speaking, this grammar is pretty lightweight. Despite all of the above, you are correct. The parser is at fault, because the parser is the code that I have the ability to manipulate. In an ideal world, I'd be able to fix the underlying problem with mingw, get mingw to accept my patch, and get mingw serialization users to update their mingw to trunk. However, I am content to try and fix things by modifying my grammar. Unfortunately, the problem arises from an ICE, and I have been unable to find any reliable instructions on how to build mingw with debugging symbols (the mingw configure script from mingw's svn trunk will not work on either Windows or Linux; the mingw website is outdated and information is sparse). So, at this point I would taking shots in the dark. Usually, if I have a Spirit grammar that causes a GCC ICE on Linux, I can run GCC under GDB and locate the specific part of my code that it is choking on (I used to do some work on GCC, so I have a decent understanding of it's internals, enough to allow me to extract the information I need in GDB). My hope is that after I've spent some time investigating the ICE on Intel's compiler, I'll have a better idea of where in the grammar the problem is happening. Using my template instantiation profiler and GCC's - -fdump-class-hierarchy/-fdump-tree-all options, I can easily locate the template metaprograms in the grammar that have the greatest time complexity and memory costs. However, given that GCC on Linux can compile this code without significant strain on system resources, and given that mingw is causing a win32 stack overflow exception, the location of this issue might not have any correlation to the most intensive metaprograms in the grammar. If mingw documentation was less sparse, my motivation to fix this would be much greater. I'll certainly continue to work on this (if I don't make any progress I'll contact the mingw mailing list), but my exception is this will not be easy to pin down. - Bryce - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky8N3oACgkQO/fqqIuE2t7iqQCgsHaOCYFfMENJQ9iAv9QjLYx2 F6oAnRFz+r6BLS69WNWJ5p8S47Fw25KH =s3NZ -----END PGP SIGNATURE-----

On 10/17/2010 10:28 PM, Robert Ramey wrote:

Bryce Lelbach wrote:

...

On Sun, 17 Oct 2010 09:25:35 -0800 "Robert Ramey" wrote:

...

Pending issues for the serialization library that hope to see fixed soon.

a) screw up that prevents loading binary archives created by some previous versions of the library.

Just to confirm, you're looking into this?

yep

Robert, what's the current status of this issue? I notice there's been no activity on this bug: https://svn.boost.org/trac/boost/ticket/4660 -- Eric Niebler BoostPro Computing http://www.boostpro.com

Vladimir Prus wrote:

Robert Ramey wrote:

...

Bryce has re-implemented the xml_loader in terms of the the current version of spirit. This is something that people have asked for and he has stepped up and done it. He has coordinated with me and has run all the serialization tests with his new code on his own machine. I've endorsed trunk access privileges for him (weeks ago) but as far as I know nothing has happened. I would have hoped to see his code tested in the test matrix by now.

What's wrong with applying a patch yourself?

Ahhh - that's a very good question. I have a very good reasons for not doing this. First, it's not just a "patch". It's recoding the xml_parser in the latest version of spirit. Second, it's not just "applying the patch". It's also monitoring how the tests go on all the platforms in the matrix. It's often the case that some or other combination of os, C++ compiler, standard library implementation or whatever fails and then one has to spend an unexpected amount of time tracking down the problem. It's also quite possible that this is not the end of it. When it get's unleashed to users, it's possible that other issues will arise. I see this as unavoidable since spirit pushes the envelope of what some compilers can do. So, I could say I don't have the time right now - and it would be true. I could also say I have more urgent issues regarding the library which should be addressed first - and that would also be true. I could also say I work with pottery barn rules - "you break it - you bought it" (pottery barn is a chain of stores in the US which sells pottery and other fragil objects). That would also be true. But that's not my reason. My real reason is that I want to make my job smaller by permitting those who have an interest in the library to take responsability for the aspects that interest them. This will make my job smaller and increase the amount of effort which will be expended in in proving the library. This "hands off" attitude has resulted in the MPI version of serialization, now Bryce's effort, and now someone want's to take responsability for the utf-8 conversion facet. It has also permited numberous other libraries to add thier own serialization support without any effort or conflict with what I'm doing. Regretable, I haven't been able to accept all proposals since sometimes they conflict with my view of what the library should be, but I've tried to accomodate the enhancements that I can. I also have a "To Do" list at the end of the documentation which anyone is free to work on. There are some things I reserve for myself alone. e.g. fixing my most arcane bugs, refining interface and concepts, etc. Robert Ramey

Vladimir Prus wrote:

Robert Ramey wrote:

...

It's also quite possible that this is not the end of it. When it get's unleashed to users, it's possible that other issues will arise. I see this as unavoidable since spirit pushes the envelope of what some compilers can do.

I am not sure why you think that giving Bryce SVN access automatically makes those issues handled by him -- at least, SVN documentation does not mention such effect ;-)

It's got nothing to do with SVN. It's really a matter of allocation of responsability and authority. Bryce want's to upgrade the xml_archive implemenation. He's convinced me that he's willing to do the work. So I've decided to give him the responsability for doing this and along with that I delegate to him the authority to make the changes. Of course, along with that authority goes the responsability to maintain the conceptual integretity, testing standards, portability, etc of the library (such as they may be). The only way any effort on this scale can be successful is to permit the decoupling of efforts by dividing the job into pieces and delegating responsability and requisite authority to those interested and qualified.. SVN supports this - as it must but's not really about SVN.

...

I could also say I work with pottery barn rules - "you break it - you bought it" (pottery barn is a chain of stores in the US which sells pottery and other fragil objects).

[Aside: I was surprised at idea that US laws even permit such practice, and yeah, Wikipedia claims this shop does not have such practice and it's illegal anyway]

lol - damn - another urban myth busted. I'm disappointed it's not true for the pottery barn. But I can tell you for a fact that it's true for the serialization library. Robert Ramey

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 17 Oct 2010 13:08:18 -0700 Patrick Horgan <phorgan1@gmail.com> wrote:

On 10/17/2010 10:17 AM, Robert Ramey wrote:

...

Vladimir Prus wrote: ... elison by patrick...

...

[Aside: I was surprised at idea that US laws even permit such practice, and yeah, Wikipedia claims this shop does not have such practice and it's illegal anyway] lol - damn - another urban myth busted. I'm disappointed it's not true for the pottery barn. But I can tell you for a fact that it's true for the serialization library. The concept shall forever after be called the "serialization library rule".

Patrick _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

How about, the serialization barn rule? :P - -- Bryce Lelbach aka wash http://groups.google.com/group/ariel_devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAky7XAAACgkQO/fqqIuE2t7EjACfYdR+DlP2X16qqpTUrIhcS/fy yLwAniKtjoh8StP3ROjGVpxvODoMRy2l =QaVw -----END PGP SIGNATURE-----