- Boost - lists.preview.boost.org

IOStreams zlib_decompressor does not check crc for empty string
by John Wallace 05 Jun '15

05 Jun '15

When decompressing an empty file, the zlib_decompressor does not check the crc, which can cause an exception to be raised on a correctly-formatted gzip file, especially when the gzip file is created by concatenating compressed blocks. This comes up especially when working with bgzip files (see specification at: http://bioinformatics.oxfordjournals.org/content/27/5/718.full; all bgzip files end in a special empty block) I have attached a demonstration file (zlib_test.cpp) that illustrates the problem. This program simply reads a compressed input line by line from stdin and outputs to stdout (nearly equivalent to zcat). Assuming we compile to a.out, the following BASH commands illustrate the issue: $ cat <(echo "foo" | gzip -c) <(echo -n "" | gzip) <(echo "bar" | gzip) | ./a.out foo Whereas, we expect the following: $ cat <(echo "foo" | gzip -c) <(echo -n "" | gzip) <(echo "bar" | gzip) | zcat foo bar I have attached a patch to zlib.cpp (base version 1.51, but I confirmed that it persists in 1.58) which I believe should resolve the issue by simply setting the crc to 0 when an empty string is decompressed. Thanks for your time, John Wallace

1 0

[metaparse] review
by Andrzej Krzemienski 05 Jun '15

05 Jun '15

Hi, I wish I had more time to thoroughly review and test the implementation, but as this is not possible I will have to take risk and vote without the in-depth investigation. I vote YES, this library is needed in Boost. Although it will be used only by a fraction of community, it is an essential complimentary part to the Standard C++ user-defined literals (in raw form). The Standard offers a way to decompose a a user-defined literal into a sequence of characters each passed as a separate template argument. This enables to build at compile time constants of user defined numeric types, but doing it is hard, and the Standard offers little help. This is where Abel's library comes into rescue. Although it was designed to do much more, it also fills this sort of hole in the Standard. I have read Abel's posts, and know he has been working on the subject for a couple of years now, so I am confident in his competence. In a way, I see two components inside: (1) A tool for converting a string literal into instantiation of string<> template. (2) tools for parsing those string<>s. I see use cases for each of these components separately. Parsing templates with characters, is painstaking but fairly easy, so I may want to do it myself, provided I have a way to convert a string literal to a template. I was trying to hack it for a long while, without success. There is quite a bit of trickery put into it. It is really ingenious. On the other hand, when processing my raw user-defined literals, I may need your parsers, but I care not about BOOST_MS_STRING() (or how you want to call it). The standard questions: - What is your evaluation of the design? > Adequate: this is string parsing what the library does, so the parser interface seems the right choice. > - What is your evaluation of the implementation? > I haven't looked at the parsers. Regarding, BOOST_MS_STRING(), it is a nice trick to implement the impossible. > - What is your evaluation of the documentation? > Well, after having read it, I understood what to expect of it, so in that sense, it is adequate. But there is something about it that makes it "heavy". Maybe it is just the problem domain, I do not know. > - What is your evaluation of the potential usefulness of the library? > It is useful, although to a fraction of community: libraries with user-defined literals, printf, other EDSLs. > - Did you try to use the library? > I tested BOOST_MP_STRING() (couldn't believe it works). - With what compiler? MinGW on Windows, GCC 4.8.2 - Did you have any problems? > Nope > - How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? > I read the docs (not all of them); downloaded and played with BOOST_MP_STRING(). -- two hours. - Are you knowledgeable about the problem domain? I never had to do parse at compile time for my commercial programs, but as a hobby I studied the subject, and have even written some posts about it. Thanks Abel, the library is cool. --&rzej

2 5

[metaparse] Update review
by Edward Diener 04 Jun '15

04 Jun '15

This is an updated review of the metaparse library. In my initial review I reacted to the fact that library presentation had changed during the review period. Although I feel I made a point I was too harsh in doing so. So now I would like to review the library as it has been presented in its form suitable for review as a modular boost library. - What is your evaluation of the design? The design of the library appears well thought out to me, providing both low-level and higher-level metafunctions for use. The higher level metafunctions take other metafunctions as input in a classic design of metafunction programming as explained in MPL. The design of the library makes it easy for anybody with knowledge of MPL to understand how to use the metafunctions in the library and to create one's own higher level metafunctions for particular parsing tasks if necessary. A couple of quibbles about the design based merely on my own sense of 'naming'. Since I have roundly criticized "naming wars" <g> in the past I no doubt leave myself open for criticism, but these are just personal suggestions, which I can live without: * BOOST_STRING is very generic and can easily lead to a macro clash. Something like BOOST_METAPARSE_STRING or maybe BOOST_MPS_STRING would be more distinct and safer. I favor shortened names for libraries in macros to be 3 characters rather than 2 characters so as to decrease macro clashes. * The various 'foldxxx' names, even with the Cheat-Sheet explanation of their name formation in the docs, seems needlessly confusing. Would it not be better to make longer, less confusing names, which actually spell out what the metafunctions are about. Is it really so hard to write "fold_left', 'fold_right', 'fold_left_fail' etc. etc. Sometimes clarity trumps how many keys one types. - What is your evaluation of the implementation? I didn't look at the code of the implementation. I generally trust that if the code works as documented an experienced programmer's implementation is his own affair. - What is your evaluation of the documentation? The documentation is fairly profuse approaching the subject of compile-time parsing in many different ways. I found that the work needed to produce a solid set of documentation had been done. I have some quibbles and a few suggestions so let me give it as a general list of items: * A general discussion about the purpose of the functionality of the library would be welcome. Such as the possible metaprogramming scenarios in which a programmer would want to accept and analyze ( parse ) an MPL string in creating his metafunctions. * A general comparison of when it might be better/worse to use Metaparse as oppose to Boost.Spirit. Pluses and minuses of both approaches might be discussed. * On the first page of documentation it mentions the various types of output which Metaparse can generate. Both 'types' and 'metafunctions' are very understandable, but whatever is meant by "Objects" and "Callable C++ functions" doesn't follow the definition of these terms in C++ which I know. Some discussion, or link, of what these two types of output mean in the context of Metaparse should be given. * The Versioning section appears to be wrong in quite a number of places. Please proofread what was written there and correct/update it. * In the Reference section, among the items enumerated at the end are 'Metafunctions and metafunction classes' and "Utilities' but when you click on the links to go the actual headings it looks like the metafunctions are split before and after the utilities. - What is your evaluation of the potential usefulness of the library? I think it give the template programmer another important tool for designing templates for end-users. As such it has intrinsic value for programmers writing templates for end-users to use. - Did you try to use the library? Yes. - With what compiler? I used the Metashell tool to test things out. I then tried manually compiling some examples in the documentation with Mingw/gcc-4.9.2 with '-std=c++11'. I also ran the tests successfully with Mingw/gcc-4.9.2 with '-std=c++11'. - Did you have any problems? No. Everything worked as expected. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? My main effort was in trying to understand how the parts of the library worked with each other. It is not easy to understand everything, and I do not claim that I do, but I could understand from the documentation how the major areas of parsing worked and what metafunctions are used for what tasks. I could also see that if would be easy to incorporate one's own metafunctions into generalized parsing tasks using what Metaparse provides. I did need the full documentation set, which is profuse, to understand the parts of the library. Part of my initial reaction when the library was first presented was that I could only find the documentation for the 'Getting Started' link, and I am one of those people who is just mentally incapable of understanding code from the point of view of "here's how to use" documentation only. Once I discovered the full documentation in the updated presentation of the library in modular Boost form I could see and understand the library much better. - Are you knowledgeable about the problem domain? Generally knowledgable about parsers and how they work, but not an expert. Finally I vote YES to accept the library into Boost, reversing my previous initial vote. The library will prove useful for template programming in some contexts, has obviously been designed thoughtfully, and the full documentation presents everything one would need to use the library, explaining things in various effective ways, one or more of which ways should satisfy all programmers who do template programming.

5 4

Link to Bind library doc broken.
by Alex Mohr 04 Jun '15

04 Jun '15

This link for the Bind library is broken on this page: http://www.boost.org/doc/libs/ The link goes here, which gives a 404 not found: http://www.boost.org/doc/libs/1_58_0/libs/bind/doc/html/bind.html Alex

2 1

Moving the include directory to $BOOST_ROOT/include? (again)
by Peter Dimov 04 Jun '15

04 Jun '15

I want to reiterate my suggestion that we move Boost's include directory from $BOOST_ROOT to $BOOST_ROOT/include. This has the following benefits: - it follows the established naming practice, - it does not make the whole Boost reachable from the include path as it does now, - when building Boost, the message given at the end would be The Boost C++ Libraries were successfully built! The following directory should be added to compiler include paths: C:\Projects\boost-git\boost\include The following directory should be added to linker library paths: C:\Projects\boost-git\boost\stage\lib instead of the current The Boost C++ Libraries were successfully built! The following directory should be added to compiler include paths: C:\Projects\boost-git\boost The following directory should be added to linker library paths: C:\Projects\boost-git\boost\stage\lib which is more in line with a user's expectations. - it's more consistent because it builds the virtual include/ subdirectory from the contents of the multiple libs/*/include/ subdirectories. For backward compatibility, "b2 headers" could create a boost/ symlink to include/boost/. Also for backward compatibility, monolithic releases can continue to ship the headers in boost/ if moving the directory would be seen as too disruptive. Failing acceptance of the above proposal, can we at least use a path constant (BOOST_INCLUDE?) somewhere in Jamroot or boostcpp.jam that identifies where the header links are to be placed? Its current contents would be "." but I would be able to change that to "include" instead of locating and patching all the places that refer to the include path.

8 19

[metaparse] review
by Gordon Woodhull 04 Jun '15

04 Jun '15

Hi all, I believe that Metaparse should be accepted into Boost. > - What is your evaluation of the potential usefulness of the library? As mentioned by others, Metaparse is a library for library developers. In terms of functionality, it is closest to Boost.Proto, although it allows defining arbitrary EDSLs as strings, instead of using the built-in C++ operators. In order to test the effectiveness of Metaparse, I built a parser for the Graphviz "dot" language [1]. This is of practical importance to me, because I have an interest in graph metaprogramming [2], specifically generating types and networks of types from graph metadata. Having a nice language for defining graphs in metadata, to support a syntax beyond angle brackets, would be very helpful. One limitation of this class of EDSL is that it can produce arbitrarily complex types, but it can't refer to any existing types. E.g. you might have the string "int" in your EDSL, but in order for it to refer to the C++ int type, you are going to need an mpl::map from type-strings to types. I didn't get this far in my evaluation, but it's really beyond the scope of the library. > - Are you knowledgeable about the problem domain? Fairly knowledgeable. I have built an experimental parser for compile-time EDSLs over type expressions called "Angly" [4] (presented at C++Now 2012). I have learned from the considerably simpler design of Metaparse, and hope to apply it to my own library (in particular using recursive descent rather than trying to wrangle a deterministic finite automaton). > - How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? I spent about 5 hours getting set up and working through the tutorial, and about 7 hours getting up and running with my own parser. Once I "got it", it only took an hour or two to implement the rest of the grammar (and about 4 hours to write this review). My results (grammar and a few basic tests) are in a Gist available here: [3]. My grammar is able to parse "dot" graph descriptions that look like this "graph g { A [x=y]; \ B->C [y=z, w=a]; \ D->E; \ F; \ G->H [a=b, b=c, c=d, d=e, e=f, f=g]; \ I->J; \ }" Because I ran out of time, my parser only produces an AST; it does not yet produce MPL.Graph or Fusion.Graph types. > - Did you try to use the library? > - With what compiler? > - Did you have any problems? I mostly used metashell for my evaluation, but I also tested compilation times with clang 3.5 once I had something running. Any problems I had were the usual difficulties of TMP. I was not expecting the compile times to be tolerable, but even in metashell, the turnaround was usually around 15-20s. Compiling my longest test with clang, just shy of 256 characters, took only about 4-5 seconds. > - What is your evaluation of the documentation? Excellent. Working through the tutorial was enough for me to absorb the particular constructs needed for building grammars and semantic actions. I also used the reference documentation when building my own parser. This documentation is more terse, very much in the style of the MPL documentation. But it functions. I only skimmed the user manual, since the content looks a little bit stale (doesn't mention build_parser) and somewhat redundant with the tutorial. As Andrzej mentions, there may need to be an introduction concerning the scope of the library and what you can and can't do with this kind of EDSL, where you would use Proto instead of Metaparse, etc. I think that as the library evolves, the documentation will probably need more general parsing background, of the sort you will find in Spirit documentation. For example, the tutorial covers the unary `-` operator, but it doesn't mention why there is no ambiguity with the binary `-` operator. Also, it seems (as mentioned in the design discussion below) that the grammar must be unambiguous, and this is mentioned nowhere. Overall, the documentation is stronger on the meta- than the -parse. > - What is your evaluation of the implementation? I did not peer into the code very much, beyond finding some examples I needed. My impression from Abel's walk-through at C++Now 2012 is that the implementation is quite simple and straightforward, based on established techniques from parsers in Haskell. > - What is your evaluation of the design? Very good. However, I think the API might be fleshed out a little bit once the library gets more usage. In particular, I found myself writing a few helper metafunctions that looked like this: template<typename EdgeDecl> struct declare_edge { typedef edge_decl<typename mpl::at_c<EdgeDecl, 0>::type, typename mpl::at_c<EdgeDecl, 2>::type, typename mpl::at_c<EdgeDecl, 3>::type> type; }; invoked from a transform like this: using edge_stmt = transform< sequence<ID, arrow_token, ID, opt<attr_list, mpl::vector<>>>, mpl::quote1<declare_edge> >; It may be my inexperience, but it struck me that it would be helpful to have a "spread" or "apply" transform so that the items in the mpl::vector produced by sequence would be supplied as parameters to declare_edge, instead of having to pull them out using mpl::at_c. There may be some trick I don't know here. I will mention two other points of confusion. These are probably not design flaws in the library, but I thought I would bring them up in case they are. First, I started out defining my AST nodes like this: template<typename Tail, typename Head, typename Attrs> struct edge_decl {}; Seems like a reasonable basic type-holder to me. As shown above, this is instantiated and returned by a helper metafunction invoked by the transform which pulls stuff out of an mpl::vector. But then somewhere in metaparse, it tries to again "invoke" this result with ::type when appending it to another mpl::vector. Really? So my transform needs to return not the value, but a metafunction which returns a value? I couldn't figure this out, so I gave each of my AST nodes a self-referential ::type in the way that mpl::vector has: template<typename Tail, typename Head, typename Attrs> struct edge_decl { typedef edge_decl type; //why? }; Something is not right here! The second problem I encountered is about ambiguous grammars. For the "dot" language it seems natural to define a statement this way: using stmt = one_of<node_stmt, edge_stmt>; where a node statement looks like ID [attr=value, attr=value]; and an edge statement looks like ID -> ID [attr=value]; But there's an ambiguity here, and once the parser successfully reads a node, it is going to get upset if it sees a `->`. So the rule has to be written the other way: using stmt = one_of<edge_stmt, node_stmt>; Similarly, the `;` separator for declarations is optional in the "dot" language, but I couldn't figure out how to implement this, because if you try to have an optional element at the beginning of a rule: template<typename Parser, typename Default = void> using opt = one_of<Parser, return_<Default>>; using stmt_list = foldlp< sequence<opt<semi_token>, stmt>, transform<stmt, mpl::quote1<mpl::vector1>>, // ... then the parser will greedily eat the epsilon and have nowhere to go (at least according to my amateur understanding of parsers). I gave up here and made the separator mandatory. Again, this is probably not a flaw in the design, but the documentation should describe what kinds of grammars will work, and perhaps give pointers to other resources about disambiguating grammars. Probably there is a different construct which would help here. Cheers, Gordon [1] http://www.graphviz.org/doc/info/lang.html [2] http://metagraph.info/wp-content/uploads/2011/05/mpl.graph_.pdf [3] https://gist.github.com/gordonwoodhull/dcbb332eea06dd3279b9 [4] https://github.com/gordonwoodhull/metagraph

2 5

[build] Error building bjam on cygwin
by Emil Dotchevski 03 Jun '15

03 Jun '15

I'm trying to build bjam from Boost 1.56.0 under cygwin. Running ./build.bat mingw errs with: gcc -DNT -o bootstrap\jam0.exe command.c compile.c constants.c debug.c execcmd.c execnt.c filent.c frames.c function.c glob.c hash.c hdrmacro.c headers.c jam.c jambase.c jamgram.c lists.c make.c make1.c object.c option.c output.c parse.c pathnt.c pathsys.c regexp.c rules.c scan.c search.c subst.c timestamp.c variable.c modules.c strings.c filesys.c builtins.c md5.c class.c cwd.c w32_getreg.c native.c modules/set.c modules/path.c modules/regex.c modules/property-set.c modules/sequence.c modules/order.c filent.c:46:20: fatal error: direct.h: No such file or directory #include <direct.h> ^ compilation terminated More errors follow. What am I doing wrong? -- Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

2 1

[thread] Problem with thread test 'failed to write output file'
by Edward Diener 03 Jun '15

03 Jun '15

Attempting to run the 'thread' tests locally I get: failed to write output file '..\..\..\bin.v2\libs\thread\test\upg_lock_cons_move_ctor_sh_lock_untl_p_lib.test\msvc-12.0\debug\threading-multi\sync\mutual_exclusion\locks\upgrade_lock\cons\move_ctor_shared_lock_until_pass.obj.rsp'! This is with the --abbreviated-path option. If I try using --hash or not using either on the b2 command line I still get the same result of 'failed to write output file ... etc,' probably because the output file spec is still too long. This happens on Windows thread tests with msvc-12.0 but not with gcc-4.9.2. Any ideas how I can get this to run successfully ?

1 1

[admin] General Boost email?
by Rene Rivera 02 Jun '15

02 Jun '15

I've been experimenting with the various cloud based CI systems out there in an effort to make it easy to have all Boost libraries CI tested. As part of that I'm trying to set up an organization account in Appveyor and I'm stuck.. As I need a global org email to register. Hence I would like to request that we set up an "owners(a)boost.org" email that we can use for such circumstances (that just forwards to the git boostorg owners). Does that sound OK? Whom would set up such an email? -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

1 0

[test] possible bug in 1.47.0
by Damian Vicino 02 Jun '15

02 Jun '15

Hi, I’m working in the safe_float library and while working on it I defined a partial specialisation of numeric_limits for safe_float. I wanted the safe_float<T> to mimic the T numeric_limits in most of the values and just change a few, but nothing complex, one liners only. Then, I created a test for validating those methods/attributes mimicking the internal T are equal to those on T, as follows: //types to be tested using test_types=boost::mpl::list< float, double, long double >; using namespace boost::safe_float; BOOST_AUTO_TEST_CASE_TEMPLATE( safe_float_numeric_limits_basic_fp_types, FPT, test_types){ BOOST_CHECK_EQUAL(std::numeric_limits<safe_float<FPT>>::is_specialized, std::numeric_limits<FPT>::is_specialized); } The test does not even compile: Undefined symbols for architecture x86_64: "std::__1::numeric_limits<boost::safe_float::safe_float<double, boost::safe_float::policy::check_addition_overflow, boost::safe_float::policy::on_fail_throw, boost::safe_float::policy::cast_none> >::is_specialized", referenced from: safe_float_numeric_limits_suite::safe_float_numeric_limits_basic_fp_types<double>::test_method() in numeric_limits_test.o After looking at the numeric_limits specialisation for a while, I tried changing the test as follows: if (std::numeric_limits<safe_float<FPT>>::is_specialized == std::numeric_limits<FPT>::is_specialized){ BOOST_CHECK(true); } else { BOOST_CHECK(false); } And as follows: BOOST_CHECK(std::numeric_limits<safe_float<FPT>>::is_specialized == std::numeric_limits<FPT>::is_specialized); Both alternatives work. Is there something wrong with BOOST_CHECK_EQUAL? I’m using version 1.47.0 Best regards, Damian

2 1