March 2008 - Boost - lists.preview.boost.org

[gsoc 2008] Neural Networks library
by Milan Rusev 26 Mar '08

26 Mar '08

Hello, I'm an AI student and I'd like to participate in this year's SoC with an Artificial Neural Networks library. The library will consist of basic elements of a NN: neurons, layers, teaching algorithms, neuron functions (plus common usages: linear, sigmoid, gaussian, winner-takes-all). It will also include the implementation of some popular NNs: Multilayer Perceptron, Hopfield Network, Kohonen Network. I know it's a bit of a high level library, and I'd like some feedback whether it's appropriate for Boost. I also have a question. The proposal on the GSoC 2008 page for the implementation of Fixed Decimal also sounds appealing to me. So should I choose one project and apply for it or should I apply for both of them separately? Milan

6 8

validating explicit-failures-markup.xml?
by David Abrahams 26 Mar '08

26 Mar '08

The comment at the top of explicit-failures-markup implies the following command should work, but it doesn't for me. What am I missing? $ xmllint --valid explicit-failures-markup.xml --schema explicit-failures.xsd explicit-failures-markup.xml:3: validity error : Validation failed: no DTD found ! xsi:noNamespaceSchemaLocation="explicit-failures.xsd"> ^ ... -- Dave Abrahams Boost Consulting http://boost-consulting.com

4 3

Proto review: a humble request
by Eric Niebler 26 Mar '08

26 Mar '08

Proto's review ends this Friday. There's been good discussion, but very few actual reviews. Please consider writing a review, if you haven't already. Thanks in advance, -- Eric Niebler Boost Consulting www.boost-consulting.com

4 7

[1.35][intel-win-10.0] Is this toolset just broken?
by David Abrahams 26 Mar '08

26 Mar '08

http://beta.boost.org/development/tests/release/developer/issues.html looks really suspicious to me. Things are failing that never failed on any recent compiler in the past. When I run the Boost.Python tests locally, I get at least one crash (!) that isn't reflected in the results above, suggesting a codegen bug. I'm trying to figure out how best to address these issues, and am wondering 1. whether all these errors ought to be marked up 2. whether we really ought to be making this compiler a required release platform Thoughts? -- Dave Abrahams Boost Consulting http://boost-consulting.com

3 4

My Proto Review
by Joel de Guzman 26 Mar '08

26 Mar '08

Hi, I'm sorry if this is very late. This month has been a whirlwind for me. So many things happening. ***I vote to accept Proto.*** - What is your evaluation of the design? Proto is very well designed. I've seen it progress through the years. It's very mature and stable now. I've been using it as early as 2005 as a Spirit-2 core library. Spirit-2's design and evolution somehow influenced Proto's design. I'm very happy with Eric's design decisions. Some history... Eric and I had our share of squabbles in the past ;-) related to how I expected Proto to be. I knew from the start that I needed something like it. Joao Abecasis, a Spirit developer, started one like it, from my urging. Fusion was developed, in part, due to the need for manipulation of heterogeneous data structures, specifically for spirit: syntax trees. I recall that our biggest disagreement, and one that we resolved soon enough, was my need for flattened sequences. Eric insisted on presenting the results with no flattening, I wanted flattening. For example, I wanted: a >> b >> c >> d to be a flattened fusion sequence, not a tree. I was reluctant to have Spirit go through a 2-step process that necessitated a conversion from a tree to a fusion-sequence: proto-tree-->fusion-sequence I had code that didn't use Proto, that produced tight code and fast compile times. The challenge was for Eric to beat that or at least come close to its performance. I wanted to take advantage of fusion algorithms to *efficiently* iterate over the sequence. At first, this lead to the development of segmented fusion iterators. Eric actually implemented this: http://lafstern.org/matt/segmented.pdf into Fusion! It was OK and efficient at runtime. Unfortunately, compile time suffered considerably. Our endless discussions culminated with: http://article.gmane.org/gmane.comp.parsers.spirit.devel/2765 Seems some folks were tuning in to our discussions. The C++ version of "Scrap Your Boiler Plate", a very powerful, well studied and widely used design pattern for generic traversal from the Haskell world: http://portal.acm.org/citation.cfm?id=1159861.1159871 references Eric's post above. So far so good, yet, SYB ("Scrap Your Boiler Plate") was not available in a mature form in C++ yet. This then, again, lead to Dan Marsden's proposed traversal library. What Eric ended up doing was to provide a zero-overhead tree structure that's converted very efficiently to a flattened fusion sequence. It was as fast as my non-proto Spirit2 implementation and with some compile time performance hit. In due time, I'll be using Dan traversal library. At any rate, Eric's intuition to not flatten the trees is dead-on correct. It's the right approach. I'm glad that it influenced interesting solutions to interesting generic programming problems. I was happy :-) Proto proved it's worth. Oh, there were more disagreement on things big and small. Development happened in an evolutionary manner spanning years. If you guys are interested in what transpired, it's all archived in the Spirit-devel list. I gave him a very tough time :-) The space here won't do justice to what actually happened, nevertheless, I already did my review a long time ago, spaning years. In the end, Eric pulled it all through. I gave him a very tough time :-) - What is your evaluation of the implementation? A+ - What is your evaluation of the documentation? A+ - What is your evaluation of the potential usefulness of the library? Oh man! - Did you try to use the library? With what compiler? Did you have any problems? Yes. With all compilers required by Spirit. No problems. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? Many years of study; from its inception. - Are you knowledgeable about the problem domain? Yes. very. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net

3 5

[smart_ptr] new shared_from_this behavior with multiple owners
by Frank Mori Hess 26 Mar '08

26 Mar '08

I just want to point out that even with the fix that went in today, the behavior of enable_shared_from_this::shared_from_this when multiple shared_ptrs take independent ownership is inconsistent with the old behavior. The old behavior was that shared_from_this would return a shared_ptr that shared ownership with whichever shared_ptr took ownership of the object last. The new behavior is shared_from_this always returns a shared_ptr that shares ownership with the first shared_ptr to take ownership of the object. The attached example illustrates the difference when compiled against the old lib and svn trunk. I don't want to make a big deal of this, as I actually think the new behavior is more sane, but I thought I should point it out. My preferred solution would be to just point out the change in the next release notes or something like that. -- Frank

2 3

1704 [any] 4512 warning from VC8/9 on warning level 4
by Alexander Nasonov 25 Mar '08

25 Mar '08

Hi, I'm wondering if we care about level 4 MSVC warnings. As far as I remember default warning level is lower and I don't see why boost sources should have MSVC-specific pragmas just to disable level 4 warnings. I'd rather resolved the bug as invalid. As an alternative, we could // Beginning of file #include <boost/config/warnings_off.hpp> ... // end of file #include <boost/config/warnings_on.hpp> and have an option BOOST_CONFIG_TURN_WARNINGS_OFF. Though, I don't know whether it's doable for other compilers. -- Alexander

5 7

[trac] email addresses not mangled
by Maik Beckmann 25 Mar '08

25 Mar '08

Hello, I just reported a bug http://svn.boost.org/trac/boost/ticket/1715 and as I see my email address appears to be not mangled. I guess this is not by intention, so please fix it (I hate S P A M)! Best, -- Maik

3 3

[gsoc2008] Draft proposal for cpp-netlib and pion-net integration
by Divye Kapoor 25 Mar '08

25 Mar '08

Hello developers, I'm attaching below a draft proposal for merging the cpp-netlib and pion-net projects. Its still in the making stage and I would specially like your feedback about the timeline and the deliverables. I've deliberately kept the timeline quite loose to allow for unforseen delays but I really want to get an estimate of how much more can be squeezed into the project without making the it overambitious given the timeframe involved (I'm pretty sure that implementation of rfc1867 would require less than 3 weeks but I would like more feedback on this point). Also, I would like to know if any of the Boost library authors would be interested in mentoring this proposal because without them, this proposal is a sunken ship ;-) Yours sincerely, Divye Kapoor PS: Its not yet in the prescribed format for an application, but there's an almost one to one mapping. ---- Synopsis During the course of the Google Summer of Code, I would like to work on the cpp-netlib [1] and the pion-net [2] projects to ensure that pion-lib HTTP client-server library is seamlessly integrated with the general messaging framework specifed in the cpp-netlib architecture [3]. This will not only address one of the most important use cases of the Boost Asio library as it will provide a general, cross platform implementation of network communication via the the HTTP protocol but it will also lower the entry barrier for creating cross-platform, network aware C++ applications. Project While working on this project, I will have a large codebase to build upon (~ 15000 lines in the pion-net project). To ensure robustness of the code, my first priority will be to create a test suite with Boost.Test and Python (using the BaseHTTPServer module) to ensure standards conforming behaviour of the initial code. After a testing suite is set up and any discovered bugs are removed, I would proceed to refactor the pion-net interface of the library to conform to the interface exposed by the cpp-netlib message classes. This would involve non-trivial changes in the code base as it would require conversion of the pion-HTTP* classes to conform to the transformation layer restrictions of the cpp-netlib classes. This might cause breaking changes to the implementation. All such changes will be documented and appropriate test cases generated. Support for parsing multipart-form data (both encoding and decoding) will then have to be coded in accordance with rfc1867 [4]. Currently, the implementation correctly handles only GET parameters and POST parameters encoded using url-encoding. This creates a limitation of not being able to send and receive files and other such binary chunks of data via a POST request over the network. A correct implementation will enable multiple files potentially as large as 2 GB to be sent and received correctly over the HTTP protocol. Robustness of the code will be a serious challenge in this case as disk file security and large file sizes have to be handled. Ensuring reliable operation in the face of timeouts and incomplete transfer will form an important consideration of this part of the project. This will take up a bulk of the time in the project. The multipart/form-data handling capabilities will then be augmented by support for Compression headers (via gzip compression). This will enable efficient communication on bandwidth constrained networks and will be useful even for simple HTTP requests as this compression will be handled in a transparent manner. Complete support for cacheing will then have to be built in to the HTTP request/response model. In particular, support for the Cache-control directives and Expires directives will have to be built into the client to prevent unnecessary bandwidth usage. Documentation and examples concerning the most common use cases will be created and in case any time is left, a Boost Extension based plugin system will be created to allow easy extension of the HTTP protocol stack to handle custom headers. Timeline Prior to actual coding: April 14-May 26: Finish off examinations at college. Examinations till 13th May. Get familiarized with boost extension, Boost.Test, Boost.Asio, gzip, pion-net and cpp-netlib projects May 26 - July 7: 6 weeks mark July 14: Midterm evaluation 2 weeks - Create a testing suite, ensuring automation and ironing out any unexpected bugs. 2 weeks - Break the old interface and make it conform with the message transform API. 3 weeks - Implement the multipart/form-data encoding and decoding algorithms. Generate test cases. Clean up code July 14-August 11: 1 week - Document all the code changes and the new API to which it conforms. 1 week - Implement the compression header support via optional gzip support 2 weeks - Implement the Cacheing directives. < 1 week - Clean up supporting docs, test cases, etc. I believe that the timeline is flexible enough to handle any unforseen bugs and eventualities that tend to crop up yet is tight enough to keep me busy. Deliverables A feature complete HTTP 1.1 header only implementation conforming to the message <==> rfc data structure mapping of the cpp-netlib project resulting in an integration of the pion-net and cpp-netlib projects released under the Boost Software License. Documentation, Test and Use cases for the most common scenarios will be provided. The library will compile and run successfully on atleast 2 platforms (intended Win32 and linux) and with 2 compilers (gcc 4.2 and MSVC 8) in accordance with Boost portability policies and will follow the Boost naming conventions. Benefits for Boost A network library supporting HTTP(S) and the most common use cases will be ready for review. Learning Experiences in store As a result of this project, I'll gain valuable experience in coding to rfc specifications and gain valuable insight into the working of the HTTP protocol. In addition, I'll get a hands-on experience in working with Boost.Asio and achieve a deeper understanding of the Boost libraries. About Me I'm a 19 year old student from the small town of Jamshedpur, India. I'm pursuing my Integrated Dual Degree course in Computer Science and Engineering (Bachelors) with a Masters specialization in Information Technology at the Indian Institute of Technology, Roorkee. I have a strong algorithmic and coding background. I've been ranked 2nd at the International Informatics Olympiad 2005 and 7th All India at the National Cyber Olympiad 2006. I've been coding in C++ for the past 6 years and I have been using the STL for the past 3 years. Over the past year, I've come to use the Boost.Spirit, Boost.Bind and Boost.Lambda libraries and I have gained valuable experience from them. Now, I wish to make my own contribution to Boost by helping to bring network abstraction to the Boost library. [1] http://sourceforge.net/projects/cpp-netlib/ [2] http://www.pion.org/projects/pion-network-library [3] http://cpp-netlib.wiki.sourceforge.net/Architectural+Documentation [4] http://www.ietf.org/rfc/rfc1867.txt -- Whether the chicken crossed the road or the road moved beneath the chicken depends upon your frame of reference. My official web site: http://www.iitr.ernet.in/studenthomepages/homepages/061305

1 0

[gsoc2008] Database Library (asynchronous)
by Daniel Pfeifer 25 Mar '08

25 Mar '08

Database access has been discussed quite often in the Boost mailing list. I read through the discussions from Oct 2004 and Jul/Aug 2005 and I am well aware of the fact that there is a lot of disagreement concerning the interface presented to the programmer. So I just hope you don't bite my head off if I propose this as a GSoC project. ;) Things I would like to implement: * generic binding of parameters and results (using Boost.Fusion) * asynchronous access (using Boost.Asio) * odbc, postgres, firebird, mysql and sqlite3 backends (using their proper C API) * DATE, TIME and DATETIME fields (using Boost.Date_Time or struct tm) Surely other backends can be added, but these are the ones I am (more or less) familiar with. BLOBs would be interesting to implement but I am afraid time won't permit it. The following code snippet is a quick draft using sqlite3 as a backend. This is not the interface I am going to propose, please read on. boost::asio::io_service io_service; boost::database::sqlite3::connection conn(io_service); conn.open( "testdb" ); std::cout << "client version: " << conn.client_version() << std::endl; std::cout << "server version: " << conn.server_version() << std::endl; boost::database::sqlite3::statement stmt = db.statement(); stmt.prepare( "CREATE TABLE employee" "(id INT,name CHAR(20),salary INT,PRIMARY KEY(id))" ); stmt.execute(); // prepared statement with placeholders stmt.prepare( "INSERT INTO employee (id,name,salary) VALUES (?,?,?)" ); stmt.execute( boost::fusion::make_vector( 27, "Horst Zwiebelhacker", 5000) ); stmt.prepare( "SELECT name, salary FROM employee" ); stmt.execute(); // fetch directly into specified results std::string name; int salary; while( stmt.fetch( boost::fusion::vector_tie(name,salary) ) ) { std::cout << name << " gets " << salary << " pebbles." << std::endl; } The "open", "prepare" and "execute" functions have "async_" counterparts. The sync functions throw exceptions while the async ones pass an error to the bound handle as the first parameter. Both the synchronous and the asynchronous test programs run fine, though they are not thread safe (the final library definitely will be). I checked if it would be straightforward to implement other backends, which turned out to be true for most of them. I struggled with PostgreSQL where the type of the parameters has to be known when the statement is prepared. The syntax suggested by Tito Villalobos for declaring a statement seems to be a better approach (slightly modified by me): statement< parameters<...>, // query/stored procedure parameter types results<...> // recordset column types > my_stmt(connection, query_string); This would make it easier to prepare statements in PostgreSQL and can be a benefit for other backends too. Statements could be automatically prepared when their execute function is called the first time. Finally, variadic templates in C++0x allow to hide the functions boost::fusion::make_vector and boost::fusion::vector_tie from the programmer, which makes the interface much more intuitive. Imagine stmt.execute(27, "Horst Zwiebelhacker", 50000); or while( stmt.fetch(name,salary) ) ... isn't that lovely? Just for clarity: The design I have in mind will provide the programmer with an easy way to use SQL. Still the programmer has to write her own SQL strings. A library which provides STL functionality by abstracting SQL could be built on top of this one. I would like to concentrate on making it asynchronous rather than on the STL part. Cheers. Daniel Pfeifer

9 15