web service client based on boost?
Anyone tried building a web service client based on existing boost library components such as asio? Like to share, or brag :-) about your experience? A web service server is probably more involved. Doing a client should be a nice first step, and helps build a common foundation forward. Thanks, Greg
Hi Gregory, On Fri, Jul 25, 2008 at 1:31 PM, Gregory Dai <gregory.dai@gmail.com> wrote:
Anyone tried building a web service client based on existing boost library components such as asio? Like to share, or brag :-) about your experience?
When you say web service client, is this just generally anything that passes through HTTP? In this case there's a currently on-going effort (not making too much progress though with other "real world" things going on) to create a networking library on top of Boost.Asio as the base and provide simple implementations of protocol clients. Initial protocol scheduled to be implemented is an extensible HTTP client. You can get more information on the poject here: http://sourceforge.net/projects/cpp-netlib.
A web service server is probably more involved. Doing a client should be a nice first step, and helps build a common foundation forward.
I personally have done a high performance HTTP container in C++ using modern C++ programming techniques (policies, metaprogramming, etc.) on top of Boost.Asio -- and it's proving to be very nice indeed. Can't share more than that though. ;-) -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
I don't think it passes through only HTTP but also SIP. BTW, does anyone think SCTP should be implemented into asio? On Fri, Jul 25, 2008 at 2:31 PM, Dean Michael Berris <mikhailberis@gmail.com
wrote:
Hi Gregory,
Anyone tried building a web service client based on existing boost
On Fri, Jul 25, 2008 at 1:31 PM, Gregory Dai <gregory.dai@gmail.com> wrote: library
components such as asio? Like to share, or brag :-) about your experience?
When you say web service client, is this just generally anything that passes through HTTP?
In this case there's a currently on-going effort (not making too much progress though with other "real world" things going on) to create a networking library on top of Boost.Asio as the base and provide simple implementations of protocol clients. Initial protocol scheduled to be implemented is an extensible HTTP client. You can get more information on the poject here: http://sourceforge.net/projects/cpp-netlib.
A web service server is probably more involved. Doing a client should be a nice first step, and helps build a common foundation forward.
I personally have done a high performance HTTP container in C++ using modern C++ programming techniques (policies, metaprogramming, etc.) on top of Boost.Asio -- and it's proving to be very nice indeed. Can't share more than that though. ;-)
-- Dean Michael C. Berris Software Engineer, Friendster, Inc. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- ------------------------------- Enjoy life! Barco You
On Thu, Jul 24, 2008 at 11:31 PM, Dean Michael Berris < mikhailberis@gmail.com> wrote:
Hi Gregory,
Anyone tried building a web service client based on existing boost
On Fri, Jul 25, 2008 at 1:31 PM, Gregory Dai <gregory.dai@gmail.com> wrote: library
components such as asio? Like to share, or brag :-) about your experience?
When you say web service client, is this just generally anything that passes through HTTP?
In this case there's a currently on-going effort (not making too much progress though with other "real world" things going on) to create a networking library on top of Boost.Asio as the base and provide simple implementations of protocol clients. Initial protocol scheduled to be implemented is an extensible HTTP client. You can get more information on the poject here: http://sourceforge.net/projects/cpp-netlib.
HTTP is a foundational protocol for web services, which include following components: * SOAP: XML based, and over HTTP (or other similar protocols). * WSDL: XML based, which is where services are defined. * UDDI: Protocol for discovering, describing, and integrating web services out there in the Internet cloud.
A web service server is probably more involved. Doing a client should be a nice first step, and helps build a common foundation forward.
I personally have done a high performance HTTP container in C++ using modern C++ programming techniques (policies, metaprogramming, etc.) on top of Boost.Asio -- and it's proving to be very nice indeed. Can't share more than that though. ;-)
I'm sure your HTTP server is super cool, but web services are not just HTTP, but include other components as well.
On Fri, Jul 25, 2008 at 3:35 PM, Gregory Dai <gregory.dai@gmail.com> wrote:
On Thu, Jul 24, 2008 at 11:31 PM, Dean Michael Berris < mikhailberis@gmail.com> wrote:
Hi Gregory,
Anyone tried building a web service client based on existing boost
On Fri, Jul 25, 2008 at 1:31 PM, Gregory Dai <gregory.dai@gmail.com> wrote: library
components such as asio? Like to share, or brag :-) about your experience?
When you say web service client, is this just generally anything that passes through HTTP?
In this case there's a currently on-going effort (not making too much progress though with other "real world" things going on) to create a networking library on top of Boost.Asio as the base and provide simple implementations of protocol clients. Initial protocol scheduled to be implemented is an extensible HTTP client. You can get more information on the poject here: http://sourceforge.net/projects/cpp-netlib.
HTTP is a foundational protocol for web services, which include following components: * SOAP: XML based, and over HTTP (or other similar protocols). * WSDL: XML based, which is where services are defined. * UDDI: Protocol for discovering, describing, and integrating web services out there in the Internet cloud.
I don't want to start a web services debate here, but how about providing something that supports REST first? It maps perfectly with the HTTP protocol easily and requires less hacking in terms of parsing (a json parser is easier to implement than an XML parser) and removes the need to do state maintenance. That said, I think we'll need the underlying components first as generic libs before we actually go into a focused approach to supporting web services. Thinking bottom up allows us to solve a larger set of problems with specifically generic (?) libraries to work with.
A web service server is probably more involved. Doing a client should be a nice first step, and helps build a common foundation forward.
I personally have done a high performance HTTP container in C++ using modern C++ programming techniques (policies, metaprogramming, etc.) on top of Boost.Asio -- and it's proving to be very nice indeed. Can't share more than that though. ;-)
I'm sure your HTTP server is super cool, but web services are not just HTTP, but include other components as well.
Right. So do you have an approach to develop something in mind? -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
On Fri, Jul 25, 2008 at 12:31 AM, Gregory Dai <gregory.dai@gmail.com> wrote:
Anyone tried building a web service client based on existing boost library components such as asio? Like to share, or brag :-) about your experience?
The answer here for me is no, but... I did look at the architecture coupled with the new boost::iostreams library so here is what I basically asked at BostCon 2008 during the ASIO presentation. Would it make sense to use ASIO and boost iostreams (filters) to make a communications foundations layer which would provide the following: Client->boost::iostream::HTTP->boost::iostream::SSL->ASIO->~network~->ASIO->boost::iostream:SSL::boost::iostream::HTTP->Server iostreams library has the concept of filters which could provide transformations on data. I also looked into creating stream operators like std::endl (an example) to preform actions on the stream. These are just my thoughts for what they are worth. Brian
A web service server is probably more involved. Doing a client should be a nice first step, and helps build a common foundation forward.
Thanks, Greg _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 7/25/08, Brian Davis <bitminer@gmail.com> wrote:
On Fri, Jul 25, 2008 at 12:31 AM, Gregory Dai <gregory.dai@gmail.com> wrote:
Anyone tried building a web service client based on existing boost library components such as asio? Like to share, or brag :-) about your experience?
The answer here for me is no, but... I did look at the architecture coupled with the new boost::iostreams library so here is what I basically asked at BostCon 2008 during the ASIO presentation.
Would it make sense to use ASIO and boost iostreams (filters) to make a communications foundations layer which would provide the following:
Client->boost::iostream::HTTP->boost::iostream::SSL->ASIO->~network~->ASIO->boost::iostream:SSL::boost::iostream::HTTP->Server
iostreams library has the concept of filters which could provide transformations on data. I also looked into creating stream operators like std::endl (an example) to preform actions on the stream. These are just my thoughts for what they are worth.
Brian
I think it would be a great idea. But would you write this communications foundations layer ? -- Alp Mestan --- http://blog.mestan.fr/ --- http://alp.developpez.com/ --- In charge of the Qt, Algorithms and Artificial Intelligence sections on Developpez
I think it would be a great idea. But would you write this communications foundations layer ?
Take this one line: Client->boost::iostream::HTTP->boost::iostream::SSL->ASIO->~network~->ASIO->boost::iostream:SSL->boost::iostream::HTTP->Server Multiply by 5000 and that is about how many lines of code if not more this would take This is also entering the realm of frameworks and arcitectures and inevitably "cat herding" and bicycle shed discussions. Try opening a socket to a HTTP server in JAVA (yea I know I don't want to go there either) and then try the same in C++... standard lib... standard architecture? It becomes in C++ may hours of searching source forge and trying out libraries.. reading various documentation and learning many different software architectures then picking the right one then of course there are the bugs... hopefully you chose one opensource or you could be in for a world of hurt.
From Boost.IOStreams lib Filters section "The third Filter helper, symmetric_filter<http://www.boost.org/doc/libs/1_35_0/libs/iostreams/doc/classes/symmetric_filter.html>, is useful for defining filter based on C-language API such as zlib, libbz2 or OpenSSL"
which (OpenSSL) is what spawned off the idea for web services using streams and filters. Can't say I would be the best person for the job, but sure I am interested. I was looking at iostreams and asio for use as a dataflow model which Stjepan has already created. My interest in asio was/is two fold 1) extending ASIO for RS232 and other asynchronous IO 2) using iostreams (possibly) for database to preform ODBC connections to remote databases. I am not talking about writing an entire database (with the exception of as simple text/file based database provided as a default), but rather providing a front end / back end design one that allows any database to be connected to on the backend and a standard C++ front end one that takes advantage of all the boost goodies. Initially I looked iostreams for writing a simple csv filter for file operations to create a table of data then I looked at it for full blown data base interface. This is were iostreams start to break down when they are used for more than simple streaming of characters and other concepts are needed. There is a common theme between WebServices, Database, and XML The data in the stream is non uniform (changing) or not of a specific format. This data must be coerced into type safe containers that can be iterated through. Transactions can be Asynchronous even for XML where DTD and linked xml files may be on a remote server and a request may not complete instantaneously and blocking is not the answer.. There is a common themes between Database and WebServices - Transactions - Manage non uniform data or changing data formats - Asynchronous event notification - The need to convert incoming data to type safe containers (fusion may be the key here) - The need to be be able to iterate and search data. - The need to configure how the data flows through the program (i.e. is the stream encrypted using OpenSSL as my one liner at the top shows). Consider the PSEUDO code (I stress by no means have I started this.... it is shear contemplation. Database is also another topic I brought up at BoostCon 2008) : // Where database is a boost:iostream::BidirectionalDevice boost::database myDB; // Where query is a boost::iostream::filter::BidirectionalFilter boost::database myQuery( "SELECT [Employee ID], [Employee Name] FROM [Employees]" ); std::string employee; boost::uint32_t employee_id; // Since Database is a stream data base can be modified using SQL commands, but only thoes which affect a database structure such as CREATE TABLE, DROP TABLE etc" myDB << "ODBC;127.0.0.1:1234,user=myuser, password=mypw" << std::endl; // or myDB.open( "ODBC;127.0.0.1:1234,user=myuser, pw=mypw" ); // or for default boost text or binary based boost::database implementation boost::filesystem::path = "/opt/var/mydb"; // Open and create if not in existence. myDB.open( path, ios::binary | std::ios::trunc ); // hey maybe we could also change the file storage format filter. myDB.push( csvFilter ); // Add the query filter to the database. mydb.push( myQuery ); while( !myDB ) { myDB >> employee_id >> employee; } I am not certian how using streams for databases would fully look. Maybe a stream for each query or table object. Maybe the database object would not be a stream object. In my example down stream filters could change the underlying device(source/sink) which would change the data retrieved at the source (database) and would not actually "filter" the full data coming in the stream. I think this deviates from the IOStream design where each down stream filter receives all data and filters (transforms) it accordingly. Though sending all the data to each filter is possible and still an option here, though it is not as efficient as it could be. So the full table would not be streamed to the query, but rather the query filter would change the database device to select just the two columns of data in the database table Employees and stream the result. There's nothing saying that the filter stream must modify the data as it passes through... it could modify the base stream object itself either way the result should be the same. Of course this probably entering the realm of "shoe horning" iostreams to meet database needs. Consider mydb.push( myQuery) call. The push can be overloaded to get from the query filter string. The cautious would remember the following from iotreams lib doc: "Non-blocking Devices do not interact well with standard streams and stream buffers, however, so most devices should be Blocking<http://www.boost.org/doc/libs/1_35_0/libs/iostreams/doc/concepts/blocking.html>. *See* Asynchronous and Non-Blocking I/O<http://www.boost.org/doc/libs/1_35_0/libs/iostreams/doc/guide/asynchronous.html> ." And we all know databases especially remote ones do not return transactions immediately. Which may require stream callbacks notifying when a Asynchronous I/O data transaction request has fully completed and data is ready. Most of boost, to me personally (and I likes me the boost :-) ), seems like a collection of basic low level APIs which the programmer can use to construct what ever software they need. Lets call these Tier1. Tier2 could be the extension and use of these Tier1 libraries to create architectures to really "boost" application developers to assist in creating web services, database applications, GUIs (cppgui) etc. I also asked at BoostCon for the 10 step process for submitting a library and if you go to: http://www.boost.org/development/submissions.html <--- Boost should really make this a big green button next to the "get boost" button so say "develop boost" or in the time honored tradition "set boost" I read: "A bit of further description or snippet of code may be helpful. Messages should be plain text; not rich text, HTML, etc" so I submit these and other concepts. If there is enough interest then sure I can help or possibly lead the way. I am still just getting started. My current focus is on dataflow and databinding. There is currently a submission on databases in the vault which was posted by Nicola Musatti which provides much better type checking that streams would provide, but provides no support for streams. http://www.boostpro.com/vault/index.php?PHPSESSID=dc8a8cca1f95373201ace3827a3827b9&direction=0&order=&directory=Database Nothing on the radar for web services that I can see in vault, trunk, or sandbox. So maybe it's time to look into Boost.WebServices. I would like to know what the road map is for Architectures in boost (database, gui, webservices, XML, etc). Looks like Dean Berris has a great start: https://cpp-netlib.svn.sourceforge.net/svnroot/cpp-netlib
From client.cpp in cpp-net-lib
// use Asio's iostream interface to perform // a synchronous request for data via HTTP // and then craft a response object to be returned // based on the headers received, and the contents using namespace boost::asio;
ip::tcp::iostream socket_stream(request_.host(), boost::lexical_cast<typename tag::str_type>(request_.port()), ip::tcp::resolver_query::numeric_service); socket_stream << "GET /" << request_.path() ; << shows that net-lib is directly bound to ASIO making SSL encryption difficult as depicted by: Client->boost::iostream::HTTP->boost::iostream::SSL->ASIO->~network I am not saying that net-lib is bad, matter of fact I found it quite slick, just that it does not support the kind of stream chaining I was talking about. Maybe this is not desired. Brian
Hi Brian, On Sun, Jul 27, 2008 at 10:27 AM, Brian Davis <bitminer@gmail.com> wrote:
I think it would be a great idea. But would you write this communications foundations layer ?
Take this one line:
Client->boost::iostream::HTTP->boost::iostream::SSL->ASIO->~network~->ASIO->boost::iostream:SSL->boost::iostream::HTTP->Server
Multiply by 5000
and that is about how many lines of code if not more this would take
This assessment may even be conservative. Considering that you're going to be making these generic pieces try to work together in a matter that doesn't break, then you've got yourself a very large scale project that although a lot of people want, not a lot of people have the time/resources/motivation to write.
This is also entering the realm of frameworks and arcitectures and inevitably "cat herding" and bicycle shed discussions. Try opening a socket to a HTTP server in JAVA (yea I know I don't want to go there either) and then try the same in C++... standard lib... standard architecture? It becomes in C++ may hours of searching source forge and trying out libraries.. reading various documentation and learning many different software architectures then picking the right one then of course there are the bugs... hopefully you chose one opensource or you could be in for a world of hurt.
I know how painful this is. I think the first step (which is under way IIRC) is to develop a common base for everything network-related in C++ -- this I think is what Asio gives, and is already being considered to be made part of the next version of the C++ standard. That being said, what we have now are the building blocks to be able to come up with more specific targetted tools. Having said this, I don't necessarily think this should be made part of Boost. Anybody should be able to come up with their own Web Services framework in C++ and license it under the Boost Software License without having to be part of the Boost C++ Library.
From Boost.IOStreams lib Filters section "The third Filter helper, symmetric_filter<http://www.boost.org/doc/libs/1_35_0/libs/iostreams/doc/classes/symmetric_filter.html>, is useful for defining filter based on C-language API such as zlib, libbz2 or OpenSSL"
which (OpenSSL) is what spawned off the idea for web services using streams and filters.
Can't say I would be the best person for the job, but sure I am interested. I was looking at iostreams and asio for use as a dataflow model which Stjepan has already created. My interest in asio was/is two fold 1) extending ASIO for RS232 and other asynchronous IO 2) using iostreams (possibly) for database to preform ODBC connections to remote databases. I am not talking about writing an entire database (with the exception of as simple text/file based database provided as a default), but rather providing a front end / back end design one that allows any database to be connected to on the backend and a standard C++ front end one that takes advantage of all the boost goodies. Initially I looked iostreams for writing a simple csv filter for file operations to create a table of data then I looked at it for full blown data base interface. This is were iostreams start to break down when they are used for more than simple streaming of characters and other concepts are needed. There is a common theme between WebServices, Database, and XML The data in the stream is non uniform (changing) or not of a specific format. This data must be coerced into type safe containers that can be iterated through. Transactions can be Asynchronous even for XML where DTD and linked xml files may be on a remote server and a request may not complete instantaneously and blocking is not the answer..
WRT to Databases, have you seen SOCI? XML Stream parsing is something that's fit for the Iostreams lib along with Asio underneath -- very important for XMPP and other streamed XML protocols -- but I have very little experience with XML Stream parsing and Iostreams to be able to give a better idea about it. But it does sound natural to map stream-based protocols to the Iostreams interface. [snip]
Most of boost, to me personally (and I likes me the boost :-) ), seems like a collection of basic low level APIs which the programmer can use to construct what ever software they need. Lets call these Tier1. Tier2 could be the extension and use of these Tier1 libraries to create architectures to really "boost" application developers to assist in creating web services, database applications, GUIs (cppgui) etc.
Right. I think also the goal of Boost is to be a place for developing software/libraries not just for general use, but also for eventual standardization. Already, libraries like Regex, Thread, Tuple, and (IIRC) Asio are going to be part of the next version of the standard. [snip]
Nothing on the radar for web services that I can see in vault, trunk, or sandbox. So maybe it's time to look into Boost.WebServices. I would like to know what the road map is for Architectures in boost (database, gui, webservices, XML, etc).
Looks like Dean Berris has a great start:
https://cpp-netlib.svn.sourceforge.net/svnroot/cpp-netlib
From client.cpp in cpp-net-lib
// use Asio's iostream interface to perform // a synchronous request for data via HTTP // and then craft a response object to be returned // based on the headers received, and the contents using namespace boost::asio;
ip::tcp::iostream socket_stream(request_.host(), boost::lexical_cast<typename tag::str_type>(request_.port()), ip::tcp::resolver_query::numeric_service); socket_stream << "GET /" << request_.path() ; <<
shows that net-lib is directly bound to ASIO making SSL encryption difficult as depicted by:
Client->boost::iostream::HTTP->boost::iostream::SSL->ASIO->~network
I am not saying that net-lib is bad, matter of fact I found it quite slick, just that it does not support the kind of stream chaining I was talking about. Maybe this is not desired.
Actually, the interface and implementation will change -- and is currently being developed (by me) offline, and at a snail-steady pace. The idea is that whether you use SSL or plain TCP, it will use Asio underneath. There will be two client implementations: a synchronous one, templated to whether you use SSL or use a proxy, etc.; and an asynchronous one, supporting pipelining and "automatic session management" with cookie and (eventual) caching support -- all while trying to be a header-only implementation. The message framework that's already there allows for building opaque messages, and a transformation layer is provided to support conversions from one message type to another either through library implemented routines or through user-generated functions. That allows you to: 1. Create a soap::message implementation that converts to an http::message or an xmpp::message (whichever you support) 2. Have that object instance passed to a client that talks the appropriate protocol 3. Get a generic protocol::message response 4. Convert to the appropriate (in this case soap::message) type for further processing. This allows me to write an HTTP(S) client without having to think about all the possible messages that come through and only support one kind and work well with it. The message framework is already done, and help with testing and further development would be greatly appreciated. :) -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
This assessment may even be conservative. Considering that you're going to be making these generic pieces try to work together in a matter that doesn't break,
In response to "doesn't break": What I would stress for instance is take databases as an example. There are ways to interface to databases with out knowing how the underlying database is implemented such as ODBC. I am not saying to implement an entire database with the exception of a simple file based database as the default - to allow nubies/learners/Stroustrup's students a quick and easy way to learn databases, GUIs, etc . What I am getting at here is to provide front end interfaces to back end service providers such as database solution providers. We define the interface and they hook up their database to the back end interface. I am sure we all do not want to spend our time writing databases, but what we can do is provide a standard interface in C++ to these databases. If "it breaks" then it is a result of: 1) Interface not providing the necessary abstraction 2) database provider not implementing the interface correctly. I know for performance applications this may be a gross oversimplification
then you've got yourself a very large scale project that although a lot of people want, not a lot of people have the time/resources/motivation to write.
I am willing to dedicate my time so that I myself and others don't have to waste theirs. Think of the problems we could solve as a society if we were not wasting our time learning obscure interfaces to databases and GUIs. My efforts may not end in a final solution, but merely tow the rope until the next stage of development. Time will tell. I know how painful this is. I think the first step (which is under way
IIRC) is to develop a common base for everything network-related in C++ -- this I think is what Asio gives, and is already being considered to be made part of the next version of the C++ standard. That being said, what we have now are the building blocks to be able to come up with more specific targetted tools.
I understand the evolution process of C++/boost... believe me.. and as you have noted we are now just getting the finite quantum of core libraries to attempt these endeavors.
Having said this, I don't necessarily think this should be made part of Boost. Anybody should be able to come up with their own Web Services framework in C++ and license it under the Boost Software License without having to be part of the Boost C++ Library.
I will never and do not want to put restrictions on the development efforts of others. My goal here is to provide standard interfaces to those software implementations. This would require software churn on the part of say a data base developer to "hook-up" to the boost standard interface. The user could choose which to use the standard interface or the direct API. The ultimate goal to to flush out standard interfaces/libraries then push them to C++ standard libraries or maybe by that time we will have the concept of packages.
WRT to Databases, have you seen SOCI?
Nope, but I remember talking to a gentleman at BoostCon on the way back from lunch. He mentioned it, but I skipped my mind to look it up before I got back. Thanks for reminding me about it!!! http://soci.sourceforge.net/ Yes this is the kind of interface I am talking about standardizing. http://soci.sourceforge.net/doc/structure.html What I wish to standardize is the "C++ syntax and core libraries" and "common backed Interface". Maybe SOCI developers would consider boostifying the interface. I have looked at the code to the database implementation checked into boost vault that Nicola Musatti wrote up. I need to contact him to see what his plans are for this latest version states 2007. It doesn't compile in 1_35_0 which is not surprising based on the version dates. I will try it with vintage versions of boost. Databases are an easier concept to provide this architecture. For GUIs is is much more difficult to abstract away the interfaces especially when the desire is to take advantage of existing libs such as QT, KDE, GNome, GTK, WxWidgits, FLTK, etc. Especially when you consider that the libs support much more than just UI libraries (such as XML) and all do not (some do) have C++ interfaces. XML Stream parsing is something that's fit for the Iostreams lib along
with Asio underneath -- very important for XMPP and other streamed XML protocols -- but I have very little experience with XML Stream parsing and Iostreams to be able to give a better idea about it. But it does sound natural to map stream-based protocols to the Iostreams interface.
Here I am not sure that streams would have to be used for XML, unless boost developers want to tackle a XML parser SAX2 and DOM, Compliant to W3C specs, as existing libs already exist such as xerces, libxml etc, but rather a common interface to bridge to C++. These libs take care of the parsing, we would take care of the standard interface to C++. I am sure readers of this would ask the question, as I am myself : if your going to create a standard interface to libraires for something like xml why not just write a full parser and not worry about standardizing a frontend/backend interface. My opinion here is for something like XML this may be possible, for databases and GUIs I do not believe it to be (though abstracting away the interfaces is no small task - especially if interoperability between different UI implemntations with a common front end interface is desired.).
Right. I think also the goal of Boost is to be a place for developing software/libraries not just for general use, but also for eventual standardization. Already, libraries like Regex, Thread, Tuple, and (IIRC) Asio are going to be part of the next version of the standard.
Yes these are good times to be a C++ programmer!!!
Actually, the interface and implementation will change -- and is currently being developed (by me) offline, and at a snail-steady pace. The idea is that whether you use SSL or plain TCP, it will use Asio underneath. There will be two client implementations: a synchronous one, templated to whether you use SSL or use a proxy, etc.; and an asynchronous one, supporting pipelining and "automatic session management"
Curious on your automatic session management. What ideas/features to you have here?
with cookie and (eventual) caching support -- all while trying to be a header-only implementation.
Sweet.
The message framework that's already there allows for building opaque messages, and a transformation layer is provided to support conversions from one message type to another either through library implemented routines or through user-generated functions. That allows you to:
1. Create a soap::message implementation that converts to an http::message or an xmpp::message (whichever you support)
An then SOAP to RPC...??? Which brings up another missing lib (RPC) in boost.and then there was this post: http://lists.boost.org/Archives/boost/2005/03/81179.php namespace IMO { Which is frankly not true. I could use it right now. And I am not talking a CORBA interface. I have no desire in my current work to have C++ talking to COBOL or JavaScript just C2C. At least Java has RMI. And this: http://lists.boost.org/Archives/boost/2005/03/81169.php 101 votes with a 4.66 rating out of 5 for the RMI for C++ article ... so someone's interested in this. Ah yes macros... oh the horror. I have seen implementations where there is a 2+ page macro wrapping a class instance in an attempt at RPC using boost with serialization and asio. Variadic templates would greatly help simplify an RPC library available in GCC 4.3 and 4.4, MSVC 2008? Hmmm I should try this with GCC. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2555.pdf } // namespace IMO
2. Have that object instance passed to a client that talks the appropriate protocol 3. Get a generic protocol::message response 4. Convert to the appropriate (in this case soap::message) type for further processing.
This allows me to write an HTTP(S) client without having to think about all the possible messages that come through and only support one kind and work well with it.
The message framework is already done, and help with testing and further development would be greatly appreciated. :)
I am currently spending my time on Dataflow and I need to boostify my Data Binding example, but let me know what I could do. I already have the code checked out. Currently I can build for MSVC 8, MSys, Cygwin, Ubuntu Linux, and Gentoo. I did try to get cpp-netlib to compile in mingw and msvc 8. It did not. Are you using trunk of boost or 1.35.0? I am attempting building the trunk of netlib. -- Brian
Hi Brian, On Sat, Aug 9, 2008 at 2:46 PM, Brian Davis <bitminer@gmail.com> wrote:
This assessment may even be conservative. Considering that you're going to be making these generic pieces try to work together in a matter that doesn't break,
In response to "doesn't break": What I would stress for instance is take databases as an example. There are ways to interface to databases with out knowing how the underlying database is implemented such as ODBC. I am not saying to implement an entire database with the exception of a simple file based database as the default - to allow nubies/learners/Stroustrup's students a quick and easy way to learn databases, GUIs, etc . What I am getting at here is to provide front end interfaces to back end service providers such as database solution providers. We define the interface and they hook up their database to the back end interface. I am sure we all do not want to spend our time writing databases, but what we can do is provide a standard interface in C++ to these databases. If "it breaks" then it is a result of:
1) Interface not providing the necessary abstraction 2) database provider not implementing the interface correctly.
I know for performance applications this may be a gross oversimplification
True. This I think is how Python does it with the standard library interfaces to DB's are concerned. If you want your DB client to be easily usable in the Python arena, you'd have to abide by the DB Api. Although this has worked with other dynamic languages (that rely heavily on runtime dispatching, and lot and lots of runtime abstraction), making it work with C++ is a little finicky at best: having no standard ABI for example is a problem. The motivation then towards static polymorphism and compile-time dynamic programming (template metaprogramming, leveraging the type system to do dispatch) becomes more evident and at time more robust than relying on RTTI and the run-time in general. That being said, for *any* interface development to work correctly, it has to be distilled from a pool of existing common implementations, rather than developed in isolation of existing solutions. Asking someone (a vendor for example) to conform to a pre-defined API sounds a little wrong to me. [snip]
WRT to Databases, have you seen SOCI?
Nope, but I remember talking to a gentleman at BoostCon on the way back from lunch. He mentioned it, but I skipped my mind to look it up before I got back. Thanks for reminding me about it!!!
Yes this is the kind of interface I am talking about standardizing.
http://soci.sourceforge.net/doc/structure.html
What I wish to standardize is the "C++ syntax and core libraries" and "common backed Interface". Maybe SOCI developers would consider boostifying the interface. I have looked at the code to the database implementation checked into boost vault that Nicola Musatti wrote up. I need to contact him to see what his plans are for this latest version states 2007. It doesn't compile in 1_35_0 which is not surprising based on the version dates. I will try it with vintage versions of boost.
I've made it work successfully with Boost 1.35. If you can bring up your concerns to the SOCI-users list, then that would be a good start. As far as standardizing, I think the people involved with SOCI have voiced their thought about not making it part of Boost, but rather making it work well with Boost instead. I forget if it's on this mailing list or in the SOCI-users mailing list.
Databases are an easier concept to provide this architecture. For GUIs is is much more difficult to abstract away the interfaces especially when the desire is to take advantage of existing libs such as QT, KDE, GNome, GTK, WxWidgits, FLTK, etc. Especially when you consider that the libs support much more than just UI libraries (such as XML) and all do not (some do) have C++ interfaces.
I don't have much experience with GUI's, but I think standardizing interfaces has to be a process of evolution rather than an end on itself. I believe the Python DB API came about from distilling the myriad of implementations to different DB backends and coming up with a framework to abstract the details of underlying implementations. This is what SOCI has done (very well I might add) that is worth emulating for developing web service clients (XML-RPC, SOAP/HTTP, REST, SOAP/XMPP, CORBA, etc.). That's again, easier said than done, but it has to be an abstraction layer that makes sense and is expressive enough to actually use and implement. You might want to look at how Boost.MPI (and the MPI standard in general) has been developed to standardize the message passing interface among the myriad implementations available out there. Ultimately the vendors agreed to a set of standard API's that they all will support. Key to this is the fact that they already had their own implementations and they simply got together and agreed on a common sub/super-set of interfaces they will implement.
XML Stream parsing is something that's fit for the Iostreams lib along
with Asio underneath -- very important for XMPP and other streamed XML protocols -- but I have very little experience with XML Stream parsing and Iostreams to be able to give a better idea about it. But it does sound natural to map stream-based protocols to the Iostreams interface.
Here I am not sure that streams would have to be used for XML, unless boost developers want to tackle a XML parser SAX2 and DOM, Compliant to W3C specs, as existing libs already exist such as xerces, libxml etc, but rather a common interface to bridge to C++. These libs take care of the parsing, we would take care of the standard interface to C++. I am sure readers of this would ask the question, as I am myself : if your going to create a standard interface to libraires for something like xml why not just write a full parser and not worry about standardizing a frontend/backend interface. My opinion here is for something like XML this may be possible, for databases and GUIs I do not believe it to be (though abstracting away the interfaces is no small task - especially if interoperability between different UI implemntations with a common front end interface is desired.).
Right, but ultimately you need the underlying pieces to be able to make the pieces work together. And in making these pieces (XML parsers, dom representations, transformation engines, etc.) work on their own in an orthogonal fashion makes it a lot harder effort. [snip]
Actually, the interface and implementation will change -- and is currently being developed (by me) offline, and at a snail-steady pace. The idea is that whether you use SSL or plain TCP, it will use Asio underneath. There will be two client implementations: a synchronous one, templated to whether you use SSL or use a proxy, etc.; and an asynchronous one, supporting pipelining and "automatic session management"
Curious on your automatic session management. What ideas/features to you have here?
With HTTP 1.1, there's support for persistent connections on the server side. If the server side supports it as well as pipelining, then the client will be implemented to take these into consideration. The connection management becomes the domain of the client, and therefore clever implementations like request pipelining and even persistent connections would be leveraged for you.
with cookie and (eventual) caching support -- all while trying to be a header-only implementation.
Sweet.
Indeed. We're adapting parts of the HTTP implementation of the libpion-net library specifically the cookie management and encoding/decoding solutions to be made part of the cpp-netlib project. [snip]
The message framework is already done, and help with testing and further development would be greatly appreciated. :)
I am currently spending my time on Dataflow and I need to boostify my Data Binding example, but let me know what I could do. I already have the code checked out. Currently I can build for MSVC 8, MSys, Cygwin, Ubuntu Linux, and Gentoo. I did try to get cpp-netlib to compile in mingw and msvc 8. It did not. Are you using trunk of boost or 1.35.0? I am attempting building the trunk of netlib.
I've been working on the http_integration branch and only just recently (as in yesterday) I've revamped the implementation of the HTTP client to make use of lower level Asio primitives instead of the higher level stream implementation. We've been building/testing against Boost's trunk and Boost 1.35.0. Nobody has had success building cpp-netlib's tests for mingw. For msvc 8, we've only recently been able to test with Boost.Jam+Boost.Build -- there are no MSVC-specific build packages/solutions/projects yet, since bjam works fine for us. If you would like to help out in that effort, please join us in the mailing lists. See http://sourceforge.net/projects/cpp-netlib for details. :) HTH -- Dean Michael C. Berris Software Engineer, Friendster, Inc.
participants (5)
-
Alp Mestan -
Barco You -
Brian Davis -
Dean Michael Berris -
Gregory Dai