Request for Interest in several Modules

Hello, I have several modules/libraries I developed and released under Boost License. - CppDB Sql Connectivity - Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API. ---------------------------------- Let's start Boost.CppDB CppDB library: Sql Connectivity library currently not Boostified, released and actively maintained. http://cppcms.sourceforge.net/sql/cppdb/index.html How much interest in this library as-is with small obvious modifications like using Boost Namespace Boost.SharedObject: Simple Shared Object class for loading DLLs/SO - i.e. cross platfrom dlopen/dlclose + naming (libfoo.3.dylib/libfoo.so.3/foo-3.dll) http://cppcms.sourceforge.net/cppcms_ref_v0_99/classbooster_1_1shared__objec... Any interest in this simple class? Boost.StackTrace Collecting stack trace automatically from exception and printing it. Very Very useful for debugging. http://cppcms.sourceforge.net/cppcms_ref_v0_99/namespacebooster_1_1stack__tr... http://cppcms.sourceforge.net/cppcms_ref_v0_99/backtrace_8h_source.html Any interest in this library. It works (collects a trace and prints it) on: Windows, Solaris, Linux, Mac OS X Boost.NoWide Standard C and C++ library UTF-8 friendly API for windows classes/functions: boost { nowide { [of]stream - iostreams with UTF-8 file names f(re)open - FILE * functions with UTF-8 file names remove, rename- UTF-8 parameters getenv,putenv - UTF-8 environment variables args - UTF-8 argc, argv } } Under non windows just (using std::...) These functions (like Boost.Locale) all are parts of CppCMS C++ Web framework. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/

On Tue, Jan 10, 2012 at 3:11 PM, Artyom Beilis <artyomtnk@yahoo.com> wrote:
Hello,
I have several modules/libraries I developed and released under Boost License.
- CppDB Sql Connectivity - Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API.
Quite interested in all four, especially in the SQL connectivity library. -- gpd

On Jan 10, 2012, at 7:11 AM, Artyom Beilis wrote:
Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
Very Very useful for debugging.
http://cppcms.sourceforge.net/cppcms_ref_v0_99/namespacebooster_1_1stack__tr... http://cppcms.sourceforge.net/cppcms_ref_v0_99/backtrace_8h_source.html
Any interest in this library. It works (collects a trace and prints it) on: Windows, Solaris, Linux, Mac OS X
This would be a great (feature) addition for Boost.Exception; the ability to attach a stack trace to an exception showing where it was thrown. As you say, very, very useful for debugging -- Marshall Marshall Clow Idio Software <mailto:mclow.lists@gmail.com> A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki

On 01/10/2012 04:29 PM, Marshall Clow wrote:
On Jan 10, 2012, at 7:11 AM, Artyom Beilis wrote:
Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
Very Very useful for debugging.
http://cppcms.sourceforge.net/cppcms_ref_v0_99/namespacebooster_1_1stack__tr... http://cppcms.sourceforge.net/cppcms_ref_v0_99/backtrace_8h_source.html
Any interest in this library. It works (collects a trace and prints it) on: Windows, Solaris, Linux, Mac OS X
This would be a great (feature) addition for Boost.Exception; the ability to attach a stack trace to an exception showing where it was thrown.
As you say, very, very useful for debugging
Why not just use a debugger? They can give backtraces as well. While we're speaking of it, Boost.Exception adds a massive amount of bloat. I'm considering scrapping it for that reason.

On 10 January 2012 11:16, Mathias Gaunard <mathias.gaunard@ens-lyon.org>wrote:
On 01/10/2012 04:29 PM, Marshall Clow wrote:
On Jan 10, 2012, at 7:11 AM, Artyom Beilis wrote:
Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
Very Very useful for debugging.
http://cppcms.sourceforge.net/**cppcms_ref_v0_99/** namespacebooster_1_1stack__**trace.html<http://cppcms.sourceforge.net/cppcms_ref_v0_99/namespacebooster_1_1stack__trace.html> http://cppcms.sourceforge.net/**cppcms_ref_v0_99/backtrace_8h_** source.html<http://cppcms.sourceforge.net/cppcms_ref_v0_99/backtrace_8h_source.html>
Any interest in this library. It works (collects a trace and prints it) on: Windows, Solaris, Linux, Mac OS X
This would be a great (feature) addition for Boost.Exception; the ability to attach a stack trace to an exception showing where it was thrown.
Extremely useful. We make extensive use of in house 'developed' (i.e. assembled from myriads small snippets on the web) stacktrace classes.
As you say, very, very useful for debugging
Why not just use a debugger?
They can give backtraces as well.
It's for when you don't have a debugger attached, or don't want to stop your program from continued execution: * To log callstacks from a running program doesn't infer (very much) with its execution, such as would halting it. For large systems more powerful than debugging. * Production (server) environments can't be halted and debugged due to an exception, but retrieving the stacktrace will aid in later error analysis. - Christian

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10-01-2012 16:11, Artyom Beilis wrote:
Hello,
I have several modules/libraries I developed and released under Boost License.
- CppDB Sql Connectivity
Interested. Actually, I already use this...
- Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API.
Interested. /Brian -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPDFx/AAoJEKd8gmwzkHPJTbcH/Rn5IskrHhEVaE5+5T6DxzbI /jEBtLmfWYJyz/vbfZvCw0k3vRFpXJtaZOY5/0z66tBX+V/V1bgOge9q1Xlgybk7 Ism07X1pq/jyZLMfQuwli0TsNEZMVxyKpde/uvoovlHwi+hW5o32HSjNbtn4TrZH zvskCc611P8BuesL1OD3eqUPJ3y1qqzGbeV6ZxjUm5T9VWSEe1ln5Y7bAiT02bVn nDgdbScPIlZXVJ2LnajPaoKa8vzVj8EwBYdRetXt7LqHMbDKfZ+nlPbdNhblZ/I1 AGqla8iH///1dR+Tzju078Hh7Gg7NAlziWoDc58giOratKUddcuWR886XTdQZ+k= =+dJd -----END PGP SIGNATURE-----

Hi,
I have several modules/libraries I developed and released under Boost License.
- CppDB Sql Connectivity - Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API.
Yes, all of those would be interesting. I am especially interested in the DB connectivity and the shared objects. The UTF-8 friendly standard library is also very interesting and the stack trace could be useful for debugging in some situations. For the DB connectivity I wondered, why your result objects don't provide the input iterator interface, but next() and fetch() instead. I guess, there is a good reason, but I'd like to know it. I already had a look into your cppcms a while ago, which I mostly liked. Sadly I did not really get around to try and write an application with it. The biggest drawback for me is the use of "booster" instead of boost. I know that there might me situations where a stable ABI is important, but for me it is not. So I'd prefer to simply use boost there. Do you think, there might be a compile time option to chose either? Christof -- okunah gmbh Software nach Maß Werner-Haas-Str. 8 www.okunah.de 86153 Augsburg cd@okunah.de Registergericht Augsburg Geschäftsführer Augsburg HRB 21896 Christof Donat UStID: DE 248 815 055

For the DB connectivity I wondered, why your result objects don't provide the input iterator interface, but next() and fetch() instead. I guess, there is a good reason, but I'd like to know it.
There are several reasons. When *result_iterator would return? Who owns it? If I change it what it does. Basically I had seem several SQL libraries with iterator interface and it felt unnatural and abused, on the other hand in some other libraries (like for example JDBC, I know... I know Java/Ugly) it was much more intuitive. Finally cppdb::result is much more complex then JUST iterator. In this case next more clear operation with more clear semantics without any "temporary object" and various copy semantics etc. Now fetch has even bigger differences, it is not iterator and it has multiple parameters (under the hood it is fetch(int col,Type &)) So between the choice of creating unnatural API with side effects or clear but less "modern-C++-style-API" I had chosen the second.
I already had a look into your cppcms a while ago, which I mostly liked. Sadly I did not really get around to try and write an application with it. The biggest drawback for me is the use of "booster" instead of boost. I know that there might me situations where a stable ABI is important, but for me it is not. So I'd prefer to simply use boost there. Do you think, there might be a compile time option to chose either?
First of all Booster is very far from Boost. It has **some** boost like classes and many others with different semantics. So it is not "copy-paste" replaceable especially parts like Booster.AIO that shared general ideas with ASIO but solves some very critical problems that exist in ASIO from my point of view. Finally you don't use almost any of them most of the with some very special exceptions... And nothing prevents from you to use Boost whatever version and type you like.
Christof
-- okunah gmbh Software nach Maß
Werner-Haas-Str. 8 www.okunah.de 86153 Augsburg cd@okunah.de
Registergericht Augsburg Geschäftsführer Augsburg HRB 21896 Christof Donat UStID: DE 248 815 055
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hi,
For the DB connectivity I wondered, why your result objects don't provide the input iterator interface, but next() and fetch() instead. I guess, there is a good reason, but I'd like to know it.
There are several reasons.
[...] Basically I had seem several SQL libraries with iterator interface and it felt unnatural and abused,
OK, I have not tried, it would just have been my first idea for an interface there. The good thing about using iterators is, that you could also use STL and boost algorithms and with C++11 range based for loops on it, but maybe that is less of an advantage when you have SQL at hand.
So between the choice of creating unnatural API with side effects or clear but less "modern-C++-style-API" I had chosen the second.
OK. I did not want to question your choice, but to understand your reasons. Thanks for your explanation.
I already had a look into your cppcms a while ago, which I mostly liked. Sadly I did not really get around to try and write an application with it. The biggest drawback for me is the use of "booster" instead of boost. I know that there might me situations where a stable ABI is important, but for me it is not. So I'd prefer to simply use boost there. Do you think, there might be a compile time option to chose either?
First of all Booster is very far from Boost. It has **some** boost like classes and many others with different semantics. So it is not
OK, I understood the documentation, that booster is very similar to boost, but with the ABI as stable as possible. Seems like I misunderstood it there.
"copy-paste" replaceable especially parts like Booster.AIO that shared general ideas with ASIO but solves some very critical problems that exist in ASIO from my point of view.
Of course I'd like to learn about these problems. Eventually there ist also a way to improve boost accordingly.
Finally you don't use almost any of them most of the with some very special exceptions... And nothing prevents from you to use Boost whatever version and type you like.
Sure. I just think, that using two different libraries that serve almost the same purpose in one application is a little bit suboptimal. So maybe my judgement was based on the - as I have learned now - wrong assumption that booster is mostly a copy of some boost libraries. Christof -- okunah gmbh Software nach Maß Werner-Haas-Str. 8 www.okunah.de 86153 Augsburg cd@okunah.de Registergericht Augsburg Geschäftsführer Augsburg HRB 21896 Christof Donat UStID: DE 248 815 055

OK, I understood the documentation, that booster is very similar to boost, but with the ABI as stable as possible. Seems like I misunderstood it there.
Booster is a boost-like API for functionality required in public interface: - callbacks: booster::function and booster::callback (same as function but reference counted) - regular expressions - event loop - mutexes - some smart pointers (shared/intrusive - copied from boost) And some more. It is not boost and it does not replace it. The biggest shared part is obviously Boost.Locale :-)
"copy-paste" replaceable especially parts like Booster.AIO that shared general ideas with ASIO but solves some very critical problems that exist in ASIO from my point of view.
Of course I'd like to learn about these problems. Eventually there ist also a way to improve boost accordingly.
Several problems. ASIO uses templates were I don't think they should be, there is no reason for stream_socket for TCP and Unix domain socket to be different class... For example if I implement FastCGI over Unix socket and TCP socket, under ASIO it must be 2 different classes as under ASIO it is two different classes. It is matter of design. But when CppCMS has 5-6 such classes it becomes painful. Another thing. ASIO uses IOCP, IOCP does not provide: - Reactor like functionality - Does not allow detach socket from IOCP Thus some functionality is just missing under ASIO because it is impossible to both use IOCP for best beformance and have reactor like functionality. This is the design. I have different needs. Just don't get wrong ASIO is absolutely great library and I use it on daily basis, but it does not fit CppCMS's project needs.
Finally you don't use almost any of them most of the with some very special exceptions... And nothing prevents from you to use Boost whatever version and type you like.
Sure. I just think, that using two different libraries that serve almost the same purpose in one application is a little bit suboptimal. So maybe my judgement was based on the - as I have learned now - wrong assumption that booster is mostly a copy of some boost libraries.
Christof
Some stuff is copied, (mostly smart pointers) some stuff provide simplified interface and some stuff is very different. Artyom

Hello, On Tue, Jan 10, 2012 at 1:12 PM, Christof Donat <cd@okunah.de> wrote:
I have several modules/libraries I developed and released under Boost
License.
- CppDB Sql Connectivity - Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API.
I am interested in the four libraries.
I already had a look into your cppcms a while ago, which I mostly liked. Sadly I did not really get around to try and write an application with it. The biggest drawback for me is the use of "booster" instead of boost. I know that there might me situations where a stable ABI is important, but for me it is not. So I'd prefer to simply use boost there. Do you think, there might be a compile time option to chose either?
I share the thought of Christof about Booster versus Boost. Regards, Fernando.

Hi, I'm also interested in those libraries, in particular in cppdb but more by shared_object. Looking at it's interface documentation I'd like to ask some minor questions: 1. The boost::extension author reported (when i asked him) a lack in the C++11 standard (without precision) that makes it hard to implement correctly this library on all platforms/compilers (unfortunately he didn't say which ones). Do you have any information about that? 2. booster::shared_object::shared_object(std::string const & file_name ) Here file_name is a path, right? I'm guessing it's relative to the way the OS will look for modules? 3. Is there a specific reason that there isn't a provided abstraction (template "helper" function) that will call resolve_symbol() and try (dynamic or not) to cast it for you to the provided type? My understanding is that you wanted to let the user do as he wish but I was asking myself if common usage could be identified and provided by default. Thanks! Joël Lamotte

1. The boost::extension author reported (when i asked him) a lack in the C++11 standard (without precision) that makes it hard to implement correctly this library on all platforms/compilers (unfortunately he didn't say which ones). Do you have any information about that?
I don't exactly understand what is the problem. You create an entry factory point like extern "C" { foo* bar(); } Resolve bar withing shared object/dll and it works 100% As long as you don't mix runtimes i.e. dll and main program uses different heaps and so on there is no problems I'm aware of.
2. booster::shared_object::shared_object(std::string const & file_name )
Here file_name is a path, right? I'm guessing it's relative to the way the OS will look for modules?
Actually it is quite same on Unix and Windows. A name without "path" like "foo.so" it searches withing general path, on windows .:$PATH and on Unix under predefined set of locations + LD_LIBRARY_PATH. If full/relative path provided it searches according to it.
3. Is there a specific reason that there isn't a provided abstraction (template "helper" function) that will call resolve_symbol() and try (dynamic or not) to cast it for you to the provided type? My understanding is that you wanted to let the user do as he wish but I was asking myself if common usage could be identified and provided by default.
There is a helper function that helps to cast void * to function pointer. Genrally any plugin/so/dll should have its entry point like shown above and implement some known interface. What would do the job. It is more abstraction of dlopen/dlsym rather then generic interface to map anything you want need dynamically.
Thanks!
Joël Lamotte
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Tue, Jan 10, 2012 at 21:15, Artyom Beilis <artyomtnk@yahoo.com> wrote:
1. The boost::extension author reported (when i asked him) a lack in the C++11 standard (without precision) that makes it hard to implement correctly this library on all platforms/compilers (unfortunately he didn't say which ones). Do you have any information about that?
I don't exactly understand what is the problem.
Unfortunately I don't have the details of the problems he reported, but I sent a mail asking for details. Didn't get an answer yet. Anyway Boost.Extension is find with most cases it seems so I guess that he was speaking about runtime mixes.
You create an entry factory point like
extern "C" { foo* bar(); }
Resolve bar withing shared object/dll and it works 100%
That always was my understanding too.
As long as you don't mix runtimes i.e. dll and main program uses different heaps and so on there is no problems I'm aware of.
It might be the problem, as C++ standard don't define anything about all this, obviously. Anyway I like the simplicity of your solution, it seems easy to build on. Another minor question : what is the rational in choosing std::string in the interface instead of const char* ? Joël Lamotte.

As long as you don't mix runtimes i.e. dll and main program uses different heaps and so on there is no problems I'm aware of.
It might be the problem, as C++ standard don't define anything about all this, obviously.
C++ does not say anything on shared object and dlls :-) But we relay on OS for such stuff...
Anyway I like the simplicity of your solution, it seems easy to build on.
Another minor question : what is the rational in choosing std::string in the interface instead of const char* ?
Why not? std::string is convertable from char const * also "creating string" price is quite negligible in comparison to loading library... Artyom

On Wed, Jan 11, 2012 at 10:30, Artyom Beilis <artyomtnk@yahoo.com> wrote:
Why not? std::string is convertable from char const * also "creating string" price is quite negligible in comparison to loading library...
I agree but lot of libraries that take such inputs uses const char * to avoid forcing to use std::string, as some plateform implement it in a non efficient way (I'm thinking embedded), so I was asking to know if there is a particular reason. I personally don't mind, I also don't know if it's a important thing for boost, but it might be if you try to standardize it one day I guess. On Wed, Jan 11, 2012 at 11:50, Thorsten Ottosen <thorsten.ottosen@dezide.com
wrote:
The fourth I don't know about. Can't Boost.FileSystem do all that?
Boost.FileSystem is about the file system only, not about all the OS services. Joël Lamotte

Klaim - Joël Lamotte wrote:
On Wed, Jan 11, 2012 at 10:30, Artyom Beilis <artyomtnk@yahoo.com> wrote:
Why not? std::string is convertable from char const * also "creating string" price is quite negligible in comparison to loading library...
I agree but lot of libraries that take such inputs uses const char * to avoid forcing to use std::string, as some plateform implement it in a non efficient way (I'm thinking embedded), so I was asking to know if there is a particular reason. I personally don't mind, I also don't know if it's a important thing for boost, but it might be if you try to standardize it one day I guess.
The cost of creating a std::string may be relatively negligible, but that doesn't make it zero. Don't forget that it puts additional pressure on the free store. If a std::string is not needed, don't create one. That said, the best approach might simply be overloading. If clients have a std::string, it can bind to a std::string const & parameter, thus avoiding the need to use c_str() (in the char const *-only approach). If clients have a char const *, an overload accepting that will prevent creating a std::string unnecessarily. _____ Rob Stewart robert.stewart@sig.com Software Engineer using std::disclaimer; Dev Tools & Components Susquehanna International Group, LLP http://www.sig.com ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
Very Very useful for debugging.
http://cppcms.sourceforge.net/cppcms_ref_v0_99/namespacebooster_1_1stack__tr... http://cppcms.sourceforge.net/cppcms_ref_v0_99/backtrace_8h_source.html
Any interest in this library. It works (collects a trace and prints it) on:
Windows, Solaris, Linux, Mac OS X
Very interested. How does this work? Thanks, Nate

----- Original Message -----
From: Nathan Ridge <zeratul976@hotmail.com>
Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
Very Very useful for debugging.
http://cppcms.sourceforge.net/cppcms_ref_v0_99/namespacebooster_1_1stack__tr... http://cppcms.sourceforge.net/cppcms_ref_v0_99/backtrace_8h_source.html
Any interest in this library. It works (collects a trace and prints it) on:
Windows, Solaris, Linux, Mac OS X
Very interested. How does this work?
Very simple: http://cppcms.svn.sourceforge.net/viewvc/cppcms/framework/trunk/booster/lib/backtrace/src/backtrace.cpp?revision=2052&view=markup In short. Linux, Solaris, Mac OS X: - execinfo.h backtrace+backtrace_symbols/dladdr Windows - RtlCaptureStackBackTrace - SymFromAddr (for MSVC) Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/

Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
How does this work?
Very simple:
In short.
Linux, Solaris, Mac OS X:
- execinfo.h backtrace+backtrace_symbols/dladdr
Windows
- RtlCaptureStackBackTrace
- SymFromAddr (for MSVC)
So where does the symbol information come from? From the debug information compiled into the program? If so, what happens if I compile the program without debug info? What happens if I use an optimized build? Thanks, Nate

________________________________ From: Nathan Ridge <zeratul976@hotmail.com> To: Boost Developers Mailing List <boost@lists.boost.org> Sent: Tuesday, January 10, 2012 11:48 PM Subject: Re: [boost] Request for Interest in several Modules
Boost.StackTrace
Collecting stack trace automatically from exception and printing it.
How does this work?
Very simple:
[snip]
In short.
Linux, Solaris, Mac OS X:
- execinfo.h backtrace+backtrace_symbols/dladdr
Windows
- RtlCaptureStackBackTrace
- SymFromAddr (for MSVC)
So where does the symbol information come from?
Under ELF platforms (Linux, Solaris, Mac OS X) it uses linking symbols, they are built in in shared objects and can be added by using -rdynamic flag (which is recommended flag to use in any case) They are usually resolved using dladdr library call. Under Windows/MSVC it uses debug information.
From the debug information compiled into the program?
Using MSVC it does not have to be compiled to the program, it may be external files (PDB files)
If so, what happens if I compile the program without debug info?
You would not get resolved symbol names under Visual Studio
What happens if I use an optimized build?
Optimization does not prevent generation of debug information. For example CMake's flag RelWithDebInfo does this automatically and perfectly for you, of course you can set your own flags in VS projects to make this work. If I remember correctly using /Zi would happily generate debug information (PDB files) for release builds
Thanks, Nate
Best, Artyom

Den 10-01-2012 16:11, Artyom Beilis skrev:
Hello,
I have several modules/libraries I developed and released under Boost License.
- CppDB Sql Connectivity - Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API.
I'm very interested in the first three. So please go ahead and find a review manager and start boosty-fying. The fourth I don't know about. Can't Boost.FileSystem do all that? -Thorsten

On 01/10/2012 04:11 PM, Artyom Beilis wrote:
Hello,
I have several modules/libraries I developed and released under Boost License.
- CppDB Sql Connectivity
Taking an example from your documentation: cppdb::statement <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html> st=sql.prepare("DELETE FROM users WHERE age<? AND role<>?"); st.bind <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html#add4e469140ddd77aadc5d6c8cfb8ba11>(1,13); st.bind <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html#add4e469140ddd77aadc5d6c8cfb8ba11>(2,"moderator"); In our company we have combination of code generator and template library which allows to do the following TabUsers tab; tab.remove(tab.age<13 and role!="moderator"); The compiler can then ensure correct names for tables and columns, typesafety and correct syntax. This makes database programming much easier because there is much less room for stupid mistakes. It really is an enormous difference, but it requires a code generator and I am not sure if that would be acceptable for boost. If yes: I would really love to see a such a concept being used on top of a library like yours. Regards, Roland

----- Original Message -----
From: Roland Bock <rbock@eudoxos.de> To: boost@lists.boost.org
In our company we have combination of code generator and template library which allows to do the following
TabUsers tab; tab.remove(tab.age<13 and role!="moderator");
The compiler can then ensure correct names for tables and columns, typesafety and correct syntax. This makes database programming much easier because there is much less room for stupid mistakes.
It really is an enormous difference, but it requires a code generator and I am not sure if that would be acceptable for boost. If yes: I would really love to see a such a concept being used on top of a library like yours.
See, CppDB is not ORM and it is not ought to be ORM. It is Sql connectivity library. It is more like JDBC for C++. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/
Regards,
Roland
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 01/11/2012 05:39 PM, Thorsten Ottosen wrote:
Den 11-01-2012 13:36, Artyom Beilis skrev:
See, CppDB is not ORM and it is not ought to be ORM. It is Sql connectivity library.
It is more like JDBC for C++.
I guess we can always built a type safe layer on top later, if we want.
-Thorsten
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Makes sense :-)

On 11 January 2012 12:29, Roland Bock <rbock@eudoxos.de> wrote:
On 01/10/2012 04:11 PM, Artyom Beilis wrote:
- CppDB Sql Connectivity
Taking an example from your documentation:
cppdb::statement <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html> st=sql.prepare("DELETE FROM users WHERE age<? AND role<>?"); st.bind <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html#add4e469140ddd77aadc5d6c8cfb8ba11>(1,13); st.bind <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html#add4e469140ddd77aadc5d6c8cfb8ba11>(2,"moderator");
In our company we have combination of code generator and template library which allows to do the following
TabUsers tab; tab.remove(tab.age<13 and role!="moderator");
The compiler can then ensure correct names for tables and columns, typesafety and correct syntax. This makes database programming much easier because there is much less room for stupid mistakes.
Sounds similar to http://www.codesynthesis.com/products/odb/ Best regards, -- Mateusz Loskot, http://mateusz.loskot.net

On 01/11/2012 03:06 PM, Mateusz Loskot wrote:
On 11 January 2012 12:29, Roland Bock <rbock@eudoxos.de> wrote:
On 01/10/2012 04:11 PM, Artyom Beilis wrote:
- CppDB Sql Connectivity Taking an example from your documentation:
cppdb::statement <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html> st=sql.prepare("DELETE FROM users WHERE age<? AND role<>?"); st.bind <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html#add4e469140ddd77aadc5d6c8cfb8ba11>(1,13); st.bind <http://cppcms.sourceforge.net/sql/cppdb/classcppdb_1_1statement.html#add4e469140ddd77aadc5d6c8cfb8ba11>(2,"moderator");
In our company we have combination of code generator and template library which allows to do the following
TabUsers tab; tab.remove(tab.age<13 and role!="moderator");
The compiler can then ensure correct names for tables and columns, typesafety and correct syntax. This makes database programming much easier because there is much less room for stupid mistakes. Sounds similar to http://www.codesynthesis.com/products/odb/
Best regards, Thanks for the link!
As far as I can tell from a few minutes of reading, their approach uses the same idea, but starts from C++ classes, where we start from SQL table definitions. I wonder how changes in the data model look like (e.g. adding a new column)? We can migrate the database, dump the new table structure and generate the code. I'll keep on reading! Thanks again! Regards, Roland

Den 11-01-2012 13:29, Roland Bock skrev:
In our company we have combination of code generator and template library which allows to do the following
TabUsers tab; tab.remove(tab.age<13 and role!="moderator");
The compiler can then ensure correct names for tables and columns, typesafety and correct syntax. This makes database programming much easier because there is much less room for stupid mistakes.
Maybe we can get 90% of it by something as simple as class Users { BOOST_DECLARE_SQL_CLASS( "USERS" ); BOOST_SQL_VAR( int, "column_age", age ); BOOST_SQL_VAR( std::string, "column_role", role ); ... }; -Thorsten

On 01/11/2012 07:31 PM, Thorsten Ottosen wrote:
Den 11-01-2012 13:29, Roland Bock skrev:
In our company we have combination of code generator and template library which allows to do the following
TabUsers tab; tab.remove(tab.age<13 and role!="moderator");
The compiler can then ensure correct names for tables and columns, typesafety and correct syntax. This makes database programming much easier because there is much less room for stupid mistakes.
Maybe we can get 90% of it by something as simple as
class Users { BOOST_DECLARE_SQL_CLASS( "USERS" );
BOOST_SQL_VAR( int, "column_age", age ); BOOST_SQL_VAR( std::string, "column_role", role ); ... };
-Thorsten
Probably yes, but based on our current code generator I'd say it would be quite a lot of macro code, which I find hard to debug.

Den 12-01-2012 08:43, Roland Bock skrev:
On 01/11/2012 07:31 PM, Thorsten Ottosen wrote:
Maybe we can get 90% of it by something as simple as
class Users { BOOST_DECLARE_SQL_CLASS( "USERS" );
BOOST_SQL_VAR( int, "column_age", age ); BOOST_SQL_VAR( std::string, "column_role", role ); ... };
-Thorsten
Probably yes, but based on our current code generator I'd say it would be quite a lot of macro code, which I find hard to debug.
Well, I don't see why it has to be that way. We can maybe avoid macros a little by doing class Users { BOOST_DECLARE_SQL_CLASS( "USERS" ); BOOST_DECLARE_SQL_COLUMNS( "column_age", age, "column_role", role ); boost::sql::variable<int> age; boost::sql::nullable_variable<std::string> role; ... }; AFAICT, the only thing you have to debug is mismatch in the column/table names. But that should be easy to detect at run-time, thus easy to catch in unit-tests. -Thorsten

On 01/12/2012 10:20 AM, Thorsten Ottosen wrote:
Den 12-01-2012 08:43, Roland Bock skrev:
On 01/11/2012 07:31 PM, Thorsten Ottosen wrote:
Maybe we can get 90% of it by something as simple as
class Users { BOOST_DECLARE_SQL_CLASS( "USERS" );
BOOST_SQL_VAR( int, "column_age", age ); BOOST_SQL_VAR( std::string, "column_role", role ); ... };
-Thorsten
Probably yes, but based on our current code generator I'd say it would be quite a lot of macro code, which I find hard to debug.
Well, I don't see why it has to be that way. We can maybe avoid macros a little by doing
class Users { BOOST_DECLARE_SQL_CLASS( "USERS" ); BOOST_DECLARE_SQL_COLUMNS( "column_age", age, "column_role", role );
boost::sql::variable<int> age; boost::sql::nullable_variable<std::string> role; ... };
AFAICT, the only thing you have to debug is mismatch in the column/table names. But that should be easy to detect at run-time, thus easy to catch in unit-tests.
-Thorsten You may be very well be right. Currently we generate lots of additional code, to allow for stuff like this:
Users users; users.insert( users.age(42), users.firstName("Roland"), users.lastName("Bock")); The insert method knows the required columns (you can specify the non-required columns, too, of course). It is certainly possible that this can be done with as little code as you suggested and with very little amounts of macro code. I'll definitely think about it in more detail. Regards, Roland

Den 12-01-2012 12:10, Roland Bock skrev:
You may be very well be right. Currently we generate lots of additional code, to allow for stuff like this:
Users users; users.insert( users.age(42), users.firstName("Roland"), users.lastName("Bock"));
The insert method knows the required columns (you can specify the non-required columns, too, of course).
So you get a compile error if a column is required, but not specified? -Thorsten

On 01/12/2012 03:34 PM, Thorsten Ottosen wrote:
Den 12-01-2012 12:10, Roland Bock skrev:
You may be very well be right. Currently we generate lots of additional code, to allow for stuff like this:
Users users; users.insert( users.age(42), users.firstName("Roland"), users.lastName("Bock"));
The insert method knows the required columns (you can specify the non-required columns, too, of course).
So you get a compile error if a column is required, but not specified?
-Thorsten
Right. That's the whole idea of what we created: Let the compiler check as much as possible if our code is correct. This applies to insert statements, to where expressions, to retrieved values (the compiler knows which value (name, type) is returned in each column), etc. Earlier on, I tried to avoid database interaction where possible, but these days, with the compiler telling me most of the things I could do wrong, database programming is fun :-)

Den 13-01-2012 17:43, Roland Bock skrev:
On 01/12/2012 03:34 PM, Thorsten Ottosen wrote:
So you get a compile error if a column is required, but not specified?
-Thorsten
Right. That's the whole idea of what we created: Let the compiler check as much as possible if our code is correct. This applies to insert statements, to where expressions, to retrieved values (the compiler knows which value (name, type) is returned in each column), etc.
I think that should be possible too, but I wonder if the difference between an runtime and compile time error is a major problem here. -Thorsten

On 01/17/2012 04:35 PM, Thorsten Ottosen wrote:
Den 13-01-2012 17:43, Roland Bock skrev:
On 01/12/2012 03:34 PM, Thorsten Ottosen wrote:
So you get a compile error if a column is required, but not specified?
-Thorsten
Right. That's the whole idea of what we created: Let the compiler check as much as possible if our code is correct. This applies to insert statements, to where expressions, to retrieved values (the compiler knows which value (name, type) is returned in each column), etc.
I think that should be possible too, but I wonder if the difference between an runtime and compile time error is a major problem here.
-Thorsten
Compile time checks reduce the number of unit tests you have to write. And if this doesn't increase compile time too much, it improves the overall database programming experience :-)

Artyom Beilis wrote
I have several modules/libraries I developed and released under Boost License.
- CppDB Sql Connectivity
Artyom, I'd really like to see CppDB proposed for review. This would be a great opportunity to resurrect discussion on design and implementation of super-duper C++ library for communication with DBMS. There is a great deal of history of such discussions: - SOCI library has been considered as a candidate for Boost'fication [1] - In 2009, BoosCon hosted std::rdb workshops [2] with plenty of discussions [3] - Jean-Louis Leroy worked hard to implement Boost.Rdb proposal [4], discussed here too. - number of ORM proposals (external) discussed All these activities has died or suspended due to lack of time, interest, consensus, whatever...but the subject seems to be alive as from time to time people ask for it, give new proposal, express interest, etc. So, a real review of DBMS library could bring the topic back and perhaps helped to achieve more. [1] http://soci.sf.net [2] http://boostcon.boost.org/program/previous-years/2009-program/ [3] http://mail-lists.crystalclearsoftware.com/listinfo.cgi/std_rdb-crystalclear... [4] http://code.google.com/p/boost-rdb/ Best regards, Mateusz ----- -- Mateusz Loskot, http://mateusz.loskot.net Charter Member of OSGeo, http://osgeo.org Member of ACCU, http://accu.org -- View this message in context: http://boost.2283326.n4.nabble.com/Request-for-Interest-in-several-Modules-t... Sent from the Boost - Dev mailing list archive at Nabble.com.

On Wed, Jan 11, 2012 at 10:46 PM, mloskot <mateusz@loskot.net> wrote:
I'd really like to see CppDB proposed for review. This would be a great opportunity to resurrect discussion on design and implementation of super-duper C++ library for communication with DBMS.
A generic RDBMS library using typesafe modern C++ is great, but one thing to keep in mind is performance and not fall into the most-common-denominator pitfall. For example, using Oracle OCI, I see a 5x performance improvement on inserts from using array operations (like OCIBindArrayOfStruct and co.) and minimizing round-trips to the server, compared to doing "scalar" inserts, as most wrapper libraries I know of perform. Blob read/write can similarly be greatly optimized via array-ops (from 4x to > 20x depending in size and number of blobs). Unless the community decides the high-performance use cases are out-of-bounds as too DB-vendor specific, but let that be a conscious decision at least, not an unforeseen consequence of the library's design/API. --DD

Den 12-01-2012 09:08, Dominique Devienne skrev:
Unless the community decides the high-performance use cases are out-of-bounds as too DB-vendor specific, but let that be a conscious decision at least, not an unforeseen consequence of the library's design/API.
Well, if a library could not support high-performance cases because of its design, I would vote "reject" immediately. I'm ok with the fact that its first release does not support it, but I want to be able to do so in the future. -Thorsten

----- Original Message -----
From: Thorsten Ottosen <thorsten.ottosen@dezide.com> To: boost@lists.boost.org Cc: Sent: Thursday, January 12, 2012 11:24 AM Subject: Re: [boost] Request for Interest in several Modules
Den 12-01-2012 09:08, Dominique Devienne skrev:
Unless the community decides the high-performance use cases are out-of-bounds as too DB-vendor specific, but let that be a conscious decision at least, not an unforeseen consequence of the library's design/API.
Well, if a library could not support high-performance cases because of its design, I would vote "reject" immediately. I'm ok with the fact that its first release does not support it, but I want to be able to do so in the future.
Actually it is high performance by design from my experience I get sometimes x2-x3 performance advantage over other connectors I've used, however it does this in a different way: - It uses prepared statements exclusively by default - It caches prepared statements by default in transparent way, such that any following use of the statement reuses it - It has very easy and transparent connection pooling. Not little bit about type system. CppDB type system is dynamic, it allows you to cast almost anything to anything (integer to string, string to integer, date to string and so on) This make cross DB programming much easier as same queries and statements work across different RDBMS. It was one of the major ideas and goals behind CppDB because I was frustrated with some existing connectors that had too strict type system. So if you are looking for vendor specific functions that allow you to do some kind of data transfer directly from socket to memory buffer it is not for you, especially when only one API around supports it (OCI) Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.sf.net/ CppDB - C++ SQL Connectivity: http://cppcms.sf.net/sql/cppdb/

On Thu, Jan 12, 2012 at 11:02 AM, Artyom Beilis <artyomtnk@yahoo.com> wrote:
So if you are looking for vendor specific functions that allow you to do some kind of data transfer directly from socket to memory buffer it is not for you, especially when only one API around supports it (OCI)
You have to use a vendor-specific API in the back-end anyways. What matters is to have the user-facing API not preventing optimizations in the back-end in the case you want to insert 100,000 rows to a table. If the library forces to call stmt.execute() for each row, and thus a network round-trip to the server, you're far from high-performance. I doubt Oracle OCI is the only vendor or DBMS API that offers ways to improve such a case (and I'm not talking of its Direct Path API which bypasses the normal DML path). As an example, you showed binding a scalar to a prepared statement's placeholder, which Oracle supports of course, but it also allows binding an array of scalars almost as simply, and sends all those in a single round-trip. One could agree it's the job of a back-end to accumulate the scalar values and behind the scene accumulate them in an array, similar to Oracle's prefetching but in reverse, but it could also be in the API itself, which is simply converted to a loop for back-ends that do not support bulk array operations. I don't dispute that your wrapper is 2x or 3x faster than existing wrappers, (and I was already taking prepared statements reuse for granted in fact), but was discussing the kind of large insert or select above. SQLite doesn't have the kind of bulk operations I mention, simply because it's a server-less SQL engine talking directly to the filesystem, with no network overhead, but for client-server RDBMS you simply cannot afford to ignore network round-trips to have the best performance. As you rightly point out, you don't aim to be an ORM (which often by their nature of traversing an object graph and doing small "scattered" scalar inserts, can't leverage bulk operations, and therefore can't achieve best performance) and concentrate on just the DBMS connectivity, but all I'm saying is that bulk insert or select are precisely something a targeted library like this should address, one way or another, to call itself high-performance. Just my $0.02. --DD

On 12 January 2012 09:24, Thorsten Ottosen <thorsten.ottosen@dezide.com> wrote:
Den 12-01-2012 09:08, Dominique Devienne skrev:
Unless the community decides the high-performance use cases are out-of-bounds as too DB-vendor specific, but let that be a conscious decision at least, not an unforeseen consequence of the library's design/API.
Well, if a library could not support high-performance cases because of its design, I would vote "reject" immediately. I'm ok with the fact that its first release does not support it, but I want to be able to do so in the future.
Thorsten, Dominique, Yes, I agree. As a PostgreSQL, it also is important to me to perform transfer in binary mode instead of textual, also for performance. I have exchanged a few e-mails with Artyom and our opinions regarding text vs binary transfer mode are different here. SOCI and CppDb use text mode. (I've worked on binary mode for SOCI. Unfortunately, SOCI has lost the lead developer and its future is uncertain. If I manage to release binary mode, it will be as separate/forked library, I guess.) Best regards, -- Mateusz Loskot, http://mateusz.loskot.net

On Wed, Jan 11, 2012 at 2:11 AM, Artyom Beilis <artyomtnk@yahoo.com> wrote:
- Stack Trace - Shared Object Loading - Nowide UTF-8 friendly standard C/C++ library API.
No objections against the SQL library, but because I don't do any heavy development that 'touches' that area I will leave my opinion out of it (even though it's a positive one). I would definitely love to see the other three libraries proposed for review though, as I would use all of them (especially the stack trace one, which would make a much nicer replacement for my current implementation, which is specific to Windows).
participants (17)
-
Artyom Beilis
-
Brian Ravnsgaard Riis
-
Christian Holmquist
-
Christof Donat
-
Dominique Devienne
-
Fernando Pelliccioni
-
Giovanni Piero Deretta
-
Joshua Boyce
-
Klaim - Joël Lamotte
-
Marshall Clow
-
Mateusz Loskot
-
Mathias Gaunard
-
mloskot
-
Nathan Ridge
-
Roland Bock
-
Stewart, Robert
-
Thorsten Ottosen