Thorsten Ottosen wrote:

...

Please see <http://soci.sourceforge.net/doc/boost.html> for details.

All the optional examles use 'is_initialized()'. It seems like the non-idiomatic way:

if (name.is_initialized())

should be

if( name )

IMO.

I disagree, even though it is not my responsibility to define Boost idioms. ;-) The point is that being optional and being a boolean flag are two different things. Merging them under the same syntax saves keystrokes, but not the effort to review and understand the code that is already written. BTW - what about optional<bool>? optional<bool> optional_switch = ... if (optional_switch) { // the switch can be "false" here! } Isn't it confusing?

John Maddock has made the tool "bcp"

http://www.boost.org/doc/libs/1_35_0/tools/bcp/bcp.html

that allows one to easily make a boost library into an independent project.

I should have a look at this. -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Felipe Magno de Almeida wrote:

Do you find confusing to use a pointer to a bool too?

Yes. That's why I stick to the following convention: if (pointer != NULL) ... instead of: if (pointer) ...

But this is only personal taste IMHO.

Yes, although a sum of such tastes often leads to a coding standard, where they are no longer personal. -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Ion Gaztan~aga wrote:

1) I think a solid ODBC backend is a must for a Boost DB library. After all, ODBC is the most widely used driver. Is SOCI ODBC compatibility mature or still needs serious work?

Some SOCI users enjoy the existing ODBC backend in production code. As far as I understand, it does not require any major work and that's why I'm pretty sure that the next library release will include the ODBC backend as a supported part of the whole. Very likely the binary release for Windows users (lib+dll) will be available as well.

2) Do we know how efficient SOCI is?

One of our users performed tests comparing SOCI to other libraries, including the native client library for one database. The conclusion from the test was that with optimized build (like -O2 for g++) the performance of SOCI is comparable to other alternatives, with few percent of overhead in relation to native client library, for *these* tests. Note also that SOCI supports bulk operations in terms of vector<T>. The Oracle backend is currently the only one that fully benefits from it by performing data transmissions in blocks (it reduces network roundtrips), binding to raw underlying memory buffer for direct data transport. The scalability potential of this solution is huge, but the actual results can be very sensitive to the details of the given test.

I understand that database operations might be in most cases more expensive than SOCI operations

Yes, this is true. Of course, SOCI adds its own overhead in terms of dynamic data structures that are created here and there, but the important property of SOCI is that in its "canonical" idioms the library binds directly to data provided by the user. For example: int salary; sql << "select salary from persons where id = 123", into(salary); Above, the result of the query is put *directly* in the salary variable if the given backend supports binary data transmission (Oracle, ODBC) and there is only one conversion step if the backend operates on strings internally (PostgreSQL, MySQL, ...). The same is true for explicit statements. In other words, the overhead of SOCI is a one-time investment for each statement and is necessary to keep the statement in shape and to associate data references with query results. After the statement is created, all subsequent operations use existing structures for guidance what to put where and if the statement is repeated many times - which is where performance actually matters - the copy/conversion overhead is minimal or just zero. (Well, we had to add some additional data copying to ensure const-correctness as perceived by the user code in *some* cases, but this would need to be done with any other alternative approach as well.) The only place where SOCI introduces some significant overhead is when complicated user-defined datatypes are involved, since data needs to be often copied and converted back and forth. This is the price for flexibility. But hey, native client libraries and even ODBC have no idea how to translate the query results into boost::fusion<std::string, boost::optional<MyIntegerWithRange>, boost::gregorian::date>, right? ;-) There are also other idioms supported by SOCI, like dynamic rowset description or stream-like data extraction and they can (depending on the backend) introduce additional overhead. Again, it is the price for ultimate flexibility. For those who are performance-paranoid, the basic statically bound idioms are recommended. Having said all the above, it is important to note that with databases the biggest opportunity for performance tuning is not in the client interface, but in SQL queries and database structures themselves. The rest is... Measure It Yourself. ;-) -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Phil Endecott wrote:

Congratulations on the new release. I've had a quick look, and as far as I can see it still always uses passes query arguments, even in prepared statements, as text strings. In my code, I have enjoyed the increased type safety and small performance improvement from passing them in binary.

It depends on the given backend, but binary format is most natural in SOCI. Oracle always uses binary transmission of query arguments, which are passed separately from the query itself (so-called bound variables, they are also essential for avoiding SQL-injection issues). PostgreSQL always uses text transmission of query arguments, and they are also passed separately from the query. I was meditating on the binary data transmission, but the interface is basically ill-designed and even PostgreSQL folks recommended the use of text format for portability (details: with PostgreSQL the binary format has to comply with *server* conventions not the client's and you never know what they are). I'm not sure about MySQL.

Is this an inherent limitation of your frontend/backend API ?

On the contrary - the frontend/backend API is inherently binary-oriented. Backends have to handle the differences if they cannot do binary transmission, but frontend is not aware of this detail. If you use Oracle, then everything is done in binary. I think it is also true for ODBC. -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Phil Endecott wrote:

...

PostgreSQL always uses text transmission of query arguments, and they are also passed separately from the query.

...

I was meditating on the binary data transmission, but the interface is basically ill-designed and even PostgreSQL folks recommended the use of text format for portability (details: with PostgreSQL the binary format has to comply with *server* conventions not the client's and you never know what they are).

Hmm. I was under the impression that ints were always sent in network byte order (i.e. big endian), and I think I would have noticed if I'd got it wrong because I've only ever used it on little-endian systems. Is there some other convention that you're thinking of?

According to <http://www.postgresql.org/docs/8.3/interactive/libpq-exec.html>, the format is: "a zero-terminated text string (for text format) or binary data in the format expected by the server (for binary format)" Nothing is said on what that format is and I understand that it might depend on how the server was compiled, what is the target platform, etc. There is a commend at the bottom of this page, by Jason Lenthe, that the format is network byte order, so this might be some hint and also in agreement with what you say. Is it worth the trouble? I'm not sure. What about object length? Let's suppose that user does this: long long salary; sql << "select salary from ...", into(salary); and the server returns binary data of length 4. Still, the access library would need to figure out that it is not what is expected, and do the necessary conversion. Let's try something even more complex: double salary; sql << "select salary from ...", into(salary); and the server returns the same as before. What now? The access library would need to figure out not only the size difference, but also *type* difference and the conversion would be inevitable anyway. Why not: std::string salary; // ... and again, the access library would need to handle the difference. The number of different types that the server might return is quite big and all conversion combinations would need to be handled. I don't think that it is worth the trouble with PostgreSQL. It is much better with Oracle, where the client defines the expected data types before executing the query and the server (or the native client library) does the rest, so that no discovery and conversion is needed in the client code. The difference between these two approaches can be shortly summarized thus: in Oracle you get what you ask for (and then binary makes sense), whereas in PostgreSQL you get whatever the server gives you (and you have to figure out how to digest it). In any case, the fact that PostgreSQL uses text for data transfer is an implementation detail of this backend. It is possible to implement alternative backend for binary transfer - the rest of the library is prepared for this. -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Phil Endecott wrote:

When necessary, the approach that I've taken is for the client to define the expected data type, but to do this using a cast inside the SQL string:

"select salary::int64 from ..."

You can then be fairly certain that you'll actually get a 64-bit value back.

Then at execution time, all I do is a quick check that the retuned column types are the expected ones, and throw if they aren't.

This is a big burden on the user and is very fragile. Consider: typedef int SalaryType; // ... SalaryType salary; sql << "select salary from ...", into(salary); Not only it is not clear at the use point what type should be put in the textual query, but whatever you put there will fail just a few months later when you upgrade the source code and change the typedef in the one place where it is defined. Facilitating such changes is one of the purposes of typedef, after all. Note also that the above solution is impractical if queries are not encoded in source code, but exist externally, for example in resource files (or even in a database). The same query might be reused in many places, even if the target data type might be different. Note also that calling stored procedures is tricky/impractical with this approach, because result types are encoded in the signature or definition of the procedure and are not normally exposed in the query that calls it. Regards, -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Is there something availaible which supports directly the Microsoft SQL Server (not only ODBC). Or is something planned for the future? Best regards Hansjörg Dr. Franz Fehringer schrieb:

Hello,

I have two questions:

1) Does SOCI support IBM DB2 in general? 2) same but for z/OS?

Thanks for the great work

Franz

Maciej Sobczak schrieb:

Hello, Dr. Franz Fehringer wrote:

I have two questions:

1) Does SOCI support IBM DB2 in general?

In general there is nothing that would inherently prohibit it, but in particular no backend exists for this database. :-)

2) same but for z/OS?

And same answer. SOCI is divided into "core" part, which handles interfacing with client code, and "backends", which handle interfacing with databases. The set of backends is open and new contributions are always welcome. Certainly, the more specialized database the more difficult it is to find a volunteer with appropriate skillset. The CVS repository contains the "empty" backend which is a scaffolding for implementing new backends. It is also recommended to have a look at the implementation of existing backends, especially those which are conceptually similar to the new one in terms of API, data types and their handling, etc. If there are volunteers interested in targeting DB2 or z/OS, please contact me privately. -- Maciej Sobczak * www.msobczak.com * www.inspirel.com

Rob Kramer wrote:

...

soci::session sql("postgresql://dbname=my_database");

As a result, the SOCI library will try to locate the dynamically loadable plugin (called backend) for PostgreSQL and handle all further database-related activity.

Sorry for being off-topic, but is this type of plugin loading a Boost framework too?

Certainly, there is a chance that some of it might be overlapping. On the other hand, the mechanism in SOCI is tightly coupled to its specific needs (like library name selection, path search, symbol caching, etc.). -- Maciej Sobczak * www.msobczak.com * www.inspirel.com