I have uploaded a new version of my relational database library to the vault. It implements the ideas I described in another thread. This time it even puts things in a real (odbc) database and retrieves them. I see that 12 have downloaded the previous versions. If you have time please send me feedback. Have you been able to compile it ? Has anybody tried on Unix or with other compilers ? Please note: * It's a work in progress ; for the curious only. * I develop on Windows with Visual Studio Express 2008. There is a solution file in the msvc dir. Nothing bjam works yet. * Currently the doc is the test suite Jean-Louis Leroy
I also happened to start casually working on this problem about a month ago without knowing anyone else was working on a similar solution. :) Curses! For the most part, I like Jean-Louis's solution better than my own, except that I think columns need to support column-constraints i.e. primary key, not null, unique, etc. Also, I'm thinking there will be a need for foreign keys to be defined. Maybe defining a table would look something like the following: Where the column macro is as such: #define BOOST_RDB_COLUMN(NAME, sql_type, constraint_list) And two new classes are defined: constraint and constraint_list and the comma operator is overloaded to allow constraints to be chained into a constraint list. namespace springfield { BOOST_RDB_BEGIN_TABLE(person) BOOST_RDB_COLUMN(id, integer, (constraint::primary_key)) BOOST_RDB_COLUMN(name, varchar<20>, (constraint::not_null)) BOOST_RDB_COLUMN(first_name, varchar<30>, (constraint::none)) BOOST_RDB_COLUMN(age, integer, (constraint::none)) BOOST_RDB_COLUMN(married, boolean, (constraint::not_null, constraint::default(false))) BOOST_RDB_END_TABLE(person) BOOST_RDB_BEGIN_TABLE(partner) BOOST_RDB_COLUMN(husband, integer, (constraint::not_null)) BOOST_RDB_COLUMN(wife, integer, (constraint::not_null)) BOOST_RDB_FOREIGN_KEY(husband, person::id) BOOST_RDB_FOREIGN_KEY(wife, person::id) BOOST_RDB_END_TABLE(partner) } Regards, -Joshua Pedrick On Sun, Sep 20, 2009 at 7:46 PM, Jean-Louis Leroy <jl@yorel.be> wrote:
I have uploaded a new version of my relational database library to the vault. It implements the ideas I described in another thread. This time it even puts things in a real (odbc) database and retrieves them.
I see that 12 have downloaded the previous versions. If you have time please send me feedback. Have you been able to compile it ? Has anybody tried on Unix or with other compilers ?
Please note: * It's a work in progress ; for the curious only. * I develop on Windows with Visual Studio Express 2008. There is a solution file in the msvc dir. Nothing bjam works yet. * Currently the doc is the test suite
Jean-Louis Leroy
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- -- יהושע
Am Monday 21 September 2009 17:13:34 schrieb Joshua Pedrick:
namespace springfield { BOOST_RDB_BEGIN_TABLE(person) BOOST_RDB_COLUMN(id, integer, (constraint::primary_key)) BOOST_RDB_COLUMN(name, varchar<20>, (constraint::not_null)) BOOST_RDB_COLUMN(first_name, varchar<30>, (constraint::none)) BOOST_RDB_COLUMN(age, integer, (constraint::none)) BOOST_RDB_COLUMN(married, boolean, (constraint::not_null, constraint::default(false))) BOOST_RDB_END_TABLE(person)
BOOST_RDB_BEGIN_TABLE(partner) BOOST_RDB_COLUMN(husband, integer, (constraint::not_null)) BOOST_RDB_COLUMN(wife, integer, (constraint::not_null)) BOOST_RDB_FOREIGN_KEY(husband, person::id) BOOST_RDB_FOREIGN_KEY(wife, person::id) BOOST_RDB_END_TABLE(partner) }
I only had a short look at your rdb code so far, as most of it is SQL syntax sugar and although that looks good that's not the most interesting part of a boost db layer for me. if you plan to develop this until a boost review please also consider an interface that allows you to dynamically define tables. your table definitions seem to rely on the static c++ type system right now, using mpl and fusion. an object relational mapper based on boost.serialization needs to be able to create tables on demand and also insert columns on demand, as a serialize() function can omit fields. I was planning to include a proof-of-concept object-relational-mapper in my object persistence library, using SQLite. eventually this should be based upon a boost database layer like rdb, but it seems a little early on to use it already. but please keep me updated on any progress, as I don't know yet when I get around to implementing the relational-mapper. some other internal stuff that is hardcoded to an own storage engine right now has to be made generic first. do you know dtemplatelib? http://dtemplatelib.sourceforge.net/ it may not be applicable in all cases but the idea of representing a db table as a STL container seems interesting.
Stefan Strasser wrote:
I only had a short look at your rdb code so far, as most of it is SQL syntax sugar and although that looks good that's not the most interesting part of a boost db layer for me.
The sql composition feature of my lib may be the tree that hides the forest. What matters is that it's typesafe and also more efficient. If the compiler does its job well ther will be no time penalty compared to hand-written code.
if you plan to develop this until a boost review Indeed ;-)
please also consider an interface that allows you to dynamically define tables. your table definitions seem to rely on the static c++ type system right now, using mpl and fusion. an object relational mapper based on boost.serialization needs to be able to create tables on demand and also insert columns on demand, as a serialize() function can omit fields.
Serialization deals with types purely at compile-time. So it would not have problems with the static approach my library is taking. Now I reckon that dynamic interaction is useful. You need to be able to implement Access after all ;-). If you want both dynamic and static mechanisms in the same library, there is a choice between wrapping a dynamic system in typesafe constructs (at a cost in efficiency but type safety is regained) and wrapping a typesafe system in dynamic constructs. But I am not sure that the same library should support both. It's certainly feasible but will it look better than two separate libs ? The argument can be extended to object-relational mappers, except that there is a lot more intelligence in them that may make the "one lib" approach more appropriate. At some point I will probably try the second approach though. I have ideas about it. Some about code (wrapping both fusion:: and std:: vectors and algorithms in a general static/dynamic abstraction), some about design (does one really need SQL composition in such tool anyway ? Writing SQL directly will always be easier, even if I'm putting a lot of effort in making the C++ emulation of the SQL inobstrusive).
I was planning to include a proof-of-concept object-relational-mapper in my object persistence library, using SQLite.
Actually I wanted to implement an orm in the style of my former work Tangram but I got sidetracked by the persisting lack of a boost rdb lib. The big problem with Tangram is that it is *too* dynamic : it took one virtual function call per member variable. I'd like to reduce it to one per table but it will be more static. Typically it will use Mirror meta-classes to achieve that.
eventually this should be based upon a boost database layer like rdb, but it seems a little early on to use it already. but please keep me updated on any progress I'll keep posting here and uploading to the vault frequently.
do you know dtemplatelib? http://dtemplatelib.sourceforge.net/
I had a look at it a long time ago. It's certainly a useful system of the dynamic type. Anyway, thanks to all for the feedback J-L
Am Monday 21 September 2009 23:03:34 schrieb Jean-Louis Leroy:
please also consider an interface that allows you to dynamically define tables. your table definitions seem to rely on the static c++ type system right now, using mpl and fusion. an object relational mapper based on boost.serialization needs to be able to create tables on demand and also insert columns on demand, as a serialize() function can omit fields.
Serialization deals with types purely at compile-time. So it would not have problems with the static approach my library is taking.
struct A{ bool have_value; B b; void serialize(Ar &ar){ ar & have_value; if(have_value) ar & b; } }; that's valid serialization code and introduces a new table column at runtime if the object is mapped to a rdb and only objects with !have_value have been serialized so far. (pseudo-code, real code would have to use nvp's to match values to columns)
If you want both dynamic and static mechanisms in the same library, there is a choice between wrapping a dynamic system in typesafe constructs (at a cost in efficiency but type safety is regained) and wrapping a typesafe system in dynamic constructs. But I am not sure that the same library should support both. It's certainly feasible but will it look better than two separate libs ? The argument can be extended to object-relational mappers, except that there is a lot more intelligence in them that may make the "one lib" approach more appropriate.
I think it should be in one lib. (static and dynamic I mean, not rdb access in general and object-relational mapping). how did you plan to represent views generated by a SELECT query? because the query results can contain columns from different tables so you'd have to have a "dynamic" row type anyway. (unless you tried to deduce the row type from the query type?)
struct A{ bool have_value; B b;
void serialize(Ar &ar){ ar & have_value; if(have_value) ar & b; } };
that's valid serialization code and introduces a new table column at runtime if the object is mapped to a rdb and only objects with !have_value have been serialized so far.
True, fully static approaches will create the column even if it's never used.
how did you plan to represent views generated by a SELECT query? because the query results can contain columns from different tables so you'd have to have a "dynamic" row type anyway. (unless you tried to deduce the row type from the query type?)
Exactly. My `select' construct remembers all the types that were presented to it. Constructs like views and subselects will be statically checked. Errors like using in the `values' clause a column from another table in an insert statement, or a column from the same table but of the wrong type are also caught at compile-time. I guess that one could also make a compile-time assertion that validates an entire schema. J-L
Am Tuesday 22 September 2009 01:03:19 schrieb Jean-Louis Leroy:
struct A{ bool have_value; B b;
void serialize(Ar &ar){ ar & have_value; if(have_value) ar & b; } };
that's valid serialization code and introduces a new table column at runtime if the object is mapped to a rdb and only objects with !have_value have been serialized so far.
True, fully static approaches will create the column even if it's never used.
that means that a user would have to manually define the tables his objects should be mapped to. I don't think that's an acceptable assumption for a boost o-r mapper, because the mapping would intrude on all user types, even though it's just another way to store an object. take e.g. the serialization of an STL container. a user has no idea how a STL container is serialized. e.g. what is the size()-NVP called? so how can he define that column? the fields in STL containers are at least named the same each time, but how can he know the private fields of a type he didn't write? so an o-r mapper must be able to dynamically create columns, and even remove columns, as there can be > 1 NVPs with the same name, so the column has to be moved to it's own table and mapped as a 1:n relationship. (STL container again)
how did you plan to represent views generated by a SELECT query? because the query results can contain columns from different tables so you'd have to have a "dynamic" row type anyway. (unless you tried to deduce the row type from the query type?)
Exactly. My `select' construct remembers all the types that were presented to it. Constructs like views and subselects will be statically
ok, now I see the benefit of C++ SQL constructs (beyond nicer syntax)
that means that a user would have to manually define the tables his objects should be mapped to. [snip] so an o-r mapper must be able to dynamically create columns, and even remove columns, as there can be > 1 NVPs with the same name, so the column has to be moved to it's own table and mapped as a 1:n relationship. (STL container again)
I have seen two philosophies wrt orm: with or without a description (call it "schema", meta-data or whatever) of the objects to be mapped. Tangram uses a schema and it creates the tables for you (see http://tangram.utsl.gen.nz/docs/Tangram/Tour.html#creating_a_schema). In the Perl world you have several modules of both type. The advantage of schema-less tools is that they are very easy to use (just throw objects at them). Schema-based tools otoh give you predictible behavior and known database schemas (i.e. that don't depend on the particular set of objects that have ben stored) that you can get past your average db admin ;-) I have used a predecessor of Tangram to store a representation of the Belgian Justice. It had 400+ business classes with many levels of inheritance and complex relationships. I doubt that a create-the-db-as-you-go tool would have cut it. OTOH there are many situations in which schema-based tools are overkill. J-L
I also happened to start casually working on this problem about a month ago without knowing anyone else was working on a similar solution. Well my lib doesn't need to remain a one-man show. As the internals stabilize I hope that others jump in. All contributions will get credited of course.
For the most part, I like Jean-Louis's solution better than my own, except that I think columns need to support column-constraints i.e. primary key, not null, unique, etc. Also, I'm thinking there will be a need for foreign keys to be defined.
That will come too. Like views. And maybe triggers. Currently my roadmap is : CRUD via direct execution ; prepared statements ; placeholders ; transactions ; dialects (Postgres would be a good candidate because it has a non-standard syntax for escaping quotes in string literals). Along the way I (or someone else) will implement "easy" things like constraints and views. I haven't thought a lot about the syntax of constraints yet. My first thoughts were to put them outside the table definition to avoid long arguments lists, e.g. BOOST_RDB_PRIMARY_KEY(Column) would specialize some template responsible for constarint creation. But your syntax looks very nice. I guess that `(constraint::not_null, constraint::default(false) ' builds a mpl::vector of types describing carrying the extra information ? Matus Chochlik uses _ for defaults in Mirror (btw most of the TABLE.COLUMN machinery is inspired by Matus' work). The COLUMN macro could automatically fish from the constraints namespace : BOOST_RDB_BEGIN_TABLE(person) BOOST_RDB_COLUMN(id, integer, (primary_key)) BOOST_RDB_COLUMN(name, varchar<20>, (not_null)) BOOST_RDB_COLUMN(first_name, varchar<30>, _) BOOST_RDB_COLUMN(age, integer, _) BOOST_RDB_COLUMN(married, boolean, (not_null,default(false))) BOOST_RDB_END_TABLE(person) Also we will need a way of forcing the C++ type to something else than the default (currently nested inside integer, varchar<>, etc). And also the column and table names (in case they conflict with C++ keywords). Food for thought... J-L
participants (3)
-
Jean-Louis Leroy -
Joshua Pedrick -
Stefan Strasser