Proposing a MySQL client library for inclusion in Boost
Hi all, I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does). The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony. Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood. I'm looking for people who officially endorse the library. You can see the code in this GitHub repository <https://github.com/anarthal/mysql>, and the documentation is here <https://anarthal.github.io/mysql/index.html>. Many thanks, Ruben.
On Wed, 30 Mar 2022, 23:00 Ruben Perez via Boost, <boost@lists.boost.org> wrote:
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
I believe, you've found a review manager, am I right? I'm looking for people who officially endorse the library. I recall the library has already received some endorsements here https://lists.boost.org/Archives/boost//2021/03/251046.php Best regards, -- Mateusz Loskot
On Wed, 30 Mar 2022, 23:21 Mateusz Loskot via Boost, <boost@lists.boost.org> wrote:
On Wed, 30 Mar 2022, 23:00 Ruben Perez via Boost, <boost@lists.boost.org> wrote:
I would like to propose my MySQL client library for inclusion in Boost.
The
library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
I believe, you've found a review manager, am I right?
Yes, Richard Hodges had volunteered as review manager.
I'm looking for people who officially endorse the library.
I recall the library has already received some endorsements here https://lists.boost.org/Archives/boost//2021/03/251046.php
Thanks for taking your time searching for those messages. Although it was endorsed then, I decided to incorporate some improvements before the review, which took me a little more time than expected.
Best regards, -- Mateusz Loskot
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Wow, together with sqlpp11 (https://github.com/rbock/sqlpp11) this becomes a killer combination ;-) I dont know if my endorsement will help, but in any case I endorse this library. Regards, Matthijs On 30/03/2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
I'm looking for people who officially endorse the library.
You can see the code in this GitHub repository <https://github.com/anarthal/mysql>, and the documentation is here <https://anarthal.github.io/mysql/index.html>.
Many thanks, Ruben.
_______________________________________________ Unsubscribe & other changes:http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, 31 Mar 2022, 16:09 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
Wow, together with sqlpp11 (https://github.com/rbock/sqlpp11) this becomes a killer combination ;-)
Yes, this library can be used to write a sqlpp11 connector to access MySQL and MariaDB. It is a good use case.
From what I've read (I'm not an expert on sqlpp11, not even a regular user), this library doesn't support async operations, so you wouldn't be employing the full power of this library. Still, it is a fully valid use case.
I dont know if my endorsement will help, but in any case I endorse this library.
Thank you so much.
Regards, Matthijs
On 30/03/2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
I'm looking for people who officially endorse the library.
You can see the code in this GitHub repository<https://github.com/anarthal/mysql> <https://github.com/anarthal/mysql>, and the documentation is here<https://anarthal.github.io/mysql/index.html> <https://anarthal.github.io/mysql/index.html>.
Many thanks, Ruben.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 31/03/2022 16:29, Ruben Perez wrote:
On Thu, 31 Mar 2022, 16:09 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
Wow, together with sqlpp11 (https://github.com/rbock/sqlpp11) this becomes a killer combination ;-)
Yes, this library can be used to write a sqlpp11 connector to access MySQL and MariaDB. It is a good use case.
From what I've read (I'm not an expert on sqlpp11, not even a regular user), this library doesn't support async operations, so you wouldn't be employing the full power of this library. Still, it is a fully valid use case.
I am a regular user of sqlpp11, in my opinion it should be possible to support async operations.
I dont know if my endorsement will help, but in any case I endorse this library.
Thank you so much.
Regards, Matthijs
On 30/03/2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
I'm looking for people who officially endorse the library.
You can see the code in this GitHub repository <https://github.com/anarthal/mysql> <https://github.com/anarthal/mysql>, and the documentation is here <https://anarthal.github.io/mysql/index.html> <https://anarthal.github.io/mysql/index.html>.
Many thanks, Ruben.
_______________________________________________ Unsubscribe & other changes:http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, 31 Mar 2022, 16:55 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
On 31/03/2022 16:29, Ruben Perez wrote:
On Thu, 31 Mar 2022, 16:09 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
Wow, together with sqlpp11 (https://github.com/rbock/sqlpp11) this becomes a killer combination ;-)
Yes, this library can be used to write a sqlpp11 connector to access MySQL and MariaDB. It is a good use case.
From what I've read (I'm not an expert on sqlpp11, not even a regular user), this library doesn't support async operations, so you wouldn't be employing the full power of this library. Still, it is a fully valid use case.
I am a regular user of sqlpp11, in my opinion it should be possible to support async operations.
Could you please point me to the relevant documentation describing the asynchronous connector API for sqlpp11? Thanks.
I dont know if my endorsement will help, but in any case I endorse this library.
Thank you so much.
Regards, Matthijs
On 30/03/2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
I'm looking for people who officially endorse the library.
You can see the code in this GitHub repository<https://github.com/anarthal/mysql> <https://github.com/anarthal/mysql>, and the documentation is here<https://anarthal.github.io/mysql/index.html> <https://anarthal.github.io/mysql/index.html>.
Many thanks, Ruben.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 31/03/2022 17:28, Ruben Perez wrote:
On Thu, 31 Mar 2022, 16:55 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
On 31/03/2022 16:29, Ruben Perez wrote:
On Thu, 31 Mar 2022, 16:09 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
Wow, together with sqlpp11 (https://github.com/rbock/sqlpp11) this becomes a killer combination ;-)
Yes, this library can be used to write a sqlpp11 connector to access MySQL and MariaDB. It is a good use case.
From what I've read (I'm not an expert on sqlpp11, not even a regular user), this library doesn't support async operations, so you wouldn't be employing the full power of this library. Still, it is a fully valid use case.
I am a regular user of sqlpp11, in my opinion it should be possible to support async operations.
Could you please point me to the relevant documentation describing the asynchronous connector API for sqlpp11?
Thanks.
I just took a look at https://github.com/rbock/sqlpp11/tree/main/connector_api And I guess there is no async connector API for sqlpp11 yet. I'm going to take a look and see if I can come up with something nice for the integration into sqlpp11. Regards, Matthijs
I dont know if my endorsement will help, but in any case I endorse this library.
Thank you so much.
Regards, Matthijs
On 30/03/2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
I'm looking for people who officially endorse the library.
You can see the code in this GitHub repository <https://github.com/anarthal/mysql> <https://github.com/anarthal/mysql>, and the documentation is here <https://anarthal.github.io/mysql/index.html> <https://anarthal.github.io/mysql/index.html>.
Many thanks, Ruben.
_______________________________________________ Unsubscribe & other changes:http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, 31 Mar 2022, 20:32 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
On 31/03/2022 17:28, Ruben Perez wrote:
On Thu, 31 Mar 2022, 16:55 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
On 31/03/2022 16:29, Ruben Perez wrote:
On Thu, 31 Mar 2022, 16:09 Matthijs Möhlmann, <matthijs@cacholong.nl> wrote:
Wow, together with sqlpp11 (https://github.com/rbock/sqlpp11) this becomes a killer combination ;-)
Yes, this library can be used to write a sqlpp11 connector to access MySQL and MariaDB. It is a good use case.
From what I've read (I'm not an expert on sqlpp11, not even a regular user), this library doesn't support async operations, so you wouldn't be employing the full power of this library. Still, it is a fully valid use case.
I am a regular user of sqlpp11, in my opinion it should be possible to support async operations.
Could you please point me to the relevant documentation describing the asynchronous connector API for sqlpp11?
Thanks.
I just took a look at https://github.com/rbock/sqlpp11/tree/main/connector_api
And I guess there is no async connector API for sqlpp11 yet.
I'm going to take a look and see if I can come up with something nice for the integration into sqlpp11.
Please don't hesitate to message me if you have any question on the library's usage.
Regards, Matthijs
I dont know if my endorsement will help, but in any case I endorse this library.
Thank you so much.
Regards, Matthijs
On 30/03/2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
I'm looking for people who officially endorse the library.
You can see the code in this GitHub repository<https://github.com/anarthal/mysql> <https://github.com/anarthal/mysql>, and the documentation is here<https://anarthal.github.io/mysql/index.html> <https://anarthal.github.io/mysql/index.html>.
Many thanks, Ruben.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 30.03.2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
Great work! Although MariaDB is a fork of MySQL and still to a large degree is compatible, at least in the simple and most common cases, both DBMSs are adding features independently and are diverging more and more. I work at MySQL Engineering at Oracle, and I've seen other attempts to treat MySQL and MariaDB as two variations of the same product, e.g., in packaging in Linux distros and client applications. As the DBMSs diverge more and more, these attempts usually fail, and the DBMSs have to be treated as two completely different DBMSs, like MySQL and Postgres. AFAIK, the basic protocol is still the same in both DBMSs, but new extensions are implemented in one and not the other, and there is no guarantee that they will remain compatible. I wouldn't be surprised if there are already subtle protocol differences when enabling some of the most recent extensions. I believe that in the long term the protocols will diverge more. Is the library designed in a way that can handle growing differences, and maybe a complete protocol incompatibility, in the future? Best regards, Norvald
On Wed, 6 Apr 2022 at 14:58, Norvald H. Ryeng via Boost < boost@lists.boost.org> wrote:
On 30.03.2022 23:00, Ruben Perez via Boost wrote:
Hi all,
I would like to propose my MySQL client library for inclusion in Boost. The library features asynchronous communication with MySQL and MariaDB servers, allowing the user to use txt queries and prepared statements. The library is based on Boost.Asio and follows its asynchronous model (in a similar way Boost.Beast does).
The library is specific to MySQL and MariaDB systems, and is not intended as a SQL framework or ORM. It fits use cases like HTTP servers (possibly already using Beast), ETL pipelines or any other application that needs to access a MySQL database and wants to benefit from Asio's asynchrony.
Please note that the library is a full implementation of MySQL wire protocol and does not use libmysqlclient under the hood.
Great work!
Thanks :)
Although MariaDB is a fork of MySQL and still to a large degree is compatible, at least in the simple and most common cases, both DBMSs are adding features independently and are diverging more and more.
I work at MySQL Engineering at Oracle, and I've seen other attempts to treat MySQL and MariaDB as two variations of the same product, e.g., in packaging in Linux distros and client applications. As the DBMSs diverge more and more, these attempts usually fail, and the DBMSs have to be treated as two completely different DBMSs, like MySQL and Postgres.
I think your knowledge would be invaluable during the formal library review. If you have the time and the energy, we would all appreciate leaving your review, when the time comes.
AFAIK, the basic protocol is still the same in both DBMSs, but new extensions are implemented in one and not the other, and there is no guarantee that they will remain compatible. I wouldn't be surprised if there are already subtle protocol differences when enabling some of the most recent extensions.
I believe that in the long term the protocols will diverge more. Is the library designed in a way that can handle growing differences, and maybe a complete protocol incompatibility, in the future?
At this point in time, the library treats both systems the same. There are, however, the following measures in place: * In addition to unit tests, the library has a large set of integration tests, which run against real MySQL and MariaDB systems, in an attempt to catch such differences as soon as possible. I have caught a couple of these minor differences in the past, but was able to solve them in a way that made both DBMS happy (there was something MySQL accepted and MariaDB didn't). * These tests are run in CIs, which do differentiate between them (e.g. run SHA256 auth tests for MySQL 8.x but not for MariaDB or MySQL 5.x). The connection object stores a state object, holding stuff like the server capabilities, which are set after the initial handshake. These capabilities are then passed to the individual operations, to handle any possible difference between servers that may be expressed in terms of server capabilities. A couple questions: * Which kind of protocol incompatibilities would you think would be possible in the future? Are we talking about minor differences (like different auth plugins, and things that can be expressed in terms of server capabilities)? Or are we talking about a complete redesign of the MySQL protocol? * Is there any place in the documentation where I could read about these new additions we are talking about? * Do you think anyone else in MySQL server team would be interested in participating in the formal review for this library? I think their help would be much appreciated. Cheers, Ruben.
Best regards,
Norvald
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 07.04.2022 11:18, Ruben Perez wrote:
A couple questions:
* Which kind of protocol incompatibilities would you think would be possible in the future? Are we talking about minor differences (like different auth plugins, and things that can be expressed in terms of server capabilities)? Or are we talking about a complete redesign of the MySQL protocol?
I'm expecting new auth plugins and new server capabilities. New data types, perhaps. I'm not expecting a major redesign in the foreseeable future.
* Is there any place in the documentation where I could read about these new additions we are talking about?
I think the best starting point would be https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_PROTOCOL.html
* Do you think anyone else in MySQL server team would be interested in participating in the formal review for this library? I think their help would be much appreciated.
I'm not an expert on the protocol myself, so I'll try to find someone when the formal review starts. Best regards, Norvald
participants (4)
-
Mateusz Loskot -
Matthijs Möhlmann -
Norvald H. Ryeng -
Ruben Perez