Proposal for Inclusion of Async.MQTT5 Library in Boost
Dear Boost Developers, we are writing to introduce a new C++ Async.MQTT5 library that we believe would be a valuable addition to Boost. MQTT has become the de-facto standard for IoT communication, with billions of IoT devices relying on it to publish or retrieve data from the cloud. Despite its widespread use, integrating MQTT into a C++ embedded application remains a complex and error-prone task. This challenge primarily arises from the absence of a higher-level library that offers a straightforward interface for message publishing and receiving while abstracting MQTT protocol intricacies and effectively managing various corner cases inherent in MQTT implementation. The primary objective behind developing Async.MQTT5 was to address these challenges comprehensively, allowing developers to focus on the messages they need to send without concerns about underlying network connectivity issues. Async.MQTT5 is a C++20 client implementation of the MQTT 5.0 protocol, leveraging Boost Asio and Boost Spirit. It fully adheres to the Asio asynchronous model, providing comprehensive support for all forms of asynchronous completion tokens, per-operation cancellation, and custom allocators. The library is designed to work seamlessly with any ordered network transport, typically TCP, SSL/TLS over TCP, or WebSockets (secured and unsecured). Our goal with Async.MQTT5 is to offer developers a simple and clear C++ interface for sending and receiving messages using the MQTT 5.0 protocol. Emphasizing code readability was a key design principle of Async.MQTT5. For instance, tedious protocol message parsing/composing is implemented declaratively (partially utilizing Boost Spirit). Additionally, Asio composed operations are implemented using a novel, clear, and highly efficient "chaining functional operators" approach. It is worth noting that the library is already written following Boost style guidelines, and the documentation is crafted and formatted in line with Boost conventions. You can find the library at https://github.com/mireo/async-mqtt5, and the documentation is available at https://spacetime.mireo.com/async-mqtt5/. We highly value your opinion and would appreciate any feedback you can provide on the Async.MQTT5 library. Thank you for your time and consideration. Best regards, Ivica Siladic Bruno Iljazovic Korina Simicevic
seconded On Thu, 30 Nov 2023 at 16:55, Vinnie Falco via Boost <boost@lists.boost.org> wrote:
On Thu, Nov 30, 2023 at 7:24 AM Ivica Siladic via Boost < boost@lists.boost.org> wrote:
we are writing to introduce a new C++ Async.MQTT5 library that we believe would be a valuable addition to Boost.
I endorse this library for review.
Thanks
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
czw., 30 lis 2023 o 16:24 Ivica Siladic via Boost <boost@lists.boost.org> napisał(a):
Dear Boost Developers,
we are writing to introduce a new C++ Async.MQTT5 library that we believe would be a valuable addition to Boost.
MQTT has become the de-facto standard for IoT communication, with billions of IoT devices relying on it to publish or retrieve data from the cloud. Despite its widespread use, integrating MQTT into a C++ embedded application remains a complex and error-prone task. This challenge primarily arises from the absence of a higher-level library that offers a straightforward interface for message publishing and receiving while abstracting MQTT protocol intricacies and effectively managing various corner cases inherent in MQTT implementation.
The primary objective behind developing Async.MQTT5 was to address these challenges comprehensively, allowing developers to focus on the messages they need to send without concerns about underlying network connectivity issues.
Async.MQTT5 is a C++20 client implementation of the MQTT 5.0 protocol, leveraging Boost Asio and Boost Spirit. It fully adheres to the Asio asynchronous model, providing comprehensive support for all forms of asynchronous completion tokens, per-operation cancellation, and custom allocators. The library is designed to work seamlessly with any ordered network transport, typically TCP, SSL/TLS over TCP, or WebSockets (secured and unsecured).
Our goal with Async.MQTT5 is to offer developers a simple and clear C++ interface for sending and receiving messages using the MQTT 5.0 protocol.
Emphasizing code readability was a key design principle of Async.MQTT5. For instance, tedious protocol message parsing/composing is implemented declaratively (partially utilizing Boost Spirit). Additionally, Asio composed operations are implemented using a novel, clear, and highly efficient "chaining functional operators" approach.
It is worth noting that the library is already written following Boost style guidelines, and the documentation is crafted and formatted in line with Boost conventions.
You can find the library at https://github.com/mireo/async-mqtt5, and the documentation is available at https://spacetime.mireo.com/async-mqtt5/.
We highly value your opinion and would appreciate any feedback you can provide on the Async.MQTT5 library.
Thank you for your time and consideration.
Thank you for submitting this proposal. But let me be the devil's advocate here. I did not look at the implementation, or documentation in detail. My concern is about "adding an ASIO-based client for a protocol". I am not familiar with MQTT 5.0 and this is the first time I have seen this name. My impression is that it is not popular, surely far less than the HTTP protocol. My question to the Boost community is, does every good library need to be included in Boost? I thought that Boost was supposed to be an "extended Standard Library", containing: 1. Portability fixes 2. Implementing missing language features 3. Vocabulary types (I consider Boost.ASIO one) 4. Frequently used components 5. Ground-breaking designs and ideas Async.MQTT5 doesn't fit into any of these categories. (Neither does Boost.AEDIS or Boost.MySql.) Do we have a clear set of criteria for what belongs to Boost? And I do not mean only the quality bar, but also the range of business domain. What's wrong with a good library hosted on its own in GitHub? Regards, &rzej;
On Fri, Dec 1, 2023 at 8:25 AM Andrzej Krzemienski via Boost <boost@lists.boost.org> wrote:
My impression is that it is not popular, surely far less than the HTTP protocol.
Nothing is going to come close to the popularity of HTTP but this MQTT is a super-annoying protocol which unfortunately has a dedicated niche for which the use-case is ideal. People invent their own MQTT libraries almost as often as they write their own JSON libraries.
My question to the Boost community is, does every good library need to be included in Boost?
In general, I would say no.
I thought that Boost was supposed to be an "extended Standard Library", containing:
1. Portability fixes 2. Implementing missing language features 3. Vocabulary types (I consider Boost.ASIO one) 4. Frequently used components 5. Ground-breaking designs and ideas
Async.MQTT5 doesn't fit into any of these categories. (Neither does Boost.AEDIS or Boost.MySql.)
I agree that Aedis, MySQL, and MQTT5 do not precisely fit your list above. However, there is a certain synergy with Asio, Beast, Aedis, MySQL, and the proposed MQTT which has merit. The reality is that users are increasingly turning away from Boost in favor of the standard library (this is a conversation for another thread). If Boost only limits itself to items one through four in your list above, the value proposition for Boost becomes unclear over time. One of the ways we might envision Boost reinventing itself is to capitalize on the one area where Boost is so far ahead that there is simply no competition: "The C++ standard can't even connect to the Internet." - Vinnie Falco Based on the papers coming out of wg21 I suspect that committee-designed networking and its accompanying replacement for Kohlhoff's executors are going to be at the same level of quality and usability as, say, ranges. Make of that what you will. In the meanwhile if we can position Boost to have a strong set of networking libraries that work together well this gives us a unique advantage. People will grudgingly download Boost despite the usual irrational complaints ("too big", "too many dependencies", "inconsistent quality") because they cannot get this set of harmonious features anywhere else. There is a group of authors and contributors working on a technology demo which integrates all of the networking libraries in Boost into several applications to showcase how the libraries might be used together to build enterprise-ready systems. This project is called "Boost ServerTech" and you can learn more about it here: BoostServerTech project proposal https://docs.google.com/document/d/1ZQrod1crs8EaNLLqSYIRMacwR3Rv0hC5l-gfL-jO... Architecture Documentation https://anarthal.github.io/servertech-chat/01-architecture.html Boost ServerTech Chat https://github.com/anarthal/servertech-chat Ruben Perez is a C++ Alliance Staff Engineer who is spearheading this project.
Do we have a clear set of criteria for what belongs to Boost? And I do not mean only the quality bar, but also the range of business domain. What's wrong with a good library hosted on its own in GitHub?
There is not a clear set of criteria, but I think that our Contributor Guide should offer some kind of guidance on "what belongs in Boost." A rigid set of rules doesn't really make sense for me. Since libraries are quite unique as well as their authors, we should evaluate each library on a case by case basis, with some general principles posted in the documentation so that newcomers to the mailing list can orient themselves. Thanks
My question to the Boost community is, does every good library need to be included in Boost?
In general, I would say no.
In contrast, I would say yes. I have always wanted to be able to compile any possible program by merely linking with a single massive library. Dependency management is hell. MFC was horrid, but at least you could write a useful program that included a UI and sound with it. The standard library will clearly never serve this role.
There was a huge onus to accept Boost.MySQL because of its licensing. libmysqlclient is GPL'd where Boost.MySQL proposed a permissive license which did mostly the exact same stuff. To that end, Boost.MySQL is a welcome addition to Boost because it essentially sets in stone a quality client implementation of the MySQL protocol without users being hampered by the GPL. The other libraries such as this? It's less so. There's often multiple libraries already being used in production that aren't as restrictively licensed which means there's less onus for accepting them. In general, I'd defer to users in this regard. If Boost users think it's useful to bundle all this up, I'd say accept but without that, I'm cautious about this wave of web-based tech hitting Boost. - Christian
pt., 1 gru 2023 o 20:30 Christian Mazakas via Boost <boost@lists.boost.org> napisał(a):
There was a huge onus to accept Boost.MySQL because of its licensing.
libmysqlclient is GPL'd where Boost.MySQL proposed a permissive license which did mostly the exact same stuff.
To that end, Boost.MySQL is a welcome addition to Boost because it essentially sets in stone a quality client implementation of the MySQL protocol without users being hampered by the GPL.
So, I understand there was a need to have a MySQL client with the Boost Software License (BSL). But it still isn't obvious that this should imply making this library part of Boost. We have good libraries on the Internet, such as SOCI <https://github.com/SOCI/soci>, distributed under BSL, and not being part of Boost.
The other libraries such as this? It's less so. There's often multiple libraries already being used in production that aren't as restrictively licensed which means there's less onus for accepting them.
In general, I'd defer to users in this regard. If Boost users think it's useful to bundle all this up, I'd say accept but without that, I'm cautious about this wave of web-based tech hitting Boost.
I am not sure how this would work. If nothing else, users' expectations are often contradictory. Different users want different things.Some users only want a distribution system, and since there isn't any for C++, they would overuse Boost. Is 10 users wanting to have a library in enough to let the library in? Regards, &rzej;
Andrzej Krzemienski: Is 10 users wanting to have a library in enough to let the library in?
My guess is that there will be quite a bit more than that. I would certainly appreciate support for MQTT, and other messaging protocols like AMQP, in boost. Best regards Espen Harlinn -----Original Message----- From: Boost <boost-bounces@lists.boost.org> On Behalf Of Andrzej Krzemienski via Boost Sent: Sunday, 3 December 2023 18.43 To: boost@lists.boost.org Cc: Andrzej Krzemienski <akrzemi1@gmail.com> Subject: Re: [boost] Proposal for Inclusion of Async.MQTT5 Library in Boost pt., 1 gru 2023 o 20:30 Christian Mazakas via Boost <boost@lists.boost.org> napisał(a):
There was a huge onus to accept Boost.MySQL because of its licensing.
libmysqlclient is GPL'd where Boost.MySQL proposed a permissive license which did mostly the exact same stuff.
To that end, Boost.MySQL is a welcome addition to Boost because it essentially sets in stone a quality client implementation of the MySQL protocol without users being hampered by the GPL.
So, I understand there was a need to have a MySQL client with the Boost Software License (BSL). But it still isn't obvious that this should imply making this library part of Boost. We have good libraries on the Internet, such as SOCI <https://github.com/SOCI/soci>, distributed under BSL, and not being part of Boost.
The other libraries such as this? It's less so. There's often multiple libraries already being used in production that aren't as restrictively licensed which means there's less onus for accepting them.
In general, I'd defer to users in this regard. If Boost users think it's useful to bundle all this up, I'd say accept but without that, I'm cautious about this wave of web-based tech hitting Boost.
I am not sure how this would work. If nothing else, users' expectations are often contradictory. Different users want different things.Some users only want a distribution system, and since there isn't any for C++, they would overuse Boost. Is 10 users wanting to have a library in enough to let the library in? Regards, &rzej; _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
pt., 1 gru 2023 o 18:42 Vinnie Falco <vinnie.falco@gmail.com> napisał(a):
My impression is that it is not popular, surely far less than the HTTP
On Fri, Dec 1, 2023 at 8:25 AM Andrzej Krzemienski via Boost <boost@lists.boost.org> wrote: protocol.
Nothing is going to come close to the popularity of HTTP but this MQTT is a super-annoying protocol which unfortunately has a dedicated niche for which the use-case is ideal. People invent their own MQTT libraries almost as often as they write their own JSON libraries.
My question to the Boost community is, does every good library need to be included in Boost?
In general, I would say no.
I thought that Boost was supposed to be an "extended Standard Library", containing:
1. Portability fixes 2. Implementing missing language features 3. Vocabulary types (I consider Boost.ASIO one) 4. Frequently used components 5. Ground-breaking designs and ideas
Async.MQTT5 doesn't fit into any of these categories. (Neither does Boost.AEDIS or Boost.MySql.)
I agree that Aedis, MySQL, and MQTT5 do not precisely fit your list above. However, there is a certain synergy with Asio, Beast, Aedis, MySQL, and the proposed MQTT which has merit. The reality is that users are increasingly turning away from Boost in favor of the standard library (this is a conversation for another thread). If Boost only limits itself to items one through four in your list above, the value proposition for Boost becomes unclear over time.
Boost used to be a testing and experimentation ground before LWG. Are you saying LWG doesn't need new libraries? Or that the community doesn't need standard-library-like libraries?
One of the ways we might envision Boost reinventing itself is to capitalize on the one area where Boost is so far ahead that there is simply no competition:
"The C++ standard can't even connect to the Internet." - Vinnie Falco
"Reinventing"? It sounds like "we are a group of people, one adventure ended, let's look for a new one". Is that what you mean?
Based on the papers coming out of wg21 I suspect that committee-designed networking and its accompanying replacement for Kohlhoff's executors are going to be at the same level of quality and usability as, say, ranges. Make of that what you will.
In the meanwhile if we can position Boost to have a strong set of networking libraries that work together well
So, are Beast, AEDIS, MySql and now this MQTT5 working well together? Do they use similar structure, naming conventions and the design? this gives us a unique
advantage.
Advantage over what? Regards, &rzej; People will grudgingly download Boost despite the usual
irrational complaints ("too big", "too many dependencies", "inconsistent quality") because they cannot get this set of harmonious features anywhere else.
There is a group of authors and contributors working on a technology demo which integrates all of the networking libraries in Boost into several applications to showcase how the libraries might be used together to build enterprise-ready systems. This project is called "Boost ServerTech" and you can learn more about it here:
BoostServerTech project proposal
https://docs.google.com/document/d/1ZQrod1crs8EaNLLqSYIRMacwR3Rv0hC5l-gfL-jO...
Architecture Documentation https://anarthal.github.io/servertech-chat/01-architecture.html
Boost ServerTech Chat https://github.com/anarthal/servertech-chat
Ruben Perez is a C++ Alliance Staff Engineer who is spearheading this project.
Do we have a clear set of criteria for what belongs to Boost? And I do
not
mean only the quality bar, but also the range of business domain. What's wrong with a good library hosted on its own in GitHub?
There is not a clear set of criteria, but I think that our Contributor Guide should offer some kind of guidance on "what belongs in Boost." A rigid set of rules doesn't really make sense for me. Since libraries are quite unique as well as their authors, we should evaluate each library on a case by case basis, with some general principles posted in the documentation so that newcomers to the mailing list can orient themselves.
Thanks
On Sun, Dec 3, 2023 at 10:12 AM Andrzej Krzemienski <akrzemi1@gmail.com> wrote:
Boost used to be a testing and experimentation ground before LWG. Are you saying LWG doesn't need new libraries? Or that the community doesn't need standard-library-like libraries?
I'm not saying either of things things. What I am saying is that
"Reinventing"? It sounds like "we are a group of people, one adventure ended, let's look for a new one". Is that what you mean?
In a sense, yes. Boost's first adventure was building world-class, peer-reviewed components destined for standardization with the accompanying field experience and bar of excellence that is Boost tradition. As authors have discovered that the bar of quality for standardization is considerably lower than that required for inclusion in the Boost library collection, Boost is no longer seen as a waypoint along the journey to standardization. So, are Beast, AEDIS, MySql and now this MQTT5 working well together? Do
they use similar structure, naming conventions and the design?
Yes, quite well in fact, this is precisely why I linked the Boost Servertech project! Did you have a look? All the authors and stakeholder are in the Slack C++ Language workspace working on this project together, and adjusting their respective libraries to achieve synergy (check out the #boost-servertech channel): https://github.com/anarthal/servertech-chat https://anarthal.github.io/servertech-chat/01-architecture.html https://docs.google.com/document/d/1ZQrod1crs8EaNLLqSYIRMacwR3Rv0hC5l-gfL-jO... Advantage over what?
An advantage over downloading individual standalone libraries from different authors, where those libraries come with no guarantee of maintenance and no peer-review. Individual libraries also do not leverage the up-front cost of installing Boost. That is to say, that if you have already integrated Boost into your project, new libraries such as Boost.Aedis or Boost.MySQL become available with no additional integration cost. Thanks
So, are Beast, AEDIS, MySql and now this MQTT5 working well together? Do
they use similar structure, naming conventions and the design?
Yes, quite well in fact, this is precisely why I linked the Boost Servertech project! Did you have a look? All the authors and stakeholder are in the Slack C++ Language workspace working on this project together, and adjusting their respective libraries to achieve synergy (check out the #boost-servertech channel):
https://github.com/anarthal/servertech-chat https://anarthal.github.io/servertech-chat/01-architecture.html https://docs.google.com/document/d/1ZQrod1crs8EaNLLqSYIRMacwR3Rv0hC5l-gfL-jO...
As Vinnie pointed out, yes, they do. They follow Boost.Asio universal async model, which guarantees compatibility between them. ServertechChat is written using Beast, Redis and MySQL (as well as many other Boost libs). Conforming to Asio's universal model is far from easy and the first thing I check during reviews like this. You will find many "Asio based" libs out there that just use Asio and don't follow the universal model.
On 01.12.23 17:25, Andrzej Krzemienski via Boost wrote:
I thought that Boost was supposed to be an "extended Standard Library",
That hasn't been the case since Boost.Python, which has been with us since Boost 1.19.
containing: 1. Portability fixes 2. Implementing missing language features 3. Vocabulary types (I consider Boost.ASIO one) 4. Frequently used components 5. Ground-breaking designs and ideas
Async.MQTT5 doesn't fit into any of these categories. (Neither does Boost.AEDIS or Boost.MySql.)
MQTT is at least an open standard with multiple implementations, which is more than can be said for Redis or MySQL or even Python. (Alternate Python implementations exist, but Boost.Python only interfaces with the official Python implementation.) -- Rainer Deyke (rainerd@eldwood.com)
On Thu, 30 Nov 2023 at 16:24, Ivica Siladic via Boost <boost@lists.boost.org> wrote:
Dear Boost Developers,
we are writing to introduce a new C++ Async.MQTT5 library that we believe would be a valuable addition to Boost.
<snip>
It would be great to have this functionality in Boost. I endorse it. Marcelo
On Thu, 30 Nov 2023 at 16:24, Ivica Siladic via Boost <boost@lists.boost.org> wrote:
Dear Boost Developers,
we are writing to introduce a new C++ Async.MQTT5 library that we believe would be a valuable addition to Boost.
<snip>
It would be great to have this functionality in Boost. I endorse it. Marcelo
participants (9)
-
Andrzej Krzemienski -
Christian Mazakas -
Espen Harlinn -
Ivica Siladic -
Marcelo Zimbres Silva -
Rainer Deyke -
Richard Hodges -
Ruben Perez -
Vinnie Falco