- Boost - lists.preview.boost.org

boost.mustache review reminder (ends this saturday)
by Klemens Morgenstern 14 Feb '23

14 Feb '23

This is a reminder that the boost.mustache review is currently going on and will end this saturday. Please consider submitting a review.

1 0

GSoC 2023 ideas proposals
by David Bellot 13 Feb '23

13 Feb '23

Hi everyone, Boost C++ is candidating again this year to be part of the Google Summer of Code 2023. Each year we are honoured to have students and contributors joining us for the summer and being sponsored by Google. Again, this year, I will ask for all the mentors who are interested in ... well... mentoring a student or contributor, to help us and write their ideas in this document: https://github.com/boostorg/wiki/wiki/Google-Summer-of-Code:-2023 It will help our future students to choose, to develop their own proposal and to be part of our Boost journey. Best regards, David

1 0

[mustache] The docs
by Andrzej Krzemienski 09 Feb '23

09 Feb '23

Hi Everyone, I wanted to share my initial thoughts after reading the docs for the proposed Boost.Mustache. If some library docs said "I am implementing JSON", I more-less know what to expect, because JSON is popular, I used it, so one can reliably predict what the library delivers. This is not the case with Mustache. It is the first time I have heard this name. Apparently, it is a less popular thing. Therefore, I would expect some more introduction than just an external link. The linked page also doesn't contain sufficient information but further links. For instance to https://mustache.github.io/mustache.5.html, but this information also doesn't help me find answers to my questions: 1. What is considered an "optional part" of mustache? Boost.Mustache says it only implements the non-optional parts, but docs in mustache(5) do not indicate the optional parts. 2. It is clear to me that the procedure takes a template (with mustaches) and the data to be replaced. But what is the range of the acceptable formats of this data? The examples show that it is JSON, but does it have to be JSON? Boost.Mustache docs say it could be 'described' classes? But is that all? How about Boost.PropertyTree? What does the specification for Mustache say about it? 3. I do not see any specification (neither in Boost.Mustache docc, nor in the linked mustache(5)) for what is the required behavior when I have nested mustashes with the same name, or a JSON file with nested elements of the same name. Or did I miss this description? Regards, &rzej;

5 17

Boost.Buffers: network-agnostic buffer types and algorithms
by Vinnie Falco 07 Feb '23

07 Feb '23

Greetings. I am asking if there is any interest in a Boost.Buffers library, which consists of the following: * ConstBuffer, MutableBuffer concepts * ConstBuffers, MutableBuffers concepts * const_buffer, mutable_buffer types * DynamicBuffer concept * `source` and `sink` abstract interfaces used to define the Source and Sink concepts * metafunctions for checking all type requirements * implementations of common buffers and dynamic buffers: - buffers_pair - array_of_buffers - circular_buffer - dynamic_string_buffer * Algorithms: - buffer_copy - buffer_size The twist is that this library has no dependency on Asio or Boost.Asio (but Asio will still recognize them thanks to some template heroics and a splash of forward declarations). Why am I proposing this? Well, let me tell you some things... I am currently working on these two libraries: Boost.Http.Proto Boost.Http.Io The Proto library implements HTTP/1 to a fuller feature set than Beast and fixes all the design flaws in Beast (but it is completely not API compatible). This library does not do any network I/O, it is expressed purely in terms of buffer concepts (i.e. depends on Boost.Buffer and not Boost.Asio). The Io library depends on Asio and the Proto library to provide the network algorithms which use Asio to implement HTTP/1. This is the equivalent HTTP functionality of what Beast offers today. Websockets is not included but when I am done with Http I will redo websockets with improvements and new features, and similarly they will be developed as two libraries: Boost.Websockets.Proto Boost.Websockets.Io My motivation for splitting libraries this way is as follows: * Each library is smaller and compiles more quickly * CI turnarounds are faster * Separation of concerns The HTTP Protocol library is designed in a way that much of an application's HTTP business logic can be expressed independently of Asio using only the constructs found in Http.Proto. This especially means fewer templates and faster compilation because Http.Proto is designed from the ground up to avoid templates whenever possible. However during the development of Http.Proto I find myself having to reach for the same buffer algorithms and containers that I developed for Beast (such as the circular buffer and the dynamic buffer). So Http.Proto has been steadily growing these things: <https://github.com/CPPAlliance/http_proto/blob/a3a284f06597ad12ca6cc1782beb…> Extract this independent buffer functionality into a separate library Boost.Buffers would bring these benefits: * Again a smaller library, faster compilation * Faster CI turnarounds * Separation of concerns Additionally, algorithms can be expressed in terms of asio-independent buffer types and concepts. For example, Boost.JSON could implement the boost::buffers::sink and boost::buffers::source interfaces which would enable its use in Boost.Http.Proto and Boost.Requests (an upcoming HTTP client library from Klemens), without having to depend on either of those libraries or Asio. Is this style of development of smaller, fine grained libraries that separate concerns something that is desirable for Boost? Glad for any feedback. -- Regards, Vinnie Follow me on GitHub: https://github.com/vinniefalco

11 29

Interest in cpp-member-accessor?
by Hans Dembinski 07 Feb '23

07 Feb '23

Dear all, Hubert Liberacki wrote this library https://github.com/hliberacki/cpp-member-accessor that uses a loophole in the C++ standard which allows one to access private members and methods in classes in external libraries that are not under the control of the library author. Since the loophole is legal, it works with all major compilers and on all platforms. No special compiler flags required nor illegal macros like #define private public. I think this should be integrated in Boost, is there interest other from myself? Before you say it is evil, consider all the good that can be done with this in responsible hands. You can add serialisation support to classes that you do not control and you can write efficient Python bindings. I have used this library now in three projects. I used it to add serialisation support and to provide memory views of C++ structures in Python (via the Numpy module), both wouldn't have been possible to do or not would not have been efficient without this. It is a single-header library with 85 lines of code, some of which are comments. This seems a bit small for a separate library, but I am not sure where it would fit in. Best regards, Hans

3 4

[ublas][odeint] PR merge questions
by Antony Polukhin 06 Feb '23

06 Feb '23

Hi, 1) What is the status of adding CMakeLists.txt to ublas and odeint libraries? That file are essential for using Boost in big companies. Especially it is crucial for integrating Boost into buld systwms other that b2. I see there is a https://github.com/boostorg/ublas/pull/142 but no actions for a year. Please, just merge it. 2) Please, merge the fix https://github.com/boostorg/ublas/pull/153 3) Please, merge the fix https://github.com/boostorg/ublas/pull/153 There are many other PR waiting for merge. I do realise that the maintainers could be short on time. May be there are some developers that you trust and write permission could be given to them?

1 0

[gsoc] GSoC mentors
by David Bellot 06 Feb '23

06 Feb '23

Hi Boost mentors for GSoC 2023, If you're interested in mentoring for the Google Summer of Code 2023, please contact me now by email and if you haven't done it already, please start writing your ideas and proposals. The links on github are provided below. The GSoC overview page at Boost is : https://github.com/boostorg/wiki/wiki/Google-Summer-of-Code%3A-Overview The ideas page is here too : https://github.com/boostorg/wiki/wiki/Google-Summer-of-Code:-2023 It's the one you should start filling in. The git repository for your future students and contributors is here: https://github.com/BoostGSoC23 Welcome aboard. Best, David

1 0

The C++ Alliance
by Louis Tatta 03 Feb '23

03 Feb '23

Hello, I am writing to update you on the recent activities of The C++ Alliance. Our newest team members include Matt Borland, Ruben Perez. Matt is developing Boost.Charconv, a C++11 library which polyfills the equivalent C++17 functionality, and contributing to the maintenance and improvement of Boost.Math. Ruben is making significant improvements to Boost.MySQL. Our long-running Boost website project continues to progress and will be deployed in beta this year. As some authors and maintainers may have observed, the code coverage services provided by https://codecov.io have been varying greatly in quality in terms of the user experience of viewing the reports, with the experience being typically worse. To mitigate these risks we are researching the development of a new set of HTML, CSS, and JS templates for presenting high quality coverage reports in the original style. In our continuous efforts to improve documentation quality we have contracted HDoc to explore generating Doxygen compatible XML output so that we might move away from Doxygen in favor of HDoc. This tool is focused specifically on C++ and could be a better fit for Boost libraries that currently use Doxygen. Our vision for documentation includes a significant investment in Asciidoc as the choice of markdown. This means rendering Doxygen/HDoc output in Asciidoc and commissioning a great set of stylesheets for Asciidoc to replace the stock sheets. In the longer term we are investigating Antora as a solution for producing documentation that is well integrated into the new website, to offer a better navigation experience, improved accessibility, more features such as C++ syntax highlighting and code snippet interactivity, and others. We are excited to work on these developments, with the hope they will add vitality and enhance Boost. We look forward to sharing more updates with you in the near future. Best regards Louis Tatta CEO The C++ Alliance

3 2

Two Container Hash libraries on the website
by Andrey Semashev 01 Feb '23

01 Feb '23

Hi, For some reason, Container Hash is listed twice on this page: https://www.boost.org/doc/libs/1_81_0/ The two entries seem equivalent except the "C++ Standard Minimum Level" entry.

2 2

Decommissioning Trac (svn.boost.org)
by Glen Fernandes 31 Jan '23

31 Jan '23

We have Trac in read-only mode after we migrated to Git, but we're still paying for the server. Do people still want to preserve any part of the content in some way? The wiki aside, there are old issues, with comments, and some of them have patches attached. Peter had the idea of generating static HTML content from Trac and hosting that on an archive section on boost.org. (I don't know how difficult this is to do). Thoughts? Suggestions? Glen

5 7

Boost TCP socket read/write issues
by Rahul K R 31 Jan '23

31 Jan '23

Hello All, As part of my project activity, I have to create an asynchronous TCP server application using the boost library. I have created a sample application and it's working. But the issue I am facing is the boost::asio::read API takes a long time to read from the TCP socket, almost 5 seconds. I am using boost library version 1.76.0. I have the following queries regarding this, 1. Is there any known issues for boost::asio::read to consume more time? 2. I am doing reading and writing to the same socket from two threads parallelly, Is that possible with the boost? 3. Is the boost lib can perform well if we get too many requests at a time and process parallelly? I am attaching the sample application code herewith. Please let me know your feedback asap. Regards Rahul

2 1

GSoC 2023 (Google Summer of Code)
by David Bellot 31 Jan '23

31 Jan '23

Hi Boost community, the GSoC 2023 organisation application is now open. Boost is again a candidate this year. We're just waiting for the results to see if we're accepted. In the meantime, we are "recruiting" mentors. Some of you already contacted me a few weeks ago, but maybe some others would be interested to join this fascinating journey. Please contact me now, by mailing list or private email if you're interested. Best, David

2 1

Building the proposed Boost.Mustache
by Peter Dimov 30 Jan '23

30 Jan '23

It's been pointed out that the proposed Boost.Mustache https://github.com/pdimov/mustache (whose formal review is upcoming) is relatively hard to evaluate, because the contents of the repository are in a form that is intended for inclusion into a Boost release. Here's how to build the library, starting from scratch: 1. Clone Boost from Github: git clone --recurse-submodules https://github.com/boostorg/boost 2. Clone the Mustache library into the Boost tree: cd boost/libs git clone https://github.com/pdimov/mustache cd .. 3. Build b2: ./bootstrap.sh (We assume Linux and g++ from now on. On Windows, it's just `booststrap`.) 4. Generate the header links in the boost/ directory: ./b2 headers 5. Build the Mustache library and its dependencies (e.g. JSON): ./b2 --with-mustache This will place the compiled libraries into the stage/lib directory. It should contain libboost_container.a/.so, libboost_json.a/.so, and liboost_mustache.a/.so. You can now build and run the Mustache examples by using e.g. g++ libs/mustache/example/html.cpp -o html -I . -L stage/lib -lboost_mustache -lboost_json ./html

1 1

aedis is ACCEPTED into boost with conditions
by Klemens Morgenstern 28 Jan '23

28 Jan '23

Boost.aedis is ACCEPTED with conditions. Accept: 3 ( + 1 late one) Conditional: 1 Reject: 4 All but one rejections asked for changes that were not fundamental to the design, but rather were about particular implementation details or high-level API design issues. Thus I conclude, based on the reviews, that there is no need for another mini-review. This is also based on Marcelo's engagement, so I am confident that many issues will be addressed in the future . These are the conditions - name it boost.redis - clearly document/specify the behaviour of async_exec, async_receive & async_run, how they interact and all the edge cases (and unit test them) - unit tests for error handling, especially for pipes. - add a design discussion & FAQ page. That should mostly copy & paste from the review discussions. - from_bulk & to_bulk should either us tag_invoke or be be renamed to aedis_from_bulk or aedis_to_bulk - remove memory_resource usage and optimize the allocation strategy of a request object (using new/delete), so that it ideally allocates once and then reuses the memory Please note that the inclusion into boost does not mean that it has to be shipped with the next release. I also strongly recommend development of typed commands, possibly with code generation. This is however not a condition and does not need to be in the first release that's included in boost. Another recommendation is to add some very basic explanation of redis, as some users will read your doc before they look into redis. I want to thank everyone who submitted reviews, especially Ruben who really drilled down on some details and brought in his experience from MySQL. Thank you Zach for getting the discussion started on type-safe command access; based on the discussions I am confident that some feature like that will be added on top of the existing APIs. Similarly I want to thank Дмитрий, Sam & Alan, who's review helped me to formulate the conditions. I am also grateful for (who I think are) first time reviewers, Mohammed Nejati, Robert Leahy, Sam Hartsfield & Miguel, and of course our boost veterans Vinnie & Christian. Likewise I want to thank Marcelo for all his responses to reviews and questions and of course for all the work that went into developing aedis. Congratulations Marcelo!

5 6

[Aedis] Formal Review: We could have done better
by Vinnie Falco 26 Jan '23

26 Jan '23

In hindsight, I feel like perhaps we collectively failed the review process. I think that Boost.Aedis was in fact not ready for a review and it did a disservice both to the library and the author that we rushed to review it. Instead, the relevant subject matter experts such as I, Alan de Freitas, Klemens Morgenstern, Reuben Perez, to a lesser extent Christian Mazakas, Mohammed, and a few others should have taken Aedis under our wing and helped to "get it into shape" before the review. Speaking for myself I wish I had put more time into helping the author get the library into the best possible condition. Instead of waiting until the review, I should have helped Marcelo address the flaws ahead of time. For example: <https://github.com/mzimbres/aedis/issues/51> and <https://github.com/mzimbres/aedis/issues/47> The library should not have gone into the review without one or more revisions to the documentation so that common questions that came up during the review were already answered, perhaps in a FAQ section. In particular, reviewers should not have had to ask why async_run or async_exec (see my example exposition in issue 51 above). I also think that the library would have benefited from a concerted effort to get it in the hands of users. I spent a bunch of energy ahead of time to get users for Beast before submitting it for review. It was already deployed in production systems, and there were other projects that started integrating it both open source and closed. We should have at the very least advertised Aedis in places like the redis++ repository (perhaps with a new GitHub issue). Forums, and such. And looked for individual C++ projects already using Redis and informed them of Aedis. For any new libraries we should look at the lesson of Aedis and consider whether or not a similar treatment would result in a better review of the candidate library. Regardless, I have the highest regard for Marcelo as he demonstrates a mastery of Asio and a commitment to quality, even if the library's outcome of the review is not favorable. -- Regards, Vinnie Follow me on GitHub: https://github.com/vinniefalco

4 6

Fwd: aedis review starts today
by Vinnie Falco 25 Jan '23

25 Jan '23

The following review came in after the review period ended. I am reposting it with permission. A copy was also sent by the reviewer to the review manager. ---------- Forwarded message --------- From: M P <miguelportilla64(a)gmail.com> Date: Wed, Jan 25, 2023 at 11:14 AM Subject: RE: [boost] aedis review starts today To: <klemensdavidmorgenstern(a)gmail.com>, Vinnie Falco <vinnie.falco(a)gmail.com> > Please explicitly state that you either *accept* or *reject* the inclusion of this library into boost. I ACCEPT this library into boost. My only concern is that although Redis may be the flavor of the day, project popularities fade. If Redis is supplanted, I wonder how effective boost maintainers down the road will be to remove it. > Also please indicate the time & effort spent on the evaluation and give the reasons for your decision. > Additionally, stating your knowledge of redis and asio would be helpful for Me. I had no prior knowledge of Redis before reviewing Aedis, but I am familiar with Boost and Asio. Before diving into Aedis, I began by studying Redis, its RESP protocol, and real-world use to help me understand why the Aedis project might be useful to the c++ community. It quickly became apparent that Redis is very popular. It is the most popular database used on Amazon Web Services, the world’s largest cloud service provider. It is also ranked first in its category and the top ten of all categories by DB-Engines. In today’s full-stack web applications, users expect responsiveness, and Redis is an important component in scaling architecture to meet demands. Aedis grants c++ developers Redis access, empowering their software to participate in this evolving sphere of technologies. I spent about three days learning about Redis, RESP3, and Aedis. While evaluating Aedis, I built the project in Linux and Windows and performed tests using Visual Studio 2022. The project examples provide a variety of scenarios that may be useful as starting points for folks with different requirements. > - Does this library bring real benefit to C++ developers for real world use-case? Absolutely. Any c++ project with heavy utilization of Relational Databases can benefit from the cache performance Redis provides. Projects that do use a Relational Database may also be able to use Redis as a mechanism to share data across servers. There are many real-world use cases ranging from web, multi-user games, fintech, and blockchain. > - Do you have an application for this library? I do not have a software application at the moment that uses Redis. I am working on a project that might end up using Redis and if so, would likely use Aedis. > - Does the API match with current best practices? Yes, the API is well-matched with current practices. Aedis is based on Asio and it does a good job of keeping in line with it. > - Is the documentation helpful and clear? The documentation within the sources (Doxygen) is helpful. The project readme.md file can be improved. Having steps to build the project for various operating systems would be nice. Providing cmake examples to generate project solutions for Visual Studio might be a good idea. Eg. cmake -G "Visual Studio 17 2022" -A x64 -B bin64 -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake Although seemingly trivial to some, it helps reduce friction (and frustration) to newcomers and increase adoption. > - Did you try to use it? What problems or surprises did you encounter? Yes, I built the included examples and tested them with a Redis server. It may not be obvious to new users that most functions can throw an exception. Some exception messages should be elaborated. For instance, upon a failure to a server that refused a connection, the exception message was "multiple exceptions". Small nits… May want to consider renaming the function 'healthy_checker' to 'health_check'. Rename the abbreviated response variable 'resp' in the examples to `response`. `resp` can be confusing in this context considering RESP is an acronym for the communication protocol. It might be helpful to use an enumeration (or something else) for request commands. Not only would it provide type safety but it also aids in knowing the commands without having to reference them in the docs. > - What is your evaluation of the implementation? Overall I believe Aedis is well-designed and written, Marcelo has done an excellent job. Unfortunately, this library requires c++ 17 with an emphasis on c++ 20. This likely eliminates a large population of c++ users. Best, Mickey -- Regards, Vinnie Follow me on GitHub: https://github.com/vinniefalco

1 0

aedis review period has ended
by Klemens Morgenstern 25 Jan '23

25 Jan '23

The review for aedis has concluded. I will announce the result soon. Thanks to everyone who participated in the review and discussion.

1 0

aedis review starts today
by Klemens Morgenstern 24 Jan '23

24 Jan '23

The formal review of the aedis starts today (15th) and ends next week on Tuesday (24th). The library was developed & submitted by Marcelo. Aedis ================= Aedis is a Redis client library built on top of Boost.Asio that implements the latest version of the Redis communication protocol RESP3. Some of its distinctive features are * A connection class that provides high-level functions to execute Redis commands and receive server pushes concurrently. * Automatic pipelining of commands. * Support for STL containers and serialization of user-defined data types. * Low-latency: Responses can be read in their final data-structure without creating copies. Aedis links Github: https://github.com/mzimbres/aedis Homepage: https://mzimbres.github.io/aedis/ Git tag: v1.4.1 Tarball: https://github.com/mzimbres/aedis/archive/refs/tags/v1.4.1.tar.gz Redis Links: Redis: https://redis.io/ RESP3: https://github.com/redis/redis-specifications/blob/master/protocol/RESP3.md Pipelines: https://redis.io/docs/manual/pipelining/ Redis commands: https://redis.io/commands/ Local Redis =========== Redis can be also easily installed on your local machine, for example, on Debian run "apt install redis". You might also want to use docker https://www.docker.com/blog/how-to-use-the-redis-docker-official-image/ Redis server for the Boost review ================================= We also have a public aedis server available for testing, please contact me or Marcelo for the details. Review ======== Please explicitly state that you either *accept* or *reject* the inclusion of this library into boost. Also please indicate the time & effort spent on the evaluation and give the reasons for your decision. The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Some questions to consider: - Does this library bring real benefit to C++ developers for real world use-case? - Do you have an application for this library? - Does the API match with current best practices? - Is the documentation helpful and clear? - Did you try to use it? What problems or surprises did you encounter? - What is your evaluation of the implementation? Additionally, stating your knowledge of redis and asio would be helpful for me. More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html Reviews can also be submitted privately to me, and I will not disclose who sent them. I might however cite passages from those reviews in my final decision. Thanks to Marcelo for providing us with a new library & thanks for all the reviews in advance.

13 43

Fwd: Aedis review (Mohammad Nejati)
by Klemens Morgenstern 24 Jan '23

24 Jan '23

---------- Forwarded message --------- From: Mohammad Nejati [Ashtum] <ashtumashtum(a)gmail.com> Date: Tue, Jan 24, 2023 at 7:02 PM Subject: Aedis review (Mohammad Nejati) To: klemensdavidmorgenstern(a)gmail.com <klemensdavidmorgenstern(a)gmail.com> First I want to say thank you to Marcelo for the effort he put into creating such a library and improving the ecosystem around Asio. >Does this library bring real benefit to C++ developers for real world use-case? Yes, Redis is extensively used in server-side services, so having a high-quality asynchronous C++ library is a need for this type of application. >Do you have an application for this library? Yes, I work on backend services and use Redis as a key-value storage and a message broker (Redis streams) and I am in need of an asynchronous Redis library. >Does the API match with current best practices? >Did you try to use it? What problems or surprises did you encounter? Yes, I am a user of Aedis already and have used it in an application too. There are some issues that I have found in the practical usage of this library, some of them are related to the API and some of them have performance implications. The connection being reconnectable seems a design flaw: The idea behind having a reconnectable connection seems to be there to make the user's life easier (by keeping the user's request and retrying them on reconnect) but there are some issues: A network disconnect leaves the user's request in an undefined state (they might have applied on the server or not) we can't simply retry them after a reconnection it requires the user's intervention to handle a reconnect. For example, we can't simply retry an INCRBY command, it might increment the value twice. Because of the possibility of reconnecting the handshake process has to be done by users (by sending HELLO command as the first request) and on a reconnect this should be repeated again, this affected the implementation in a way it has to check for each command to see if it is a HELLO command or not. If a user is using async_receive there is no way to detect a disconnect, users have to use some form of async condition variable (which Asio doesn't have at the moment) to signal the disconnect event. On a reconnect all Redis subscriptions are dropped and again a user has a hard time signaling the async_receive part of the code to subscribe again. Because of the possibility of reconnecting, each request gets a lot of boolean config parameters to handle the reconnect scenario (cancel_on_connection_lost, cancel_if_not_connected, cancel_if_unresponded, hello_with_priority) that made the API difficult to use and understand. So if all these efforts have a limited use case (implicitly retrying the requests that are read-only or do not matter if executed twice) I think a user can handle this scenario with a custom wrapper type would be better instead of making the whole connection reconnectable. There are a lot of rescheduling for reading a single push event (performance issue): At the moment the demultiplexing of Redis events seems to happen with a strategy https://github.com/mzimbres/aedis/blob/master/include/aedis/detail/guarded_… The problem is that for reading each event message around 4 rescheduling on executor happens (it might be less or more than 4 it is hard to be sure from the code) and that has degraded the performance of reading push events (rescheduling on Asio executors is not a cheap operation). There is no possibility of reading push events in batches (performance issue): Redis pub/sub receives its messages as push events and because it is a way of broadcasting messages it is sometimes used at rates like 50,000 messages per second, with the current API there is no possibility for reading these messages in batches which made the reading operation very costly. Creating an Asio timer for each request (performance issue): Currently, Aedis creates an Asio timer for each request in the queue to act as an async condition variable, but it seems it can be eliminated if the exec_op is converted into a polymorphic type which can be resumed externally, after this change would a need for an Asio service too, to destroy paused operation on a service shutdown. >Is the documentation helpful and clear? I find them very clear and well written but it would be nicer to have more examples of real-world usages of Redis, I see documentation links to the example directory for examples. It would be nice if add a couple of them to the document and explain what is going on. There is no example of how we should use this library to work with Redis streams. The echo server benchmark doesn't carry any information and is misleading in my opinion, if there is some benchmark it should be a practical usage example. >Please indicate the time & effort spent on the evaluation and give the reasons for your decision. Because I had some previous experience with this library I think in total it should be around 4 days of debugging and evaluation. >Please explicitly state that you either *accept* or *reject* the inclusion of this library into boost I think this library needs to see more practical usage to find its shape in terms of the quality of implementation and the quality of the provided user interface. There are various modes we can use Redis (key-value storage, message broker, pub/sub) and each one can reveal shortages in the performance or API. At this stage, I think it is too early for inclusion in the boost.

2 1

Aedis review is about to end
by Klemens Morgenstern 24 Jan '23

24 Jan '23

This is a reminder that later today (in about 12h) the review of aedis will conclude. If you intend to submit a review, please do it soon or let me know if you need more time.

1 0

Fwd: Aedis Review
by Klemens Morgenstern 24 Jan '23

24 Jan '23

This review was sent directly to me. ---------- Forwarded message --------- From: Robert A.H. Leahy <rleahy(a)rleahy.ca> Date: Tue, Jan 24, 2023 at 1:04 PM Subject: Aedis Review To: klemensdavidmorgenstern(a)gmail.com <klemensdavidmorgenstern(a)gmail.com> Sending directly as I'm not on the Boost mailing list. *Knowledge of Redis* No more than can be gathered by Wikipedia. *Does this library bring real benefit to C++ developers for real world use-case?* Yes. Having an idiomatic (i.e. Asio-like) way to interact with something in C++ brings real benefit. *Do you have an application for this library?* Not immediately but Redis is in use within my company so I can see one arising in the future. *Does the API match with current best practices?* I share Zach Laine's concerns about raw strings (status quo) vs. a more structured way to pass data to the API. However I'm sympathetic to arguments that: - Redis-like products may extend the set of commands and that's important to support - Redis itself may extend the set of commands and it'd be a shame to force users to wait for a library upgrade (or worse a storm of forks) to get support for new features Additionally I see the request for a more structured way to pass data to the API as a pure extension: What exists right now is a good lower layer that something higher level can be built on top of. It's not one or the other, in a perfect world both would exist. I think the API should be more generic, for example: - Why is std::tuple used rather than supporting anything which supports the tuple protocol? - std::pmr::string is used internally by aedis::request but the documentation suggests that for deserialization purposes only std::string is supported (along with only std::vector et cetera) - Why the concrete reliance on aedis::resp3::request (rather than any type which obeys some protocol)? I don't think that aedis::basic_connection::reset_stream should exist: It marries the class too heavily to TCP-like streams and doesn't add anything beyond being a convenience wrapper for several functions of the underlying stream. If anything it should be a free function. Note that the above concern is mirrored somewhat in several parts of the code where members such as is_open are used. Several opportunities to add noexcept (conditional or otherwise) should be taken. I don't see any reason why detail::connection_base::get_executor shouldn't be noexcept, for example. I feel as though read and write buffers should be customizable rather than being married to std::pmr::string (detail/connection_base.hpp). If the implementation is going to maintain its own buffers I think there should be a way to inspect them for post mortems (e.g. generating clear and context-rich error messages). *Is the documentation helpful and clear?* Yes, although some things could be made clearer. It was unclear to me at first glance what it mean to "[r]eset[] the underlying stream" (documentation for aedis::basic_connection::reset_stream), for example. *Did you try to use it?* I did not. *What is your evaluation of the implementation?* I'd rather see operator[] (possibly combined with an assert) rather than at(). I dislike the AEDIS_CHECK_OP# macros (by virtue of the fact that they're macros and, in my opinion, substantially reduce the readability of the code at the point where they're used). In detail/connection_ops.hpp:271 the operation appears to complete sucessfully because stopping "was requested from the outside." My first instinct is that in this situation the operation should end with operation_canceled. *Please explicitly state that you either accept or reject the inclusion of this library into Boost* Accept. I don't think any of the concerns I've raised are breaking and so they can be addressed later. Thanks, --Robert

1 0

Fwd: aedis review starts today
by Ruben Perez 23 Jan '23

23 Jan '23

Forwarding to the whole ML, I hit the wrong button. Thanks Marcelo for noting it. ---------- Forwarded message --------- From: Ruben Perez <rubenperez038(a)gmail.com> Date: Mon, 23 Jan 2023 at 14:04 Subject: Re: [boost] aedis review starts today To: Marcelo Zimbres Silva <mzimbres(a)gmail.com> Hi Marcelo, I'll try to answer to the several points you made in a single email. > > To fix this properly I would need a tool similar to what protobuff is > to grpc: generate requests and responses stub from a schema. > > Aedis however is not trying to do that, it provides the infrastructure > with which such a tool could be created. Perhaps there is no place in > Boost for such a library, but it is still necessary. I do think there is a place in Boost for a library like this. I don't think we need neither a 100% type-safe solution, nor to completely hide the general runtime interface that request::push() gives us. But I think we do have some usability barrier here, and you need to have a plan on how to evolve it to make it better, before the library gets into Boost and you loose the ability to introduce big breaking changes. I don't think the type-safe solution should span *all* Redis commands, but only the most common ones. If the command the user wants does not have type-safety implemented, she can always use what it's in place now. Simple interfaces for simple uses, complex ones for advanced use cases (that's something I learnt in MySQL the bad way). > > As an example of this, the library's own example cpp20_containers, > > in the hgetall function, uses the type std::map<std::string, > > std::string> for the hgetall response, which will be okay unless the > > key is not set, in which case you will get an error. I think this > > has an enormous potential for causing hidden bugs (and even > > vulnerabilities). I've also noticed that if you mismatch the sizes > > of the pipeline request and response, you get an assertion (so UB). > > To avoid this you can wrap the response type in std::optional, for example > > std::optional<std::map<std::string, std::string>> > > I don't lock users in this behaviour because they might not want it. > Perhaps it is desirable to have an error instead of having to check > whether the optional has a value. The user must know what he needs. Hand't think of that. Although I don't have data to speak, I get the impression that this feature will be used unintentionally more than intentionally. But it might just be an opinion. > As an alternative I could lock-in users on resp3 types: maps, sets, > etc. But then how to represent data structures that have no > corresponding C++ types. It would also be inefficient as it would > require first copying and then converting. That's something I didn't take into account when I wrote the review, and that gets solving this harder. Is there a way, given a type T, to know whether the response to a given command (say "GET") is compatible with T? Without performing the actual parsing? I can think of the following solutions (excuse me if any of these have flaws I haven't taken into account): 1) A runtime checking option. For the most common commands, you provide a set of helper functions like request::push_set or request::push_hgetall(). request remembers which commands have been added. The deserializing function checks for type compatibility before proceeding to the actual deserialization. This would pick the std::optional<T> vs T mismatch. Not sure if it's worth paying the price, though. You could provide an option to disable this at runtime. 2) A compile-time checking option. I envision this as something like typed_request<commands::get, commands::hgetall, ...>. You can check for compatibility at compile-time. You will probably want something like commands::any to provide access to commands that don't have a type-safe interface. Note that I don't think this should replace aedis::request, but complement it. Do you find any of these solutions appealing? Knowing that a strategy is in-place to evolve the library towards type-safety would alleviate many of my concerns. > I am trying to understand how you avoid these risks in Boost.MySql but > it seems it suffers from the same problems? > > I mean this: https://github.com/boostorg/mysql/blob/ac7285c62163c0de40a466ecaeca13f7de5a… > > conn.query("SELECT salary FROM employee WHERE first_name = > 'Underpaid'", result); > double salary = result.rows().at(0).at(0).as_double(); > > The query is not type safe [1] and the result can mismatch the the > query. Why would it be considered ok for Boost.MySql and not > Boost.Aedis? > > Disclaimer: I am not trying to make a case or open a precedent, it is > purely technical. I might be wrong but I would like to clarify anyway. So we have two parts here. The "client -> server" interface has two flavours: connection::query(string_view), and statement::execute(const ParamsTuple&). query() should only be used for queries that don't have parameters provided by the user (I should change that example, BTW). statement::execute takes a tuple of parameters. The client will check for compatibility in terms of number of parameters. Types are not checked in the client because the server is *very* flexible with type conversions. The second part is the "server -> client" interface (mysql::resultset), which is the one I think you're referring to here. It's a purely variant-like interface. Still, there is code to runtime-check that what the server sent matches the query metadata, which is provided by the server. I view this variant-like interface similar to Boost.JSON parsing into a json::value. I'd say the differences I see from the Aedis case are: * There is no way for the MySQL client to know the resulting data structure from a query given the contents of a query. "SELECT product FROM products" can yield 10 different data types, and without the table definition, you can't know. I got the impression this is not the case in Redis - "HGETALL" always returns the same data structure (please correct me if I'm wrong). * From a user perspective, when I use a variant-like interface (like the one in Boost.JSON or in Python database access), my head is set to "type checking is my responsibility". I know I'm in runtime. When I use a compile-time interface, I get much more relaxed. I got the "I'm at compile-time safety" with Aedis, but that impression is not right. Again, this is just based on what I felt while using the library for the first time, other users might feel differently. I think if I had encountered an aedis::value, similar to json::value, I would have switched my mind to "I'm at runtime". I'm not saying this is the solution, I'm just trying to explain myself. * I still got criticism for this in MySQL, and I've got a safer compile-time interface planned (https://github.com/boostorg/mysql/issues/60) The idea is that you define a row type T, so you can then read a typed resultset<T> (or individual rows by calling connection::read_one_row(T&)). My idea is to fully check for type compatibility as soon as I've got the query metadata available - that would be similar to solution 1) I proposed you earlier in this email. * If you're referring to the ability to compose SQL queries in a type-safe way, as sqlpp11 does, I did determine this as out of scope. As SQL is a standard, you usually get this at the database compatibility layer, rather than in individual drivers. I see it as more involved than providing type-safety to Redis commands - please apologies if this statement is not correct. > > This is to make it safe by default and in part because Aedis is > centered around the adapter. In the same sense that you can write an > adapter to receive responses in the data-structure that makes most > sense to the application e.g. std::map, std::unordered_map, > boost::flat_map etc. You can also customize the way errors are > handled. For example, > > * Errors like type mismatch are usually unrecoverable and you might > want to fail fast and let the async_ operation itself fail instead of > proceeding in an unknown state. > > * If you can tolerate errors like that you can write an adapter that > does not communicate an error to the parser. I did not make this the > default because it looks unsafe, people will usually expect > async_exec(req, adapt(resp)) to throw when the response can't be > stored in resp. > > I am perhaps at fault here for not documenting this feature. I thought > however not many people would want this behaviour. I think you're right and that this is the most common case. I had a look at the Python redis driver, and they do the following: * By default, fail the operation if any of the pipeline requests failed. * If you pass a certain config option, individual errors are reported as error response objects. So if you sent [C1, C2, C3] and C2 failed, you will get [C1_response, C2_error_info, C3_response]. I think your default is right. Maybe you could provide the option to report errors individually by having the user provide a) std::tuple<leaf::result<T1>, leaf::result<T2>> Instead of b) std::tuple<T1, T2> If the user did b), then do the same you're currently doing, otherwise report errors individually. I would like to know your thoughts on these two matters (type-safety and error handling), as having a plan would alleviate many of my concerns. Regards, Ruben.

2 1

Regarding password protected file
by Daggumati, Rajesh 23 Jan '23

23 Jan '23

Hi Team, We have a project requirement where we store the logs in .csv file and that .csv file should be password protected. It means that User can enter the password to open the .csv file. do we have any library to create a password to open a .csv file in boost? Please let me know your thoughts on above query. Regards, Rajesh D Confidential - Company Proprietary Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.

3 3

boost.aedis ongoing review reminder
by Klemens Morgenstern 20 Jan '23

20 Jan '23

The review period for boost.aedis is half way done, please consider submitting a review!

1 0

Re: [boost] aedis review starts today
by Zach Laine 18 Jan '23

18 Jan '23

As Marcelo pointed out, I did not Reply All; here is the email: On Tue, Jan 17, 2023 at 3:40 PM Zach Laine <whatwasthataddress(a)gmail.com> wrote: > > On Tue, Jan 17, 2023 at 3:11 PM Marcelo Zimbres Silva > <mzimbres(a)gmail.com> wrote: > > > > On Tue, 17 Jan 2023 at 20:39, Zach Laine via Boost > > <boost(a)lists.boost.org> wrote: > > > > > > No. In particular, the API requires the user to write lots of Redis > > > strings by hand. A typo will not be detected until runtime. The > > > set of Redis keywords is fixed; there's no reason not to make an > > > enumeration of them all, like this: > > > > I had enums in the past but I had to remove them for a couple of reasons > > > > 1. There are a multitude of possible commands in the Redis stack, for example > > > > * Redis search: https://redis.io/commands/?group=search > > * Redis graph: https://redis.io/commands/?group=graph > > * Redis time series: https://redis.io/commands/?group=timeseries > > * Redis json: https://redis.io/commands/?group=json > > * Perhaps more I might be missing righ now. > > I don't yet see the problem. In the enumeration I made, there's a > JSON_DEBUG (JSON.DEBUG from https://redis.io/commands/?group=json, but > with s/./_/), etc. None of these enumerators collides with any other; > I compiled the resulting enumeration to check exactly this. Am I > missing something? > > > Then, we would probably agree that the command syntax is equally > > important. To check it properly I would have to write a generator that > > parses the description of each command and generates a checker, see > > https://github.com/redis/redis/tree/unstable/src/commands. > > > > I have made no thoughts so far about how that would work and if I get > > that correctly I would still have to watch for versioning (what > > changes from release to release), what new commands were implemented > > and which were deprecated. > > Yep. That's the job. The good news is that a new version need not be > that disruptive. You only ever need to add to the enumeration, and > you can add a single deprecation-detection function that compares any > enumerator used against the list of enumerators that are deprecated. > > > That might be enough for Redis, but them I would be keeping other > > Redis-like databases off, for example > > > > * https://github.com/Snapchat/KeyDB > > * https://github.com/scylladb/scylladb/blob/master/docs/dev/redis.md > > * https://github.com/dragonflydb/dragonfly > > * Perhaps more. > > > > I don't know how compatible they are with the Redis API. I can't > > afford working on this right now and in the foreseeable future, I see > > it almost as a second library of a module of my library e.g. > > aedis::redis::search::request or something. > > Ok, this is something different. If you don't want to break support > for those other DBs, that's understandable. But since you don't have > support for them at the moment, I don't think possibly adding support > for those later (which could also be added in much the same way) > should slow down adoption of a better interface for Redis today. > > Zach

1 0