May 2022 - Boost - lists.preview.boost.org

Re: [boost] Review of Boost.MySql
by Ruben Perez 13 May '22

13 May '22

> > > > I was just imagining users concatenating queries. May be a misconception, > > yours is a legitimate use case. > > I've thought more about this, and I'm guessing you were thinking > about this XKCD https://xkcd.com/327/ SQL Injection. Yes, this was definitely what I was thinking of ;) > > The proper and ideal remedy is not the XKCD one though, > it is using prepared statements and binding values. > > The still proper but less ideal next remedy is a "safe" way to concatenate > user inputs into textual SQL. I.e. the equivalent of > SQLite: https://www.sqlite.org/printf.html with %q and %Q > PostgreSQL: https://www.postgresql.org/docs/current/libpq-exec.html#LIBPQ-PQESCAPELITER… > Since you want to avoid the MySQL client, which I suppose has this > too, you have to provide your own. Since not everything can be done with prepared statements (I've been shown some legitimate use cases that are not covered), I've raised https://github.com/anarthal/mysql/issues/69 to address this, and I will likely implement something like this. > > Third and finally, this XKCD hack indeed does only work is one can > execute several semi-colon separated statements. > So on second thought, it is *definitely* be a good thing to have > separate APIs, to execute single statement, as opposed from executing > "scripts" of statements, plural. With a single-statement API, *even* > the XKCD hack won't work, since it would yield an invalid single > statement. > This is what you had in mind IMHO :). The trouble with this is that multi-statement is not a separate protocol primitive, but rather something you turn on or off for the entire session. So your connections either support it or don't. If you include such a semicolon separated set of queries in your query request, and this capability is turned on, the server will happily execute it. So maybe this should be implemented as an extra value in connection_params (https://anarthal.github.io/mysql/mysql/connparams.html) with adequate security warnings.

1 0

Re: [boost] Review of Boost.MySql
by Dominique Devienne 13 May '22

13 May '22

Meant to send this to the list, not just Ruben. Re-sending. --DD On Fri, May 13, 2022 at 11:59 AM Dominique Devienne <ddevienne(a)gmail.com> wrote: > > On Thu, May 12, 2022 at 9:02 PM Ruben Perez <rubenperez038(a)gmail.com> wrote: > > > > The server sends several resultsets after that. I haven't focused a lot on > > > > this because it sounded risky (in terms of security) for me. > > > > > > Not sure what's risky here. Maybe I'm missing something. > > > > I was just imagining users concatenating queries. May be a misconception, > > yours is a legitimate use case. > > I've thought more about this, and I'm guessing you were thinking > about this XKCD https://xkcd.com/327/ SQL Injection. > > The proper and ideal remedy is not the XKCD one though, > it is using prepared statements and binding values. > > The still proper but less ideal next remedy is a "safe" way to concatenate > user inputs into textual SQL. I.e. the equivalent of > SQLite: https://www.sqlite.org/printf.html with %q and %Q > PostgreSQL: https://www.postgresql.org/docs/current/libpq-exec.html#LIBPQ-PQESCAPELITER… > Since you want to avoid the MySQL client, which I suppose has this > too, you have to provide your own. > > Third and finally, this XKCD hack indeed does only work is one can > execute several semi-colon separated statements. > So on second thought, it is *definitely* be a good thing to have > separate APIs, to execute single statement, as opposed from executing > "scripts" of statements, plural. With a single-statement API, *even* > the XKCD hack won't work, since it would yield an invalid single > statement. > This is what you had in mind IMHO :). > > So the ability to execute multi-statement is important, especially in > my DDL use-case, > but it should be a separate API from the more regular single-statement > APIs, and come > with extra warnings in the document about such SQL injections (with > obligatory link to XKCD ;) > > PS: Fourth, the XKCD hack is possible only of the SQL is executed as > the owner of the schema, > or a DBA/Superuser, which is of course a mistake and a big no-no, > yet all too common... > Users of the DB, including mid-tier services, should NOT have DDL > privileges on the schema and its objects. > Change the DROP into a DELETE, and that's still nasty, but then you > use ROLEs to restrict some DMLs, > and possibly also RLS (Row Level Security) for finer-grained > authorization. But we are getting off-topic :)

1 0

Future of C++ and Boost
by Vinnie Falco 13 May '22

13 May '22

It might be interesting to think about where C++ and Boost will be in the next 20 years and in the next 50 years. There has been an explosion of research and development into new programming languages (Rust being the most visible example). C++ is hardly the most popular language (Python is, if the TIOBE index is to be believed). But C++ is an essential language because it occupies the unique position of being as close as possible to assembly language while remaining high level. The zero-cost abstractions, metaprogramming, and powerful compositional tools of C++ still have no competitors and they are unlikely to have them any time soon. This doesn't necessarily mean that C++ is good for everything; there's an opportunity cost for all this. C++ code is generally more complex, harder to read, and harder to write than other code. Other languages are much better suited for certain tasks. No one should be implementing a content-driven website using Asio and Beast for example. But for the things that C++ is good at, it excels at them. A recent reddit post [1] provided a decent list:: * games * databases * high frequency trading * search (Google/Bing/DuckDuckGo) * web browsers (Chrome/Safari/Edge) * virtual reality headsets (Facebook) * distributed systems at big tech companies (Google/Microsoft/Facebook) * low level libraries (machine learning libraries like tensorflow and pytorch) * compilers, virtual machines, interpreters (JVM/gcc/LLVM) * robotics & hardware (ex: self driving cars, planes, missiles, space ships, etc) My opinion is that wg21 (the C++ standards committee) has been growing increasingly out of touch with the needs of the larger C++ community. Not necessarily out of malice, but because the bureaucratic process of the committee does not align the incentives of its members with users. I predict that we will see the C++ standard library become increasingly irrelevant by not only failing to produce stable things that people want, but also by not even keeping up feature-wise with the library ecosystem of other languages. Where is Requests [2] for C++? Python has had this for over a decade. In case you think I'm exaggerating, look at this comment from Reddit: ...a major vendor is getting ready to release a product that does not allow user applications to directly access TCP sockets so the only way to get any network access is to go through their standard library which only supports backdoored HTTPS. This vendor of course is Apple, as can be seen in P1860R0 [4]. We should not expect that wg21 will be able to deliver an important feature such as networking in a form that will be usable or recognizable to non-committee members. What does this mean for C++ and Boost? I believe there is a significant opportunity for Boost to step in and become the rational collection of actually useful libraries that the C++ standard library is not. We already have networking, some protocols, and JSON. We (the existing Boost authors and contributors) might consider looking to the list above of industries that use C++, and focus our efforts more towards providing libraries that enhance those industries. Perhaps by reaching out to leaders in those fields we might establish working relationships and build, say, the next Boost.Tensorflow or something like that. I don't know. Oh, and add "cryptocurrency" to the list above. C++ is the language of blockchain, because performance matters. What do you think? Thanks Vinnie [1] <https://www.reddit.com/r/cpp/comments/u5vjni/comment/i54qbzq> [2] <https://docs.python-requests.org/en/latest/> [3] <https://www.reddit.com/r/cpp/comments/u6z8gr/comment/i5caxr6> [4] <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1860r0.html>

22 50

Re: [boost] Review of Boost.MySql
by Ruben Perez 12 May '22

12 May '22

> Indeed. But even your imagined batch interface only works well for > queries/selects, > while for inserts (or updates), the client does not need to just send > a small textual > SQL query, but potentially a bunch of data for the rows too. A true > pipeline allows > sending the rows for a second insert while the first insert is being processed. It would work for inserts too, as values are either part of the query string or part of the statement execute packet. In both cases, part of the request. > Such a mode may be useful for schema creation OTOH. We have large schemas, > with hundreds of tables, indexes, triggers, etc... Done from C++ code > client-side, > not via a DBA manually executing on the server using SQL files and the > native CLI. > For that use-case, the ability to send many DDLs in a single batch > would definitely > save on the round-trips to the server. We try hard to minimize roundtrips! Thanks for sharing this use case, I definitely wasn't aware of it and seems a reason towards implementing multi-statement. > You don't need to necessarily use Google's protobufs library. > There's https://github.com/mapbox/protozero for example, and similarly, a > from-scratch implementation to just encode-decode a specific protocol > can also be written. I wasn't aware of this.Thanks. > > The server sends several resultsets after that. I haven't focused a lot on > > this because it sounded risky (in terms of security) for me. > > Not sure what's risky here. Maybe I'm missing something. I was just imagining users concatenating queries. May be a misconception, yours is a legitimate use case. > Note that PostgreSQL's COPY is not file-specific, and a first-class citizen > at the protocol level, depending on a pure binary format (with text mode too). > I use COPY with STDIN / STDOUT "files", i.e. I prepare memory buffers and > send them; and read memory buffers, and decode them. No files involved. Unfortunately, that does not seem to be the case for MySQL. You issue the LOAD DATA statement via a regular query packet, and the server returns you another packet with the file path it wants. You then read it in the client and send it to the server in another packet. I've made it work with CSV files, and I'd say it's the only format allowed, AFAIK. > Maybe our use-case of very large data (with large blobs), *and* very > numerous smaller data, > that's often loaded en-masse, by scientific desktop applications, and > increasingly mid-tier services > for web-apps, is different from the more common ecommerce web-app > use-case of many people. It is good to hear different use cases than the regular web server we all have in mind. It will help me during further development. > > When I evaluate DB performance, I tend to concentrate on "IO" performance, > in terms of throughput and latency, independent of the speed of the > SQL engine itself. > There's nothing I can do about the latter, while the way one uses the Client API > (or underlying protocol features) is under my control. So I do mostly > INSERTs and > SELECTs, with and without WHERE clauses (no complex JOINs, CTEs, etc...). > > Because we are in the scientific space, we care about both many small rows > (millions, of a few bytes to KBs each at most), and a few (hundreds / thousands) > much larger rows with blobs (with MBs to GBs sizes). The truly large > "blobs" (files) are > kept outside the DB, since mostly read only (in the GBs to TBs sizes each, that > can accumulate to 2.6 PB for all a client's data I heard just > yesterday for example). > > I'll also compare inserting rows 1-by-1, with and without prepared statements, > to inserting multi-rows per-statement (10, 100, 1000 at a time), to the "bulk" > interface (COPY for PostgreSQL, LOCAL for MySQL, Direct-Path load in OCI). > For example with SQLite: > https://sqlite.org/forum/forumpost/baf9c444d9a38ca6e59452c1c568044aaad50bba… > (SQLite as no "bulk" interface, doesn't need one, since "embedded" > thus "zero-latency") > > For PostgreSQL, we also compared text vs binary modes (for binds and > resultsets). > > For blobs, throughput of reading and writing large blobs, whole and in-part, > with different access patterns, like continuous ranges, or scattered inside). > > A very important use-case for us, for minimizing round-trips, is how > to load a subset of rows, > given a list of primary keys (typically a surrogate key, like an > integer or a uuid). For that, > we bind a single array-typed value for the WHERE clause's placeholder > to the query, > and read the resultset, selecting 1%, 5%, 10%, etc... of the rows from > the whole table. > (selecting each row individually, even with a prepared query, is terribly slow). Thanks for sharing. It will definitely help me during benchmarking. Regards, Ruben.

1 0

Mixing C++ language versions
by Kenneth Porter 12 May '22

12 May '22

Can one mix compilation units or libraries built with different versions of C++? If I build my DLL with C++20, can a customer use it with their C++11 app? How are compiler and library vendors dealing with this? Do/will they supply multiple binaries for each C++ version? (I'm already building for each architecture plus release/debug. Adding the compiler version explodes the number of binaries to build and test.)

3 3

Re: [boost] Mixing C++ language versions
by Glen Fernandes 12 May '22

12 May '22

On Thu, May 12, 2022 at 6:52 AM Kenneth Porter via Boost <boost(a)lists.boost.org> wrote: > > Do/will they supply multiple binaries for each C++ version? (I'm already building for > each architecture plus release/debug. Adding the compiler version explodes > the number of binaries to build and test.) The variants you have so far are: - Architecture - Release or Debug I've known library vendors to even have the following variants: - Architecture - Release or Debug - C++ standard version - Static or dynamic linking with the C++ runtime library - Single or multi threaded versions of the C++ runtime library (at one point in time) Shipping library sources to your users (instead of binaries) might be easier. Glen

1 0

[filesystem] create_directories fails on exFat formatted disk under Windows
by Chris Trueman 12 May '22

12 May '22

Using an exFat formatted disk with Windows. Start with a path: e:\tmp Call filesystem::create_directories("e:\\tmp\\boost") This should succeed, creating the boost folder in e:\tmp. It fails with boost 1.79.0. It tries to create e:\tmp which already exists. In boost 1.78.0 this works as expected. I believe the root cause is the change to status(...). It calls detail::symlink_status(...) which calls GetFileInformationByHandleEx which (at least for the params used in filesystem) doesn't work on exFat formatted disks. Chris.

2 2

SOCKS-Asio library
by Alan de Freitas 11 May '22

11 May '22

Hi, everyone! I've been writing this library for SOCKS4/5 based on Asio. The library is very embryonic but there's enough to connect to proxy servers and relay messages. I could really use some negative feedback. If anyone would like to give it a try, here are the links: Repo: https://github.com/alandefreitas/socks_proto Docs: https://develop.socks-proto.cpp.al/ #boost-socks on Slack Thanks! -- Alan Freitas https://github.com/alandefreitas

2 1

Re: [boost] Boost Digest, Vol 6699, Issue 1
by Benedict Bede McNamara 08 May '22

08 May '22

You need modules urgently. Sent from Mail for Windows From: boost-request(a)lists.boost.org Sent: Sunday, 8 May 2022 10:00 PM To: boost(a)lists.boost.org Subject: Boost Digest, Vol 6699, Issue 1 Send Boost mailing list submissions to boost(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.boost.org/mailman/listinfo.cgi/boost or, via email, send a message with subject or body 'help' to boost-request(a)lists.boost.org You can reach the person managing the list at boost-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost digest..." The boost archives may be found at: http://lists.boost.org/Archives/boost/ Today's Topics: 1. Re: Any objections to making C++14 the current baseline for Boost support and testing? (Seth) 2. Re: Any objections to making C++14 the current baseline for Boost support and testing? (Christopher Kormanyos) 3. Re: Any objections to making C++14 the current baseline for Boost support and testing? (Bo Persson) ---------------------------------------------------------------------- Message: 1 Date: Sat, 07 May 2022 15:30:08 +0200 From: Seth <bugs(a)sehe.nl> To: "Boost List" <boost(a)lists.boost.org> Subject: Re: [boost] Any objections to making C++14 the current baseline for Boost support and testing? Message-ID: <e76334c1-625a-4ee2-999f-75959822562b(a)www.fastmail.com> Content-Type: text/plain > Even though 14 has lotsof bug fixes, > the only real key advanced featureis relaxed constexpr constraints > anddigit separators. I'd say that polymorphic lambdas, lambda capture initializers, return type deduction, decltype(auto) are all more important and really felt. I can live without the library features (because they're easy to provide, except the UDLs) ------------------------------ Message: 2 Date: Sun, 8 May 2022 09:31:19 +0000 (UTC) From: Christopher Kormanyos <e_float(a)yahoo.com> To: Boost List <boost(a)lists.boost.org> Subject: Re: [boost] Any objections to making C++14 the current baseline for Boost support and testing? Message-ID: <1679327021.415697.1652002279472(a)mail.yahoo.com> Content-Type: text/plain; charset=UTF-8 >> Even though 14 has lotsof bug fixes, >> the only real key advanced featureis>> relaxed constexpr constraints >> and digit separators. I apologize for my perhaps unwisely chosen(or even wrong) wording. > I'd say that polymorphic lambdas, lambda> capture initializers, return type deduction,> decltype(auto) are all more important> and really felt. ... And I do respect this comment. But I'm not quite ready to give upon eleven just yet... Let's argue the controversial C++11/14baseline issue from a purely service-oriented,product-management point of view. Boost has a deep symbiosis with C++11and vice-versa. There are clients usingBoost with C++11. To drop C++11 from a product/servicepoint of view should only be allowed ifa viable option is provided --- freelyand easily obtained --- for those clientsusing Boost with C++11. So I would say, prove to the clients,prove to ourselves what this optionactually is. Which option will allowclients stuck on C++11 (maybe notby choice) to continue to use Boostwithout C++11 support? One possibility comes to mind:Freeze at 1.80 if you need C++11,but I don't like this option. Kind regards, Chris On Saturday, May 7, 2022, 03:36:26 PM GMT+2, Seth via Boost <boost(a)lists.boost.org> wrote: > Even though 14 has lotsof bug fixes, > the only real key advanced featureis relaxed constexpr constraints > anddigit separators. I'd say that polymorphic lambdas, lambda capture initializers, return type deduction, decltype(auto) are all more important and really felt. I can live without the library features (because they're easy to provide, except the UDLs) _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ Message: 3 Date: Sun, 8 May 2022 12:37:00 +0200 From: Bo Persson <bo(a)bo-persson.se> To: boost(a)lists.boost.org Subject: Re: [boost] Any objections to making C++14 the current baseline for Boost support and testing? Message-ID: <t586gu$cfq$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 2022-05-08 at 11:31, Christopher Kormanyos via Boost wrote: > >> Even though 14 has lotsof bug fixes, >> the only real key advanced featureis>> relaxed constexpr constraints >> and digit separators. > I apologize for my perhaps unwisely chosen(or even wrong) wording. > >> I'd say that polymorphic lambdas, lambda> capture initializers, return type deduction,> decltype(auto) are all more important> and really felt. > ... And I do respect this comment. > But I'm not quite ready to give upon eleven just yet... > > Let's argue the controversial C++11/14baseline issue from a purely service-oriented,product-management point of view. > Boost has a deep symbiosis with C++11and vice-versa. There are clients usingBoost with C++11. > To drop C++11 from a product/servicepoint of view should only be allowed ifa viable option is provided --- freelyand easily obtained --- for those clientsusing Boost with C++11. > So I would say, prove to the clients,prove to ourselves what this optionactually is. Which option will allowclients stuck on C++11 (maybe notby choice) to continue to use Boostwithout C++11 support? It can also be argued that we might do some clients a disservice instead of a service. If the new features are *that* useful for the library authors, would they not be equally useful for the clients' code? By supporting old compilers and standards we let them procrastinate for another couple of years. Is that good? Or should we encourage them to move on? Perhaps a gentle push is all they need? "New and improved" is often used in product-marketing. Why not here? > One possibility comes to mind:Freeze at 1.80 if you need C++11,but I don't like this option. > > Kind regards, Chris > > ------------------------------ Subject: Digest Footer _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ End of Boost Digest, Vol 6699, Issue 1 **************************************

2 1

Copy_File Method Question
by Sean B 06 May '22

06 May '22

Good afternoon, Thank you for allowing me to join the group as I a newbie to the boost c++ libraries. My question I have to the development group is based on the copy_file method. Does the copy_file do a 'bit to bit' copy using (check sums and hash) or the original file? Thanks for any information on this.

2 1

C++ package managers
by Adam Getchell 06 May '22

06 May '22

For what it’s worth, having tried Biicode (Conan’s predecessor), Conan, Spack, Conda, and vcpkg, vcpkg is the only package manager for which I’ve been able to consistently (i.e. since I started using it a few years ago) build combinations of {macOS, Linux, Windows } x { gcc, clang, msvc}. As my research project includes CGAL which includes Boost, Eigen, and TBB, that ends up being 100+ libraries. Manifest mode just works, and vcpkg artifacts looks promising: https://devblogs.microsoft.com/cppblog/vcpkg-artifacts/ -- Adam Getchell acgetchell(a)ucdavis.edu https://adamgetchell.org

2 2