- Boost - lists.preview.boost.org

Re: [boost] library usage statistics
by Gavin Lambert 30 Apr '23

30 Apr '23

On 1/05/2023 03:19, Peter Dimov wrote: > Robert Ramey wrote: >> The basic question that occurs to me is "is this library worth maintaining and >> keeping in boost". If dropping the library would break a lot of users code, >> then the answer is an easy yes. If dropping the library would result in breaking >> no code - the answer is an easy no. >> Otherwise it's a judgement call we can argue about. But in order to do this we >> need to know the answer the question: "How much code would be broken if >> not in Boost. Hopefully this sheds light on this. > > There's probably a way to answer the question "how many Debian (Ubuntu, > Fedora...) packages would be broken if we remove library X from Boost", but > I don't know what it is. You could in principle trawl the package indexes and find how many packages have a direct dependency on "libboost-xyz" (or whatever the local package name for a specific library is), and then recursively work upwards by finding packages that depend on that package, etc. It's harder for the Boost libs that don't produce binary packages -- often there will be a dependency to a readme or something even when there isn't a binary, but not always. You *might* be able to narrow it down if the source packages declare a dependency on specific Boost library source packages rather than a monolithic libboost-dev, though.

1 0

Re: [boost] Boost.Compat
by Gavin Lambert 30 Apr '23

30 Apr '23

On 29/04/2023 02:11, Glen Fernandes wrote: > On Friday, April 28, 2023, Dominique Devienne wrote: >> Of course, Boost.Compat (or Boost.Polyfill) would need to turn >> on/off its std:: alternatives depending on the standard level in use. > > If everyone agrees, this can be one of the tenets of Boost.Compat from the > very beginning. > > The "depending on the standard level in use" might be further clarified to > mean "provides a fully functional std:: version". A Boost implementation *must not* provide a definition appearing in the std:: namespace. (Except where expressly permitted by std extension points, such as std::hash. Though those make me sad.) A Boost implementation *should not* provide a definition that is merely `using std::foo` if it does have its own alternative definition, unless there is no risk of incompatibility (type traits are relatively safe, classes are not). A Boost implementation *may* provide different implementations depending on std level in effect, but *only* in separate namespaces (with alias in the "main" namespace). (i.e. you can define boost::foo::cxx11::myfoo and boost::foo::cxx17::myfoo and then alias boost::foo::myfoo to one or the other based on std level. Ideally, a C++17 compilation would define both and a C++11 compilation would only define one, since it presumably can't define the newer one.) Where both std and boost types exist, an application should be allowed to select whichever one it prefers; including selecting the boost implementation of a type even if an equivalent or better type exists in std. Reasoning: even in this day and age, different compilation units may be compiled with different --std levels, and it is a Very Bad Idea™ to have different ABI definitions visible to different compilation units, as ODR violations are no fun for anyone. If the above is followed then you'll get either working code or a clean link error at the seams, instead of more subtle horribleness. Similarly, an uplevel compilation unit may want to explicitly select a Boost type in its interface to a downlevel compilation unit, expecting that these will be compatible types. (In an ideal world, this would apply to any other ABI-affecting condition as well; though you can usually ignore platform/compiler-level differences not as easily changed as the std level.)

1 0

Re: [boost] Replacing the ML with a forum
by Ion Gaztañaga 30 Apr '23

30 Apr '23

On 27/04/2023 2:26, Gavin Lambert via Boost wrote: > On 27/04/2023 08:40, Robert Ramey wrote: >> I use a threaded email reader - Thunderbird. >> >> I get my mailing list posts via a news reader. >> >> This is a perfectly wonderful setup and I'd be loath to change it. > > +1 on all three points. Via gmane, specifically. > Same here. Thunderbird with gmane to read the mailing list as a newsgroup. Works great. Ion

1 0

Boost CI Infrastructure
by Vinnie Falco 30 Apr '23

30 Apr '23

The C++ Alliance has an interest in, and has made ongoing efforts towards, ensuring the availability and quality of the Continuous Integration systems that Boost uses. If you recall, when Travis went pay we deployed our own instances of Drone (https://drone.io) Sam submitted pull requests to every Boost repository offering the option (but not the requirement) to have builds take place on our Drone infrastructure. Now I'm not sure if it could handle every single library doing builds of 20+ targets but it helped some. GitHub Actions became available, and there was the possibility to split the load (or make a redundant script) of targets across both Drone and GHA. However we have noticed that GHA for the Boost GitHub organization kind of sucks because we have 160+ libraries that are all submitting GHA jobs under the same user (the https://github.com/boostorg Organization account). When users submit GHA jobs for their libraries under their own account (for example if I run a GHA CI job for Boost.URL using my fork at https://github.com/vinniefalco/url) we have noticed that the turnaround is much better. Probably because the individual account is not competing with 160+ other jobs. The C++ Alliance upgraded its GHA account status for free, to have elevated access to resources for being a certified non-profit organization. We are in the process of getting the Boost GHA account upgraded as well. However, since we still have 160+ libraries all competing for resources it might still not be enough. Drone offers the possibility of self-hosted runners, but Alan ran into a problem with Drone. That is, that it's scripting language (Starlark) for lack of a better term "sucks balls." The Python version isn't much better. The main problem is that it is very hard to write scripts which are compositions of other scripts, one because of the object model of the CI and two because of these security restrictions which attempt to prevent abuse/spam. Alan de Freitas has done extensive work writing scripts for both Drone and GHA and has concluded that the GHA language and ecosystem is far superior. There is a free marketplace for GHA actions where you can just "include" a program for anything that you need to do: https://github.com/marketplace?type=actions It is very easy to compose new actions out of existing actions, and having the CI dashboard integrated into the repository and pull requests is definitely a plus. The problem still remains that GHA is overloaded during peak times. Sam Darwin (our CTO) is attempting to deploy a network of self-hosted GHA runners for the Boost organization. These are costly and we would like to only activate these runners when GitHub's shared runners are over some threshold of load. Thus Sam is wrestling with various APIs (and reaching out to Microsoft) to see how we can achieve this. The problem is that the GHA action file has to specify which runner to use (our own versus the share) and this complicates things as this file is committed to the repository. I guess what I'm trying to say here is that we care deeply about long turnaround times on CI ("they suck) have been working on the problem. I have even been working on this myself before The C++ Alliance became involved, by trying to refactor my libraries to compile faster. To break out things common to tests into their own .lib or .o file so that the same library sources are not compiled more than once. By reducing the amount of stuff in headers, avoiding overly fancy and complex metaprogramming, cutting down on the amount of instantiations of various function templates. Alan and I have also put effort into making the most out of a fixed number of CI targets. For example, by cleverly arranging the settings on the various targets so that we get the most amount of combinations of toolchains and settings out of the least amount of targets. Okay, tl;dr: time. If you are having two hour CI turnarounds, it means quite frankly that you have not put any effort into caring about your CI's workload. Just like you can't expect to keep putting more and more shit into header files and having compilations stay fast, you also cannot expect to just load your CI up with everything and have it always be responsive. You need to continually monitor and adjust your CI scripts to get the most out of the finite CI resources, just like you have to continually refactor and adjust your library to keep compile times down, to keep the number of dependencies on additional Boost libraries down (yes this matters Andrey) We can throw more resources at the problem but C++ authors (especially Boost ones) will always be able to write code and scripts that consume more resources than what is available. We might have a section in the "Contributor's Guide" outlining strategies for cutting down CI workloads and turnaround times (Peter Turcan)?. Alan de Freitas has done a considerable amount of good work in this area so if you would like to ask specific questions on the list or in the Slack workspace that is also a good choice (or you can ask me or Sam). Thanks

1 0

Re: [boost] Boost CI - a modest proposal
by Peter Dimov 30 Apr '23

30 Apr '23

Robert Ramey wrote: > I would suggest we throw the whole thing out and consider a different > approach. This approach would be more along the lines of that used by > CMake test server. Complete documentation of how CMake does it is > described in the CMake documentation. Basically it works like this. > > I run my CMake tests locally. This means that tests are not run on a server, but > on some users configuration. This spreads the load. Well, the main point of CI is to test your code on platforms other than your local one.

1 0

Re: [boost] Boost CI - a modest proposal
by Peter Dimov 30 Apr '23

30 Apr '23

John Maddock wrote: > It does however require discipline to not trigger useless builds (or cancel them > when they happen), ... For heavy CI workloads, setting up ccache correctly helps a lot in these cases.

1 0

Re: [boost] Boost CI - a modest proposal
by Andrey Semashev 30 Apr '23

30 Apr '23

On 4/30/23 18:37, Robert Ramey via Boost wrote: > the implementation of Boost CI follows the normal pattern for the > implementation of this functionality. This entails download the latest > version of the library to be tested and it's dependencies (or maybe all > of boost?) from the repo, Building everything from scratch, running all > the tests and logging the output there of. This is done for each > compiler / environment to be tested. Other than the raw console logs, > there is no presentation of the tests results (e.g boost test matrix). > > This is extremely unsatisfactory. > > a) It's ridiculously resource intensive. Rebuilding everything all the > time. Rebuilding everything is good. Incremental builds are fragile. > b) takes forever unless one has many more servers than we have. > > c) builds/tests even with the most trivial check-in. like adding a > period to a documentation page. You can mark commits to skip CI, if you want. > I would suggest we throw the whole thing out and consider a different > approach. This approach would be more along the lines of that used by > CMake test server. Complete documentation of how CMake does it is > described in the CMake documentation. Basically it works like this. > > I run my CMake tests locally. This means that tests are not run on a > server, but on some users configuration. This spreads the load. > > This means I run a special CMake target which takes the latest results > and posts them to a server. The results are not posted automatically - > I have to ask for it so it only happens when I know that a change is not > trivial. > > It also means that all configurations actually being used (and only > those configurations) are tested and the results logged. > > The server can than be queried to produce a table of results by > configuration, compiler, library, etc on demand. CMake people maintain > a basic server where one can post his results. But it's query facility > isn't really complete enough for boost needs. This facility would need > to be upgraded to boost requirements. Actually we have several > application which might serve as a starting point: my library_status app > which I use to produce a table of results and the app which produces the > boost library test matrix. > > All in all this would produce a system which would actually be useful. As a developer, I would like to know that the commit I pushed works on all configurations I care to support relatively soon (in a matter of minutes to hours), so that I can merge this commit to master. I do not want to wait until someone bothers to checkout and test this commit - on every supported system. You cannot expect this to happen any time soon, and you'll be lucky if you hear a bug report before a release deadline, when it is already too late to fix it.

1 0

Re: [boost] Boost CI - a modest proposal
by John Maddock 30 Apr '23

30 Apr '23

On 30/04/2023 16:44, Robert Ramey via Boost wrote: > the implementation of Boost CI follows the normal pattern for the > implementation of this functionality. This entails download the > latest version of the library to be tested and it's dependencies (or > maybe all of boost?) from the repo, Building everything from scratch, > running all the tests and logging the output there of. This is done > for each compiler / environment to be tested. Other than the raw > console logs, there is no presentation of the tests results (e.g boost > test matrix). I think you're wrong to call it "Boost.CI", most of what you're complaining about is the stuff that's outside our control - like the way Github actions run and so forth. > > This is extremely unsatisfactory. > > a) It's ridiculously resource intensive. Rebuilding everything all > the time. For github, I believe it can be configured to retain artifacts between builds and therefore perform incremental builds.... but, that is horrendously heavy on storage space, my expectation is that Github would throw us off for excessive usage! > > b) takes forever unless one has many more servers than we have. We have zero servers for CI, it's all Github etc. > > c) builds/tests even with the most trivial check-in. like adding a > period to a documentation page. Put [CI SKIP] in the commit message and nothing will build. > > I would suggest we throw the whole thing out and consider a different > approach. This approach would be more along the lines of that used by > CMake test server. Complete documentation of how CMake does it is > described in the CMake documentation. Basically it works like this. > > I run my CMake tests locally. This means that tests are not run on a > server, but on some users configuration. This spreads the load. > Well that's how we used to do things ;) Run it yourself, and hope for the best! I can tell you that I hated CI when it turned up, I mean what? Disturb my workflow and actually force me to change stuff! I can also tell you that a well set up CI system catches no end of small issues that I would never have spotted otherwise, also tests on platforms which I otherwise have no access to, and I simply would not be without it now! It does however require discipline to not trigger useless builds (or cancel them when they happen), and the need to update what you're testing and on what platforms periodically is a right pain, but I can also see why Github (for example) will need to deprecate and remove outdated distributions from time to time. So we all have to live with change I guess :( John.

1 0

Re: [boost] Boost.Compat
by Peter Dimov 30 Apr '23

30 Apr '23

Andrey Semashev wrote: > It doesn't really matter how many dependencies Boost.Thread adds. But it does. And you even need to build the library in this case.

1 0

Re: [boost] Boost::Regex support with ICU for unicode characters.
by John Maddock 30 Apr '23

30 Apr '23

> We are working with boost::regex library to have unicode support. I want to > form a regex for validating unicode characters so I'm trying this with > boost library. > > I am facing an issue with a boost library with ICU support so > please suggest me how to proceed with the boost with ICU support. PFA for > this. > > > 1. *has_icu_builds* is used to enable icu support. > [image: image.png] > > 2. on *b2 install* no icu is there, please let me know the procedure of how > to enable icu. > [image: image.png] ICU is an external dependency, we don't install it: it should already be installed prior to building Boost. Further, if you are using one of the last couple of Boost releases, then Regex is a header only library, and there is no need to build and install anything. You will however, have to link to ICU when you build your program. With regard to not matching specific text, if you can post the actual test code, I'll look at it. John.

1 0