Q2 2024 Transparency Report on Boost C++ Library
Hello, Louis Tatta here, CEO of The C++ Alliance, Inc, a registered 501(c)(3) non-profit organization dedicated to supporting C++. This is our Q2 2024 transparency report on our charitable work for the Boost C++ Library Collection. The C++ Alliance actively seeks out highly talented and experienced C++ experts, hiring them, often full-time, so they can fully dedicate themselves to open-source projects. Our Staff Engineers maintain existing Boost libraries, develop new ones, and contribute to other Boost projects. We also employ IT experts full-time to maintain and enhance the infrastructure supporting Boost. This includes the legacy web server, the Drone continuous integration servers, deployment scripts for GitHub Actions on Boost repositories, and scripts that publish Boost releases. Additionally, we provide on-demand technical support services for Boost authors and maintainers facing infrastructure-related issues. The Boost Mailing List remains the premier venue for high-quality discussions on Boost libraries and related C++ Standards topics. The C++ Alliance continues to support the mailing lists, upgrading them to the latest Mailman version and developing an optional web-based front-end for browser-based interaction. We also manage the Official C++ Language Slack Workspace ( https://cpp.al/slack), hosting the most popular interactive discussions on Boost libraries. The C++ Alliance sponsors a paid plan to offer full history, video and audio conferencing, collaboration tools, and unlimited document storage for seventeen dedicated Boost library channels and one main Boost discussion channel. Our largest project for Q2 2024 is the ongoing development of the new Boost website. This new site will be a C++ social media portal with rich support for user-generated content and a robust back-end that provides visibility into contributions made on GitHub and the release process. Our goal is to deliver an enhanced user experience that attracts fresh talent to Boost. Notably, the website features hand-drawn graphics by our contract artist, reflecting our commitment to ensuring Boost not only functions well but also looks visually appealing. In Q2 2024 there were a total of zero Boost Formal Reviews. There were two petitions for endorsement for potential Boost candidates: *async_mqtt from Takatoshi Kondo (https://github.com/redboltz/async_mqtt) *multi from Alfredo Correa (https://gitlab.com/correaa/boost-multi) There's another MQTT library in the works proposed on Nov 2023, Async.MQTT5 from Ivica Siladic, Bruno Iljazovic and Korina Simicevic ( https://github.com/mireo/async-mqtt5) There were a total of three messages from the @Boost_Libraries X (formerly Twitter) account, all submitted by C++ Alliance team members. One of them featured artwork from Bob Ostrom, commissioned by us. Posts with visuals are three times more engaging than those with just text. We see the X platform as a vital tool for educating newer generations about the benefits of Boost. A total of $526,000 was spent on Boost-related contributions, broken down thusly (and rounded to the nearest thousand): Staff Engineers Compensation $392,000 Website Software Development $111,000 Server Hosting $23,000 During Q2 2024 we employed a total of 13 individuals to contribute to Boost. Here is a high-level summary of accomplishments: Sam Darwin (Chief Technical Officer) - Summary: Sam Darwin led major projects, overseeing the integration of new features in the Boost infrastructure. He coordinated between teams, ensuring smooth transitions and high-quality deliverables. His leadership was pivotal in the development of the new Boost website and improvements to the mailing list system. - Link: Sam Darwin's Q2 Update https://cppalliance.org/sam/2024/07/10/SamsQ2Update.html Dmitry Arkhipov (Staff Engineer) - Summary: Dmitry Arkhipov focused on maintaining and enhancing Boost libraries. He implemented several bug fixes, optimized existing code, and contributed to the development of new libraries. Dmitry also played a significant role in the continuous integration and deployment processes. - Link: Dmitry Arkhipov's Q2 Update https://cppalliance.org/dmitry/2024/07/12/dmitrys-q2-update.html Alan de Freitas (Staff Engineer) - Summary: Alan de Freitas worked extensively on the Boost website redesign, particularly on the back-end systems that support user-generated content and GitHub integration. He also contributed to developing new libraries and maintaining existing ones. - Link: Alan de Freitas's Q2 Update https://cppalliance.org/alan/2024/07/13/AlanQ2Update.html Christian Mazakas (Staff Engineer) - Summary: Christian Mazakas worked on enhancing Boost’s infrastructure, including optimizing server performance and improving deployment scripts. His contributions were crucial for maintaining the stability and efficiency of the Boost libraries. - Link: Not provided. Krystian Stasiowski (Staff Engineer) - Summary: Krystian Stasiowski focused on developing new features for existing Boost libraries and worked on performance optimizations. His work included contributing to the ongoing development of new libraries and providing technical support to other Boost authors. - Link: Krystian Stasiowski's Q2 Update https://cppalliance.org/krystian/2024/07/15/KrystiansQ2Update.html Peter Turcan (Senior Technical Writer) - Summary: Peter Turcan was responsible for creating and updating documentation for Boost libraries. He ensured that all new features and libraries were well-documented, making it easier for users to understand and utilize Boost’s capabilities. - Link: Peter Turcan's Q2 Update https://cppalliance.org/peter/2024/07/07/PeterTurcan-Q2-2024.html Matt Borland (Staff Engineer) - Summary: Matt Borland contributed to the development of new libraries and enhancements to existing ones. His focus was on improving library performance and ensuring that the codebase remained robust and reliable. - Link: Matt Borland's Q2 Update https://cppalliance.org/matt/2024/07/08/Matts2024Q2Update.html Ruben Perez (Staff Engineer) - Summary: Ruben Perez worked on several infrastructure projects, including server maintenance and optimization. He also contributed to the development and maintenance of Boost libraries, focusing on performance improvements and bug fixes. - Link: Ruben Perez's Q2 Update https://cppalliance.org/q2_update/2024/07/09/RubenQ2.html Mohammad Nejati (Staff Engineer) - Summary: Mohammad Nejati was involved in developing new libraries and improving existing ones. He focused on optimizing code and ensuring that Boost libraries were efficient and reliable. His work also included providing technical support to other developers. - Link: Mohammad Nejati's Q2 Update https://cppalliance.org/mohammad/2024/07/10/MohammadsQ2Update.html Fernando Pelliccioni (Staff Engineer) - Summary: Fernando Pelliccioni contributed to the ongoing development of the Boost libraries, focusing on implementing new features and optimizing performance. His work was instrumental in ensuring the high quality and reliability of Boost libraries. - Link: Fernando Pelliccioni's Q2 Update https://cppalliance.org/fernando/2024/07/08/FernandoQ2Update.html Joaquin M Lopez Munoz (Staff Engineer) - Summary: Joaquin M Lopez Munoz worked on several key projects, including the development of new Boost libraries and enhancements to existing ones. His contributions focused on improving code efficiency and performance. - Link: Joaquin M Lopez Munoz's Q2 Update https://cppalliance.org/joaquin/2024/07/06/Joaquins2024Q2Update.html Kenneth Reitz (Staff Engineer) - Summary: Kenneth Reitz focused on developing and maintaining Boost libraries, implementing new features, and optimizing existing code. His work also involved providing technical support and contributing to the Boost website redesign. - Link: Kenneth Reitz's Q2 Update https://cppalliance.org/kenneth/2024/07/12/KennethQ2Update.html Braden Ganetsky (Junior Staff Engineer) - Summary: Braden Ganetsky, as a junior engineer, contributed to various Boost library projects. His work included implementing new features, fixing bugs, and assisting with infrastructure maintenance. Braden’s contributions were valuable in supporting the overall development efforts. - Link: Braden Ganetsky's Q2 Update https://cppalliance.org/braden/2024/07/14/BradenQ2Update.html
Louis Tatta wrote:
Server Hosting $23,000 [per quarter]
That's a lot of money. Does this include the Boost downloads using Fastly that were announced at the end of May? Does this include non-obvious things, like internal C++ Alliance infrastructure? It's really none of my business how you spend your money, but I do have a certain amount of experience with this sort of thing. I worry that you have over-engineered this in a way that has made it much more expensive than it needs to be. Regards, Phil.
On Fri, Jul 19, 2024 at 9:15 AM Phil Endecott via Boost < boost@lists.boost.org> wrote:
I worry that you have over-engineered this in a way that has made it much more expensive than it needs to be.
That may be true, and our principle for solving these problems is to first optimize for getting something in place quickly, with the highest possible level of service, with cost as a secondary consideration. And then once things are stable we look at other available, less expensive solutions. This lets us evaluate alternatives for reducing cost, at our own pace and after other more pressing matters are resolved, without pressure. Thanks
A few months ago, Glen informed me that the Boost download was not working and asked if we could help host it. I spoke to Sam, and we decided the quickest way to get the download working again was to use our AWS for hosting. The rarely used http://boostreleases.cppalliance.org CDN had already been configured, and we added http://archives.boost.io as well. After a few months, we switched from the AWS CDN to a Fastly CDN at a more favorable monthly rate. AWS is still involved as the origin servers for the Fastly distribution.
On Jul 19, 2024, at 9:52 AM, Vinnie Falco via Boost
wrote: On Fri, Jul 19, 2024 at 9:15 AM Phil Endecott via Boost < boost@lists.boost.org> wrote:
I worry that you have over-engineered this in a way that has made it much more expensive than it needs to be.
That may be true, and our principle for solving these problems is to first optimize for getting something in place quickly, with the highest possible level of service, with cost as a secondary consideration. And then once things are stable we look at other available, less expensive solutions. This lets us evaluate alternatives for reducing cost, at our own pace and after other more pressing matters are resolved, without pressure.
Thanks
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Phil Endecott via Boost
Louis Tatta wrote:
Server Hosting $23,000 [per quarter]
That's a lot of money.
Indeed. For perspective, for $7K/month you could rent about 45 Xeon Gold 24-core servers on Hetzner. So over 1000 cores of compute or about 7 dedicated cores per Boost library. Imagine the CI system you could build from that. (I appreciate that managing those 45 servers will cost something, potentially even more than the servers themselves. Still, for something like CI which doesn't have strict reliability/availability requirements, a lot of this management cost can be pushed upfront and made fixed.)
It's really none of my business how you spend your money, [...]
I had a related question for while now so while at it, let me ask it: Where does this money come from and how long will it last? I looked around the C++ Alliance website for the answer but all I could find was that "The Alliance is currently funded by a private endowment". Are you (as in, Boost developers) not concerned where this non-trivial amounts of money come from? Equally important, are you not concerned about what happens if/when it runs out? For example, there seem to be multiple, full-time developers working on the new Boost website. Based on that I think it's reasonable to assume that the result will require a non-trivial amount of ongoing maintenance. What happens when/if there is no more funding for this maintenance? Are the Boost developers prepared to step in and do web development? I realize am asking pointed questions while not really being part of the project (other than packaging Boost for build2), so if the answer is "this is none of your business", then that's fair enough.
El 20/07/2024 a las 9:39, Boris Kolpackov via Boost escribió:
I realize am asking pointed questions while not really being part of the project (other than packaging Boost for build2)[...] Well, contributing to Boost release processes _is_ part of the project, so thank you very much for your help :-)
Joaquin M Lopez Munoz
you could rent about 45 Xeon Gold 24-core servers on Hetzner.
Hi Boris, The $7k was affected by a one-time charge on AWS Cloudfront that will be reduced in future quarters. Much of the current billing now is CI. Hetzner is a great hosting company. We have an account there, and this is an interesting idea. Still, analyzing the number (45 servers), cut that by 50% or more, since the expected billing amount is less. And it's going towards other costs also. Let's say 23 Hetzner servers. The CI jobs we're hosting burst up to 500 simultaneous jobs. Is it enough? The solution is autoscaling (on AWS). I explored autoscaling on Hetzner also. The problem is that you want to create a disk image (an AMI) with all the preinstalled CI software, and be able to quickly launch and destroy the instances, using the custom AMI. However, Hetzner allows that to happen on a restricted number of officially provided AMIs, and then when it comes to a customized image, that must be transferred across their network. It was taking more than 10 minutes. Not an acceptable delay. A CI job should be launching instances in 1-2 minutes. I confirmed the delay issue with their tech support. No fix. Nevertheless, in spite of that, I do occasionally think the same thought as you: "Let's launch 100 dedicated machines on Hetzner". Maybe at some point, yes.
Sam Darwin
The CI jobs we're hosting burst up to 500 simultaneous jobs. Is it enough? The solution is autoscaling (on AWS).
In my experience, the cost (in engineering complexity and/or in money paid to AWS) is not really worth it for CI. In build2 we have a fixed set of dedicated CI machines (hosted in our office, which makes it even cheaper than Hetzner, long-term). Their background job is to continuously rebuild packages in our repository (in particular, this detects regressions due to new versions of dependencies). But if there is a CI job, then it takes priority over the background rebuilds. Adding a new CI machine is basically a matter of procuring the hardware since the host OS image boots over the network and requires minimal per-machine setup. Another thing to keep in mind is that for certain targets, most notably Mac OS, you are restricted to specific hardware you can run them on (legally, in case of Mac OS). For example, while AWS provides auto-scaling for Apple hardware, it's on the 24-hour minimum allocation period (to comply with the Apple license). Being able to "drop down" all the way to running own hardware on premises is the only guaranteed way of handling such cases.
On Sat, Jul 20, 2024 at 12:50 AM Boris Kolpackov via Boost < boost@lists.boost.org> wrote:
I had a related question for while now so while at it, let me ask it: Where does this money come from and how long will it last? I looked around the C++ Alliance website for the answer but all I could find was that "The Alliance is currently funded by a private endowment".
The Alliance funding could last many decades. Long enough, that we need a plan for transitioning the organization to others in the case of the loss of its principals. This is not such an easy problem to be solved as my network of contacts are around my age, and ideally the executor of any legal entity which would outlast the current principals should be significantly younger. Happy for any volunteers to reach out to me at vinnie@cppalliance.org. Are you (as in, Boost developers) not concerned where this non-trivial
amounts of money come from? Equally important, are you not concerned about what happens if/when it runs out?
This is a concern, yes. Note however that this is not a new concern as Boost already has exposure to the loss of volunteers and resources: * When an author abandons a library * When a maintainer stops working on a library * When a release managers retiresn * When Boost infrastructure is not maintained * When the custodian of Boost's shared resources turns its attention to other projects These problems have existed and will continue to exist for the lifetime of the project. The solution will be the same as it has always been. New volunteers and contributions are necessary. there seem to
be multiple, full-time developers working on the new Boost website. Based on that I think it's reasonable to assume that the result will require a non-trivial amount of ongoing maintenance. What happens when/if there is no more funding for this maintenance? Are the Boost developers prepared to step in and do web development?
The new website is an ambitious project which will require non-trivial resources over the next few years to implement, as we have many interesting features planned with the hopes of increasing qualified participation in the project. Once the site matures, the ongoing maintenance will be considerably less. We chose Python/Django for its enormous popularity and for the ease of which it is to find talented individuals who can perform the work, whether volunteer or paid. An emphasis was placed on using existing technologies that follow best practices. Boost's existing website, running on the "wowbagger" server, is already an existential risk, and essentially is unfixable. See: https://cppalliance.org/pdf/Boost-Server-Report.pdf What I am saying is that while the hypothetical problem of C++ Alliance lacking funds may be a problem in the future, the current website's lack of resources is a real problem today. My usage of the term website also encompasses the mailing lists, which run on the same machine and get served from the same domain. To mitigate the risk of short-term problems with the Alliance, we have deployed a static copy of the old website which is very cheap to maintain and runs on modern software and services. This will be available at a separate URL even with the new website up, allowing a grace period where folks can experience the old site for whatever reason. Including the case where we forgot a page or some information, and we need to refer to the old site to update the new one. This static duplicate of the current boost.org also serves another function. It is a safety mechanism if the new website goes lights-out for any reason. This way the old site can be pressed back into service. It is true that the community will suffer a loss of aesthetics and functionality but this is not much different than when someone abandons a library. Boost can and will deal with such eventualities as it always does. I realize am asking pointed questions while not really being part
of the project (other than packaging Boost for build2), so if the answer is "this is none of your business", then that's fair enough.
I think these are fair questions. And for the health of Boost I think more people should be asking these questions, and related questions. Of course, volunteering time and resources would be better still :) Thanks
participants (6)
-
Boris Kolpackov
-
Joaquin M López Muñoz
-
Louis Tatta
-
Phil Endecott
-
Sam Darwin
-
Vinnie Falco