Any Interest in a port of ZLib to C++?
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate
Boost library (with a separate review, which has nothing to do with
Beast)? The differences would be as follows:
* Ported to use C++11 idioms
* Same header-only / compiled source model as Boost.JSON
* Better documentation and clear API
* Improved tests
Programs which use this C++ port of ZLib would get their zlib
dependency from Boost instead of the official zlib distribution. And
they can integrate it into their projects the same way that Boost
libraries are integrated. Or they can include
Le 2021-03-23 06:14, Vinnie Falco via Boost a écrit :
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
I see it's a derivative work of zlib, rather than a from scratch implementation. IANAL, but isn't there a license issue there? (zlib license has non-mandatory aknowledgements requests which i'm not sure are boost-compatible). Regards, Julien
I am not a lawyer either, but I believe that this would not clash with the BSL. If the Zlib license *required* that you put an acknowledgement in the documentation etc, that would clash. But the way it is worded, I think it's compatible. To answer the original question, I think this is a nice idea and I would probably use it. Kind regards, Philip Bennefall On 3/23/2021 8:55 AM, Julien Blanc via Boost wrote:
Le 2021-03-23 06:14, Vinnie Falco via Boost a écrit :
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
I see it's a derivative work of zlib, rather than a from scratch implementation. IANAL, but isn't there a license issue there? (zlib license has non-mandatory aknowledgements requests which i'm not sure are boost-compatible).
Regards,
Julien
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hi Vinnie,
The library you propose is really nice to have !
I had tried to use zlib with boost asio stream.
https://stackoverflow.com/questions/44712852/how-to-decompress-the-data-in-b...
However, I gave up on it and then I used the raw zlib directly.
https://github.com/msgpack/msgpack-c/blob/cpp_master/example/boost/asio_send...
If I would have modern, stream friendly, and header-only zlib library,
I would be very glad.
---
Thanks
Takatoshi
On Tue, Mar 23, 2021 at 2:16 PM Vinnie Falco via Boost
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)? The differences would be as follows:
* Ported to use C++11 idioms * Same header-only / compiled source model as Boost.JSON * Better documentation and clear API * Improved tests
Programs which use this C++ port of ZLib would get their zlib dependency from Boost instead of the official zlib distribution. And they can integrate it into their projects the same way that Boost libraries are integrated. Or they can include
in any one translation unit (like JSON). The motivation for this is twofold:
* Provide an easier way for users to access ZLib's functionality * To use this in some new libraries that I'm going to propose
-- Regards, Vinnie
Follow me on GitHub: https://github.com/vinniefalco
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On di, 23. mrt 18:43, Takatoshi Kondo via Boost wrote:
Hi Vinnie,
The library you propose is really nice to have !
I had tried to use zlib with boost asio stream. https://stackoverflow.com/questions/44712852/how-to-decompress-the-data-in-b...
I second that motion. I'll submit another case, where I recently finally managed to get websocket per-message deflation working for misbehaved services: - https://stackoverflow.com/questions/66282247/c-inconsistent-behavior-with-bo... If the ZLib interface were (part of) a documented library, then perhaps it would have taken roughly a year to solve this problem first asked here many months earlier https://stackoverflow.com/questions/62496815/boost-inflate-algorithm-decompr... Cheers, Seth
On Tue, 23 Mar 2021 at 05:14, Vinnie Falco via Boost
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)? The differences would be as follows:
While there's no harm in this, is zlib still really relevant in 2021? bz2, lzma, lz4, zstd are all superior.
On Tue, Mar 23, 2021 at 3:49 AM Mathias Gaunard
While there's no harm in this, is zlib still really relevant in 2021? bz2, lzma, lz4, zstd are all superior.
Great question. Yes the deflate algorithm is still relevant because it is used in websocket's permessage-deflate implementation, and it is also a very common Content-Encoding for HTTP message payloads. Thanks
Hi Vinnie,
I think this would be a good addition and something I'd probably use.
Regards,
Pete
On Tue, 23 Mar 2021 at 05:16, Vinnie Falco via Boost
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)? The differences would be as follows:
* Ported to use C++11 idioms * Same header-only / compiled source model as Boost.JSON * Better documentation and clear API * Improved tests
Programs which use this C++ port of ZLib would get their zlib dependency from Boost instead of the official zlib distribution. And they can integrate it into their projects the same way that Boost libraries are integrated. Or they can include
in any one translation unit (like JSON). The motivation for this is twofold:
* Provide an easier way for users to access ZLib's functionality * To use this in some new libraries that I'm going to propose
-- Regards, Vinnie
Follow me on GitHub: https://github.com/vinniefalco
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 3/23/21 8:14 AM, Vinnie Falco via Boost wrote:
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)? The differences would be as follows:
* Ported to use C++11 idioms * Same header-only / compiled source model as Boost.JSON * Better documentation and clear API * Improved tests
Programs which use this C++ port of ZLib would get their zlib dependency from Boost instead of the official zlib distribution. And they can integrate it into their projects the same way that Boost libraries are integrated. Or they can include
in any one translation unit (like JSON). The motivation for this is twofold:
* Provide an easier way for users to access ZLib's functionality * To use this in some new libraries that I'm going to propose
I would most probably *not* use it and rely on zlib or zlib-ng because (a) it would probably be used by other libraries that are not part of your ecosystem, (b) because its maintenance would be a someone else's burden (e.g. a distro maintainer's) and not mine, and (c), especially in zlib-ng case, I suspect it would be better optimized than your header-only version. Because of this I'd like to ask you to at least provide an option to build your present and future libraries against zlib.
On 3/23/2021 1:14 AM, Vinnie Falco via Boost wrote:
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)? The differences would be as follows:
* Ported to use C++11 idioms * Same header-only / compiled source model as Boost.JSON * Better documentation and clear API * Improved tests
Programs which use this C++ port of ZLib would get their zlib dependency from Boost instead of the official zlib distribution. And they can integrate it into their projects the same way that Boost libraries are integrated. Or they can include
in any one translation unit (like JSON). The motivation for this is twofold:
* Provide an easier way for users to access ZLib's functionality * To use this in some new libraries that I'm going to propose
The official zlib at https://github.com/madler/zlib has 109 issues, 156 PRs, one of which is mine, so yes a Boost version would be welcome to me.
On 23.03.21 06:14, Vinnie Falco via Boost wrote:
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)?
Better to have this out in the open than lurking in the belly of the Beast. For me, the big question for a zlib derivative is where it stands in relation to the original zlib in terms of bugs and security. -- Rainer Deyke (rainerd@eldwood.com)
On Tue, Mar 23, 2021 at 1:34 PM Rainer Deyke via Boost
For me, the big question for a zlib derivative is where it stands in relation to the original zlib in terms of bugs and security.
Great question. So, the current implementation in Beast (which will become the starting point for an independent library) started out as a copy of the ZLib sources, followed by my changes to port it to header-only C++. I changed very little although I did rework some lines of code to be more idiomatic and concise using C++ features. I also removed the gzip file format support since Beast websockets did not need it, and it wouldn't have enough testing. The plans for this library in terms of bugs and security include: * Put back the gzip support by porting the relevant original zlib code again * Add Drone CI, GitHub Actions CI for C++11 and later compilers (same as Beast and JSON) * Add Code Coverage * Add Github Actions fuzzing * Manually translate and apply each of the newer commits in the upstream ZLib repository * Refactor statements to be code-coverage friendly * Add concise tests using coverage as guidance (100% coverage target) * Add tests for existing zlib input vectors * Add tests vectors for known attacks * Other tests as deemed useful Additional plans include: * Benchmark program plus CI integration to detect performance regression * Refactor public API to be idiomatic for C++ * Rewrite Javadocs to be readable * Create new exposition to show how to use the library * Add examples for common tasks As with my other libraries, this will be submitted to Bishop Fox for a periodic security review and report. In addition, there will be a dedicated maintainer so that issues can be addressed quickly and the library can be refined. Although I don't imagine there is much room left for innovation beyond cosmetic improvements since ZLib is very mature. Thanks
On Tue, Mar 23, 2021 at 1:34 PM Rainer Deyke via Boost
...
One change I forgot to mention, is that this independent zlib library
will use the new "included source" model for the header-only mode. In
other words you will have to include
wt., 23 mar 2021 o 06:16 Vinnie Falco via Boost
Beast currently has its own port of ZLib to C++:
https://github.com/boostorg/beast/tree/develop/include/boost/beast/zlib
Is there any interest in having this proposed as its own separate Boost library (with a separate review, which has nothing to do with Beast)? The differences would be as follows:
* Ported to use C++11 idioms * Same header-only / compiled source model as Boost.JSON * Better documentation and clear API * Improved tests
Programs which use this C++ port of ZLib would get their zlib dependency from Boost instead of the official zlib distribution. And they can integrate it into their projects the same way that Boost libraries are integrated. Or they can include
in any one translation unit (like JSON).
In my work we currently use ZLib. If at the point of making a decision which library to use there was an option to choose the Boost version, we would definitely go for it. For me, a library under Boost is like a "Boost certificate that guarantees certain things: * The documentation will not be just a list of classes and functions, but will also tell me the limits of the library, how I am supposed to use it, and how not to use it; what problems it solves, and what problems are beyond its scope; how it will interact with other libraries. * I know that the design has included (and documented) things like exception safety and thread safety. (There are good libraries in the wild that never mention if it is safe to call them from two threads). Of course, if a library for compression is extracted from Beast and added as a distinct library in Boost, I would expect that it comes with an interface for compression libraries that makes sense for other compression algorithm implementations. Regards, &rzej;
The motivation for this is twofold:
* Provide an easier way for users to access ZLib's functionality * To use this in some new libraries that I'm going to propose
-- Regards, Vinnie
Follow me on GitHub: https://github.com/vinniefalco
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Well, it looks like there is some interest :) Truth be told, I didn't expect so much interest. This is moving forward! Thanks
participants (11)
-
Andrey Semashev -
Andrzej Krzemienski -
Edward Diener -
Julien Blanc -
Mathias Gaunard -
Peter Barker -
Philip Bennefall -
Rainer Deyke -
sehe -
Takatoshi Kondo -
Vinnie Falco