Re: [boost] [ Interest? ] [ out_ptr ] Tiny C++ Abstraction for C-Style Output Pointers
On 06/28/18 21:03, ThePhD via Boost wrote:
out_ptr is a small abstraction for making types such as std::unique_ptr, std::shared_ptr, boost::shared_ptr, and the upcoming std::retain_ptr <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0468r0.html> work work C-Style output functions as a parameter, addressing the need of wanting to be able to do "my_c_api_init( foo, bar, &my_smart_ptr );". It's
Does out_ptr have other use-cases than C interoperability?
Dear Bjorn, This does not have any other use case than C and COM-Style API interoperability: both `out_ptr` and `inout_ptr` and streamlined interfaces purely for this purpose and to make writing against such APIs more pleasant, more performant, and (most of all) more developer-time scale-able than alternative approaches such as wrapping all such initialization-style functions in C++-isms. Sincerely, JeanHeyd On Mon, Jul 2, 2018 at 6:20 PM, Bjorn Reese via Boost <boost@lists.boost.org
wrote:
On 06/28/18 21:03, ThePhD via Boost wrote:
out_ptr is a small abstraction for making types such as
std::unique_ptr, std::shared_ptr, boost::shared_ptr, and the upcoming std::retain_ptr <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0468r0.html> work work C-Style output functions as a parameter, addressing the need of wanting to be able to do "my_c_api_init( foo, bar, &my_smart_ptr );". It's
Does out_ptr have other use-cases than C interoperability?
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
On 3/07/2018 14:42, ThePhD wrote:
This does not have any other use case than C and COM-Style API interoperability: both `out_ptr` and `inout_ptr` and streamlined interfaces purely for this purpose and to make writing against such APIs more pleasant, more performant, and (most of all) more developer-time scale-able than alternative approaches such as wrapping all such initialization-style functions in C++-isms.
Purely for bikeshedding purposes a name like c_out_ptr might be better in that case. When initially seeing "out_ptr" and the syntax in your original email, at first glance it looked like the intent was as a reference wrapper annotation (to make it more obvious at the call site that the pointer will be modified on return), rather than the typical "solution" of using an /*out*/ comment or similar non-functional and non-checked annotation. (I suppose you can use std::ref for that purpose, but the intent is less clear. And perhaps less necessary once compilers and libraries start using multiple return values via structured binding.)
Dear Gavin, In the proposal linked as part the P.S., that's actually one of the names listed. It was also recommended by another person outside this list, so I will keep that in mind going forward (the pair of names was `c_out_ptr` and `c_inout_ptr`)! This library is not explicitly an annotation, but it also doubles as that by its design: the free function makes it very noticeable that this parameter is an output parameter. Sincerely, JeanHeyd On Tue, Jul 3, 2018 at 12:15 AM, Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 3/07/2018 14:42, ThePhD wrote:
This does not have any other use case than C and COM-Style API interoperability: both `out_ptr` and `inout_ptr` and streamlined interfaces purely for this purpose and to make writing against such APIs more pleasant, more performant, and (most of all) more developer-time scale-able than alternative approaches such as wrapping all such initialization-style functions in C++-isms.
Purely for bikeshedding purposes a name like c_out_ptr might be better in that case.
When initially seeing "out_ptr" and the syntax in your original email, at first glance it looked like the intent was as a reference wrapper annotation (to make it more obvious at the call site that the pointer will be modified on return), rather than the typical "solution" of using an /*out*/ comment or similar non-functional and non-checked annotation.
(I suppose you can use std::ref for that purpose, but the intent is less clear. And perhaps less necessary once compilers and libraries start using multiple return values via structured binding.)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
First of all, this library addiion IMHO is a great idea. On Tue, 3 Jul 2018 at 15:04, ThePhD via Boost <boost@lists.boost.org> wrote:
Dear Gavin,
In the proposal linked as part the P.S., that's actually one of the names listed. It was also recommended by another person outside this list, so I will keep that in mind going forward (the pair of names was `c_out_ptr` and `c_inout_ptr`)!
Which begs the question, couldn't objects be created by fortran, python, the node v8 engine, c#, java, assembler, etc? In this regard it seems to me that your existing names are better since they describe intent rather than implementation.
This library is not explicitly an annotation, but it also doubles as that by its design: the free function makes it very noticeable that this parameter is an output parameter.
Sincerely, JeanHeyd
On Tue, Jul 3, 2018 at 12:15 AM, Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 3/07/2018 14:42, ThePhD wrote:
This does not have any other use case than C and COM-Style API interoperability: both `out_ptr` and `inout_ptr` and streamlined interfaces purely for this purpose and to make writing against such APIs more pleasant, more performant, and (most of all) more developer-time scale-able than alternative approaches such as wrapping all such
initialization-style
functions in C++-isms.
Purely for bikeshedding purposes a name like c_out_ptr might be better in that case.
When initially seeing "out_ptr" and the syntax in your original email, at first glance it looked like the intent was as a reference wrapper annotation (to make it more obvious at the call site that the pointer will be modified on return), rather than the typical "solution" of using an /*out*/ comment or similar non-functional and non-checked annotation.
(I suppose you can use std::ref for that purpose, but the intent is less clear. And perhaps less necessary once compilers and libraries start using multiple return values via structured binding.)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 4/07/2018 01:04, ThePhD wrote:
In the proposal linked as part the P.S., that's actually one of the names listed. It was also recommended by another person outside this list, so I will keep that in mind going forward (the pair of names was `c_out_ptr` and `c_inout_ptr`)!
This library is not explicitly an annotation, but it also doubles as that by its design: the free function makes it very noticeable that this parameter is an output parameter.
I guess my question is whether it could be used as a within-C++ annotation as well, eg: Caller: SmartPtr my_smart_ptr; auto r = some_function(foo, bar, out_ptr(my_smart_ptr)); Callee: bool some_function(int foo, int bar, out_ptr<SmartPtr> local_smart_ptr) { ... local_smart_ptr.reset(x); and/or local_smart_ptr = x; ... } where the out_ptr itself is received by value (cheap to copy or move since it just contains a reference to the actual smart pointer), but offers a way to alter the value of my_smart_ptr through modification of local_smart_ptr -- as close as possible to the behaviour of the argument being type SmartPtr&, but requiring the explicit annotation at the call site rather than being invisible. While serving both roles obviously can't be a requirement (I'm not sure if it's even possible), if this can be done then it would truly be deserving of a generic name like "out_ptr".
Dear Gavin, I am not sure whether this provides much more benefit than just passing the smart pointer itself: the `some_function` already sticks `SmartPtr` in its signature (so you might as well just pass the SmartPtr itself, since you've already nailed the smart pointer to the signature). Having to call `.reset()` manually here through `out_ptr` means that if we want to perform optimizations, we'd likely have to have some way of keeping track of "already reset" or "already manipulated" (this is why `out_ptr` in performance tests doesn't meet or exceed the performance of hand-written C code to manipulate these pointers, and why `inout_ptr` does: see the performance graphs here -- https://github.com/ThePhD/out_ptr#benchmarks). There might be room for a type-erased version of `out_ptr` to make working with smart pointers at an API/ABI boundary simpler, but I think that goes outside the scope of `out_ptr` and -- indeed -- further outside the scope of my knowledge...! Sincerely, JeanHeyd On Tue, Jul 3, 2018 at 7:21 PM, Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 4/07/2018 01:04, ThePhD wrote:
In the proposal linked as part the P.S., that's actually one of the names listed. It was also recommended by another person outside this list, so I will keep that in mind going forward (the pair of names was `c_out_ptr` and `c_inout_ptr`)!
This library is not explicitly an annotation, but it also doubles as that by its design: the free function makes it very noticeable that this parameter is an output parameter.
I guess my question is whether it could be used as a within-C++ annotation as well, eg:
Caller: SmartPtr my_smart_ptr; auto r = some_function(foo, bar, out_ptr(my_smart_ptr));
Callee: bool some_function(int foo, int bar, out_ptr<SmartPtr> local_smart_ptr) { ... local_smart_ptr.reset(x); and/or local_smart_ptr = x; ... }
where the out_ptr itself is received by value (cheap to copy or move since it just contains a reference to the actual smart pointer), but offers a way to alter the value of my_smart_ptr through modification of local_smart_ptr -- as close as possible to the behaviour of the argument being type SmartPtr&, but requiring the explicit annotation at the call site rather than being invisible.
While serving both roles obviously can't be a requirement (I'm not sure if it's even possible), if this can be done then it would truly be deserving of a generic name like "out_ptr".
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
On 4/07/2018 13:48, ThePhD wrote:
I am not sure whether this provides much more benefit than just passing the smart pointer itself: the `some_function` already sticks `SmartPtr` in its signature (so you might as well just pass the SmartPtr itself, since you've already nailed the smart pointer to the signature).
The benefit is, as I said, in making it obvious at the call-site that the pointer can be modified. The alternative is: Caller: SmartPtr my_smart_ptr; auto r = some_function(foo, bar, /*out*/ my_smart_ptr); Callee: bool some_function(int foo, int bar, SmartPtr& local_smart_ptr); Where obviously the comment is trying to be an annotation, but as it's just a comment it has no real effect and is easily forgotten and then there is no evidence at all (at the call site) that my_smart_ptr can be modified by the call. It's not really a pointer-specific issue, it's just a flaw in C++'s reference-passing syntax in general. Some other languages handle this better by being more explicit about the purpose of a reference parameter. So it would be nice if there were a way to have a compiler-enforced annotation, as long as it doesn't have negative performance impact. As I said, this is not really in-scope for what you're trying to do with your library, but it's a related problem space and it would be nice if one solution could solve both problems uniformly.
Dear Gavin, I think the solution for that is moreso what Jonathan Müller is proposing with his type_safe library. There's a ts::output_parameter type in there that makes your API extremely explicit. Again, this isn't really supposed to affect the callee: only the caller side of things. I don't think `out_ptr` can be shifted over to handle that responsibility without getting very, very far off track. Maybe `smart_out_ptr` and `smart_inout_ptr`/ (or replace 'smart' with 'fancy') might make the intention more clear, at the cost of being more verbose. 'c_inout_ptr' and 'c_out_ptr` are more cromulent, and generally capture what this API ends up being used for (C and COM-style APIs). My original implementation used `ptrptr` and `in_ptrptr`, as in "pointer to pointer", but I'm not sure anyone liked that name very much except for me...! Of course, the other authors in the official standards proposal proposed things like `out_ptr` and `inout_ptr`, as well as `c_ptr` and `c_in_ptr`. We went with `out_ptr` and `inout_ptr`. Sincerely, JeanHeyd On Tue, Jul 3, 2018 at 10:17 PM, Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 4/07/2018 13:48, ThePhD wrote:
I am not sure whether this provides much more benefit than just passing the smart pointer itself: the `some_function` already sticks `SmartPtr` in its signature (so you might as well just pass the SmartPtr itself, since you've already nailed the smart pointer to the signature).
The benefit is, as I said, in making it obvious at the call-site that the pointer can be modified. The alternative is:
Caller: SmartPtr my_smart_ptr; auto r = some_function(foo, bar, /*out*/ my_smart_ptr);
Callee: bool some_function(int foo, int bar, SmartPtr& local_smart_ptr);
Where obviously the comment is trying to be an annotation, but as it's just a comment it has no real effect and is easily forgotten and then there is no evidence at all (at the call site) that my_smart_ptr can be modified by the call.
It's not really a pointer-specific issue, it's just a flaw in C++'s reference-passing syntax in general. Some other languages handle this better by being more explicit about the purpose of a reference parameter.
So it would be nice if there were a way to have a compiler-enforced annotation, as long as it doesn't have negative performance impact.
As I said, this is not really in-scope for what you're trying to do with your library, but it's a related problem space and it would be nice if one solution could solve both problems uniformly.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
Dear Boost Community, I wanted to follow-up on this library interest. Because interoperation with `boost::move::unique_ptr` would need some coordination, it was suggested I put this underneath the boost::move library. I have opened an issue to gather the feeling of the maintainer about such a move (heh!): https://github.com/boostorg/move/issues/22 I'll be doing my best to make this available to boost or the open source community in some fashion. Sincerely, JeanHeyd Meneide On Wed, Jul 4, 2018 at 6:29 AM ThePhD <phdofthehouse@gmail.com> wrote:
Dear Gavin,
I think the solution for that is moreso what Jonathan Müller is proposing with his type_safe library. There's a ts::output_parameter type in there that makes your API extremely explicit. Again, this isn't really supposed to affect the callee: only the caller side of things. I don't think `out_ptr` can be shifted over to handle that responsibility without getting very, very far off track.
Maybe `smart_out_ptr` and `smart_inout_ptr`/ (or replace 'smart' with 'fancy') might make the intention more clear, at the cost of being more verbose. 'c_inout_ptr' and 'c_out_ptr` are more cromulent, and generally capture what this API ends up being used for (C and COM-style APIs). My original implementation used `ptrptr` and `in_ptrptr`, as in "pointer to pointer", but I'm not sure anyone liked that name very much except for me...!
Of course, the other authors in the official standards proposal proposed things like `out_ptr` and `inout_ptr`, as well as `c_ptr` and `c_in_ptr`. We went with `out_ptr` and `inout_ptr`.
Sincerely, JeanHeyd
On Tue, Jul 3, 2018 at 10:17 PM, Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 4/07/2018 13:48, ThePhD wrote:
I am not sure whether this provides much more benefit than just passing the smart pointer itself: the `some_function` already sticks `SmartPtr` in its signature (so you might as well just pass the SmartPtr itself, since you've already nailed the smart pointer to the signature).
The benefit is, as I said, in making it obvious at the call-site that the pointer can be modified. The alternative is:
Caller: SmartPtr my_smart_ptr; auto r = some_function(foo, bar, /*out*/ my_smart_ptr);
Callee: bool some_function(int foo, int bar, SmartPtr& local_smart_ptr);
Where obviously the comment is trying to be an annotation, but as it's just a comment it has no real effect and is easily forgotten and then there is no evidence at all (at the call site) that my_smart_ptr can be modified by the call.
It's not really a pointer-specific issue, it's just a flaw in C++'s reference-passing syntax in general. Some other languages handle this better by being more explicit about the purpose of a reference parameter.
So it would be nice if there were a way to have a compiler-enforced annotation, as long as it doesn't have negative performance impact.
As I said, this is not really in-scope for what you're trying to do with your library, but it's a related problem space and it would be nice if one solution could solve both problems uniformly.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Dear Boost Community, Just to keep everyone up to date, the discussion has moved over to Boost.Smartptr as we feel that this would be better made available through that module: https://github.com/boostorg/smart_ptr/issues/56 If this does seem acceptable, I will move to also add a friend declaration to boost::move::unique_ptr, so that individuals using that library will get the same performance benefits listed in the proposal being sent for standardization <https://thephd.github.io/vendor/future_cxx/papers/d1132.html>. Sincerely, JeanHeyd Meneide On Mon, Nov 12, 2018 at 7:05 PM JeanHeyd Meneide <phdofthehouse@gmail.com> wrote:
Dear Boost Community,
I wanted to follow-up on this library interest. Because interoperation with `boost::move::unique_ptr` would need some coordination, it was suggested I put this underneath the boost::move library. I have opened an issue to gather the feeling of the maintainer about such a move (heh!): https://github.com/boostorg/move/issues/22
I'll be doing my best to make this available to boost or the open source community in some fashion.
Sincerely, JeanHeyd Meneide
On Wed, Jul 4, 2018 at 6:29 AM ThePhD <phdofthehouse@gmail.com> wrote:
Dear Gavin,
I think the solution for that is moreso what Jonathan Müller is proposing with his type_safe library. There's a ts::output_parameter type in there that makes your API extremely explicit. Again, this isn't really supposed to affect the callee: only the caller side of things. I don't think `out_ptr` can be shifted over to handle that responsibility without getting very, very far off track.
Maybe `smart_out_ptr` and `smart_inout_ptr`/ (or replace 'smart' with 'fancy') might make the intention more clear, at the cost of being more verbose. 'c_inout_ptr' and 'c_out_ptr` are more cromulent, and generally capture what this API ends up being used for (C and COM-style APIs). My original implementation used `ptrptr` and `in_ptrptr`, as in "pointer to pointer", but I'm not sure anyone liked that name very much except for me...!
Of course, the other authors in the official standards proposal proposed things like `out_ptr` and `inout_ptr`, as well as `c_ptr` and `c_in_ptr`. We went with `out_ptr` and `inout_ptr`.
Sincerely, JeanHeyd
On Tue, Jul 3, 2018 at 10:17 PM, Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 4/07/2018 13:48, ThePhD wrote:
I am not sure whether this provides much more benefit than just passing the smart pointer itself: the `some_function` already sticks `SmartPtr` in its signature (so you might as well just pass the SmartPtr itself, since you've already nailed the smart pointer to the signature).
The benefit is, as I said, in making it obvious at the call-site that the pointer can be modified. The alternative is:
Caller: SmartPtr my_smart_ptr; auto r = some_function(foo, bar, /*out*/ my_smart_ptr);
Callee: bool some_function(int foo, int bar, SmartPtr& local_smart_ptr);
Where obviously the comment is trying to be an annotation, but as it's just a comment it has no real effect and is easily forgotten and then there is no evidence at all (at the call site) that my_smart_ptr can be modified by the call.
It's not really a pointer-specific issue, it's just a flaw in C++'s reference-passing syntax in general. Some other languages handle this better by being more explicit about the purpose of a reference parameter.
So it would be nice if there were a way to have a compiler-enforced annotation, as long as it doesn't have negative performance impact.
As I said, this is not really in-scope for what you're trying to do with your library, but it's a related problem space and it would be nice if one solution could solve both problems uniformly.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Sun, Nov 18, 2018 at 7:23 PM JeanHeyd Meneide via Boost < boost@lists.boost.org> wrote:
Dear Boost Community,
Just to keep everyone up to date, the discussion has moved over to Boost.Smartptr as we feel that this would be better made available through that module: https://github.com/boostorg/smart_ptr/issues/56
If this does seem acceptable, I will move to also add a friend declaration to boost::move::unique_ptr, so that individuals using that library will get the same performance benefits listed in the proposal being sent for standardization <https://thephd.github.io/vendor/future_cxx/papers/d1132.html>.
I want to encourage anyone who needs smart pointer/C API interoperability to read that paper. I for one did not get this paper until I saw it presented in the last committee meeting, but I find it to be very useful now that I understand it. Zach
Dear Boost Community, I have been working with both Boost and Standards Committee members for the past 6 montha to deliver an implementation of out_ptr, inout_ptr, and associated customization points to both the standard and Boost. Currently, there is a (new) public implementation using only Boost and consumable from C++11 compilers: https://github.com/ThePhD/out_ptr . As a recap of what has happened: - An attempt was made to put an implementation of this into Boost.Move, partly to ensure a friend-implementation of the customization point for boost::movelib::unique_ptr could be done for performance reasons. It was suggested to try Boost.SmartPtr instead. - A discussion of implementation, specification, performance and suitability was undertaken by myself, Mr. Peter Dimov and Mr. Glen Fernandes for Boost.SmartPtr while the proposal was moved through the standard: https://github.com/boostorg/smart_ptr/issues/56 . An implementation was made with different design choices and that experience was passed on to me, which I incorporated into the current C++11 design. - I privately created and used an implementation and shared it with others and their codebases, gaining feedback. - For C++Now Library in a w Week, I focused on creating a version of Boost.out_ptr suitable for inclusion into Boost. There was an initial in-person review. The result of that review is documented in the rationale: https://github.com/ThePhD/out_ptr/blob/master/docs/out_ptr/rationale.adoc - There is now a public boost-ready (I feel) implementation. In a week or two, I feel like -- given the amount of input on the design and specification by seversl boost and non-boost members, shows of curiosity and support in person, twitter, and on Slack, and feedback about documentation from several users -- I should prepare to move into a Review Queue by getting such a motion seconded. Do let me know if the level of interest for this library in Boost is still present and if taking the next steps is justified! It would be nice to have a C++11 implementation in Boost even after this finishes landing for C++20. Sincerely, JeanHeyd Meneide
On Sat, May 11, 2019 at 5:50 PM JeanHeyd Meneide via Boost <boost@lists.boost.org> wrote:
Currently, there is a (new) public implementation using only Boost and consumable from C++11 compilers: https://github.com/ThePhD/out_ptr .
I cloned the repository, configured the CMake, and generated a Visual Studio 2017 solution and project files. Everything built correctly although I could not figure out how to run the tests, because the generated solution and projects, while correct - are not friendly for someone who spends most of their development time isolated in the IDE (like me). Looks like nice work so far, but a disclaimer applies: I personally have no use-case for out_ptr so I don't have a strong opinion on it. Regards
On Sat, May 11, 2019 at 10:14 PM Vinnie Falco <vinnie.falco@gmail.com> wrote:
On Sat, May 11, 2019 at 5:50 PM JeanHeyd Meneide via Boost <boost@lists.boost.org> wrote:
Currently, there is a (new) public implementation using only Boost and consumable from C++11 compilers: https://github.com/ThePhD/out_ptr .
I cloned the repository, configured the CMake, and generated a Visual Studio 2017 solution and project files. Everything built correctly although I could not figure out how to run the tests, because the generated solution and projects, while correct - are not friendly for someone who spends most of their development time isolated in the IDE (like me).
Looks like nice work so far, but a disclaimer applies: I personally have no use-case for out_ptr so I don't have a strong opinion on it.
Regards
I forgot to add some documentation for running the tests: you pass the CMake Flags BOOST_OUT_PTR_TESTS=ON and BOOST_OUT_PTR_EXAMPLES=ON, then run CMake and everything generates. I didn't document it because I didn't think it'd be useful, because at the end of the day in order to use Boost.CI and friends I need to learn Jam and b2 and all that stuff. Best, JeanHeyd
participants (7)
-
Bjorn Reese -
Gavin Lambert -
JeanHeyd Meneide -
Richard Hodges -
ThePhD
-
Vinnie Falco -
Zach Laine