On 29/07/14 06:46, Edward Diener wrote:

On 7/28/2014 11:22 PM, Chris Cleeland wrote:

...

I'm resurrecting this topic:

https://groups.google.com/forum/#!searchin/boost-list/shared_ptr$20void*/boo...

because there is an unaddressed issue in it that I'm about to run into:

"... beware the pointee may be deleted if the last copy of the shared pointer goes out of scope, as the raw pointer won't have incremented the reference count."

I cannot be the first person to run into this--a common issue when crossing an API boundary. Has anyone come up with a good, safe solution wherein the reference count gets properly incremented?

The issue has nothing to do with casting a shared pointer to a void *. If you need to pass a shared pointer as any raw pointer, if the last use of the shared pointer ends the encapsulated raw pointer is deleted. If someone is still holding on to that raw pointer after that happens and tries to use it you will almost cetainly get a crash.

There is no good solution to this other than not using a shared pointer in the first place in your particular situation. Think about your problem. You want to be able to share this pointer among other objects but then you want one or more objects to share this pointer as a raw pointer. Given what a shared pointer is, doing this just does not make sense.

I concur. If the life time of the object needs to outlast the life time of the last shared pointer pointing to it the use of shared pointers is rather pointless. Otherwise it is safe to use get() and to pass the returned raw pointer. Regards, Leon

On Tue, Jul 29, 2014 at 4:49 PM, Chris Cleeland wrote:

There is no good solution to this other than not using a shared pointer in

...

the first place in your particular situation. Think about your problem. You want to be able to share this pointer among other objects but then you want one or more objects to share this pointer as a raw pointer. Given what a shared pointer is, doing this just does not make sense.

I will have to disagree with your assessment that using a shared pointer makes no sense. In my particular situation, the shared pointer is used in a C++ application, and it's a poster child for shared pointers. This mismatch occurs in a couple of places where I have to call into a C API and register a callback, providing a "context" or "cookie" or "Asynchronous Completion Token" to the registration. That context gets held in some container owned by the C API innards and, when the C API invokes the callback, it passes that context as an argument to the callback function.

Ideally, the context would be the pointer to the same object held by the shared pointer--because that is the exact context I need in order to take action within the callback function.

You just new (heap alloc) a shared_ptr (as a copy of your existing shared_ptr), and pass its address as the "user-data" as a void*, which the callback casts back into the shared_ptr*, and deletes that one (makes it go "out of scope", and if it's the last one, takes the pointed-object with him to the grave). The heap-allocated shared_ptr gives you the ref-count bump you need to keep the object alive. If your C API does not "leak", calls the callback appropriately, you don't leak anything. I used the technique with SQLite. --DD

On Tue, Jul 29, 2014 at 11:02 AM, Michael Powell wrote:

Along the lines what Edward suggested, if the "origin" of the pointer is your thread, then somehow you need to make sure that memory lives beyond the lifetime of the external, in this case C, API, context and all, etc, etc. No room for debate in that topic; it is what it is, deal with it in a responsible manner.

Yes, I agree that I'm responsible. If I didn't know or think that, then I wouldn't have asked the question in the first place. I'm not debating whether or not I *should* be responsible--I'm asking a question related to best-practices for fulfilling that responsibility in the absence of the ability to use a shared pointer and its concomitant semantics everywhere the pointer is held. While I appreciate your response and Edward's response, neither answers the question asked. Thanks to Dominique for sharing a technique. I hadn't thought of using a heap-allocated shared pointer instance as the proxy. I had thought of stashing another shared pointer instance in the C++ object that manages the callback registration itself, but wasn't sure if that lifecycle would coincide exactly. The heap is probably the better of the two since it dissociates all lifecycles.

On Tue, Jul 29, 2014 at 11:30 AM, Michael Powell wrote:

On Tue, Jul 29, 2014 at 11:25 AM, Chris Cleeland wrote:

...

On Tue, Jul 29, 2014 at 11:02 AM, Michael Powell wrote:

...

Along the lines what Edward suggested, if the "origin" of the pointer is your thread, then somehow you need to make sure that memory lives beyond the lifetime of the external, in this case C, API, context and all, etc, etc. No room for debate in that topic; it is what it is, deal with it in a responsible manner.

The C API is not managing the life time of the instance allocated to the shared_ptr. When the shared_ptr<> falls out of scope, all bets are off. Plain and simple.

Not to beat a dead horse. But here's an apt analogy: I could be wrong, but you are dividing by zero and not expecting a div by zero exception (or worse) to occur.

...

Yes, I agree that I'm responsible. If I didn't know or think that, then I wouldn't have asked the question in the first place.

I'm not debating whether or not I *should* be responsible--I'm asking a question related to best-practices for fulfilling that responsibility in the absence of the ability to use a shared pointer and its concomitant semantics everywhere the pointer is held. While I appreciate your response and Edward's response, neither answers the question asked.

Thanks to Dominique for sharing a technique. I hadn't thought of using a heap-allocated shared pointer instance as the proxy. I had thought of stashing another shared pointer instance in the C++ object that manages the callback registration itself, but wasn't sure if that lifecycle would coincide exactly. The heap is probably the better of the two since it dissociates all lifecycles.

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

On 7/29/14, 10:15 PM, Gavin Lambert wrote:

On 30/07/2014 14:01, Michael Powell wrote:

...

Not to beat a dead horse. But here's an apt analogy: I could be wrong, but you are dividing by zero and not expecting a div by zero exception (or worse) to occur.

I don't see how that's even slightly a valid analogy.

In any case, what the OP needs to do is to characterise the lifetime of the raw pointer as used by the C API, and *absolutely guarantee* that some shared_ptr exists at least as long as that.

There are various techniques for doing this, depending on what that lifetime is -- for example, if it's a single-shot callback then Dominique's suggestion might be reasonable, if you don't mind paying an extra allocation/deallocation each cycle. If it's a multi-shot callback then it may be better to make a connection-management object that registers/unregisters with the C API and then manage the lifecycle of *that* object appropriately. There's many methods.

But you have to know what the C API is going to be doing with that pointer, or all bets are off.

Another way of looking at it is that a C program that gives this API a pointer, needs to make sure that the pointer remains valid until the library is done with it. In C++ this doesn't change. The difference is that in C, if this is a heap allocation, you need to explicitly decide to destroy the block, while in C++, if it has been put under control of a shared_ptr, it may automatically go away based on the rules of shared_ptr. Thus the program needs to let shared_ptr know that the block is still in use, and one simple way of doing this is to make a shared_ptr that will exist as long as you need the block to continue. This happens anytime a pointer to an object under the control of shared_ptr move outside the control of shared_ptr. -- Richard Damon

On 30/07/2014 15:32, Chris Cleeland wrote:

In my case, it's a multi-shot callback. I don't think I'd need to do a heap allocation/deallocation each cycle, but I'm also not sure that I want to do a heap allocation in the first place.

But you have to know what the C API is going to be doing with that pointer, or all bets are off.

In my case, the C API is stashing the value of the pointer in a container, and passing that pointer value as an argument to the registered callback function. That's it. The C code doesn't *do* anything with the pointer because it treats it as a void*.

You're going to need a heap allocation *somewhere*. I don't think there's any way you can avoid that. It's just a question of whether you already have an existing heap object with the right lifetime or whether you need to make a new one. For a multi-shot callback, you will need to have some raw pointer you can guarantee will remain valid from C API registration to unregistration. One way to do this would be to make an object responsible for registration/unregistration, which forces you to keep this object alive as long as you want the registration to last -- this means that this object can itself be the void* for your C API: class SomeExternalApiRegistration { public: SomeExternalApiRegistration() { RegisterSomeExternalApi( &SomeExternalApiRegistration::DoSomethingC, this); } ~SomeExternalApiRegistration() { UnregisterSomeExternalApi( &SomeExternalApiRegistration::DoSomethingC, this); } private: static void DoSomethingC(void *arg) { static_cast(arg)->DoSomethingCpp(); } void DoSomethingCpp() { // ... } }; In order to actually do whatever it is this needs to, it can have shared_ptr or weak_ptr members, possibly even delegating all the work to them. Just remember that something has to own the above object, and if it's the same object that's going to get called, then you need a weak_ptr. (Things get trickier in a concurrent environment -- you have to know that unregistering will block until any in-progress calls are completed or cancelled, or hilarity may ensue.) I usually prefer doing something like the above as it nicely follows RAII. But you could instead allocate some functor/container object (or the shared_ptr itself) on registration, provide a similar invoker to the above, and deallocate the object on deregistration. But that ends up looking very similar, because you usually need a way to remember which object to deallocate on unregistration. I don't think doing it that way is useful unless you need something single-shot.

On 30/07/2014 19:23, Dominique Devienne wrote:

On Wed, Jul 30, 2014 at 5:32 AM, Chris Cleeland mailto:chris.cleeland@gmail.com> wrote:

In my case, the C API is stashing the value of the pointer in a container, and passing that pointer value as an argument to the registered callback function. That's it. The C code doesn't *do* anything with the pointer because it treats it as a void*.

Then it's a badly-designed C API I'm afraid...

Not necessarily. For a single-shot callback, the simplest way to manage lifetime properly is to guarantee that the callback is called exactly once (always called, even on failure), at least if the initial operation reports success. This gives the caller a chance to clean up the object if required. For a multi-shot callback, the caller is usually expected to separately unregister the callback when it's done, or register a separate-but-related "cleanup/destroy" single-shot callback (depending on which end is more likely to trigger shutdown).

SQLite introduced a bunch of _v2 APIs (e.g. [1]) specifically to add a void(*xDestroy)(void*) argument to some of its functions taking a void* user-data argument, to address lifetime issues. That's C programming done right, which interfaces just fine with C++ and shared_ptr and co.

That cited example doesn't seem like a good design to me, but rather an afterthought. (I'm not referring to the _v2 bit, I mean that sqlite3_module contains a mixture of static and instance methods and the user-data destructor should have been one of the static methods, or instead of using user-data or static methods there should have been a defined module base type.) Probably they were hampered by backwards-compatibility issues, but I still wouldn't call it "done right". :) But this is getting off-topic and into bikeshed territory.

On Tue, Jul 29, 2014 at 9:01 PM, Michael Powell wrote:

On Tue, Jul 29, 2014 at 11:30 AM, Michael Powell wrote:

...

On Tue, Jul 29, 2014 at 11:25 AM, Chris Cleeland wrote:

...

On Tue, Jul 29, 2014 at 11:02 AM, Michael Powell

...

wrote:

...

Along the lines what Edward suggested, if the "origin" of the pointer is your thread, then somehow you need to make sure that memory lives beyond the lifetime of the external, in this case C, API, context and all, etc, etc. No room for debate in that topic; it is what it is, deal with it in a responsible manner.

The C API is not managing the life time of the instance allocated to the shared_ptr. When the shared_ptr<> falls out of scope, all bets are off. Plain and simple.

Not to beat a dead horse. But here's an apt analogy: I could be wrong, but you are dividing by zero and not expecting a div by zero exception (or worse) to occur.

That's actually a completely incorrect analogy. We are obviously having a disconnect between the explanation I'm giving for what I'm attempting to deal with and your understanding of it. I could tell earlier today that this had occurred, but with a schedule deadline looming I didn't think it was worth wrestling with the so-called pig (to use another analogy that may or may not be correct). You seem to think that I want to abdicate responsibility for managing the lifecycle, and that's completely the OPPOSITE of what I want to do. The whole reason I raised the question in the first place is because I recognize and WANT to deal with lifecycle responsibilities. Ideally, the container behind the C API would participate in bumping the reference count on my pointer, but that's not realistic because (a) it's not my API--it's from a vendor and not open source and (b) it's C and participating in shared_ptr<> refcount semantics in C does not look trivial. So, I am left with figuring out a different way--some object on the C++ side whose lifecycle matches that of the container behind the C API, either by coincidence or by design. Hopefully I've done a better job explaining this time. -- Chris Cleeland