2013/7/6 Glen Fernandes

Even in Boost, as of 1.53.0, shared_ptr is a better alternative than shared_array<T>.

Why? -- Regards, niXman ___________________________________________________ Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows: http://sourceforge.net/projects/mingwbuilds/ ___________________________________________________ Another online IDE: http://liveworkspace.org/

Gesendet: Samstag, 06. Juli 2013 um 00:49 Uhr Von: "Glen Fernandes" An: boost@lists.boost.org Betreff: Re: [boost] [shared_array] Why not in C++11 ?

...

Does anyone know why 'boost::shared_array<>' was not made part of C++11, since pretty much all the other smart pointers were?

Even in Boost, as of 1.53.0, shared_ptr is a better alternative than shared_array<T>. We just need to get Boost's shared_ptr and shared_ptr to be part of the future standard (instead of adding shared_array to the future standard).

Actually its already supported, but you'll need a custom deleter: http://stackoverflow.com/questions/8624146/c11-standard-scoped-array-wrapper... Still, shared_array might offer the better interface... kind regards, Jens Weller

On 6 July 2013 00:54, Sid Sacek wrote:

...

...

Really? scoped_ptr, scoped_array and intrusive_ptr are part of the standard?

I understand 'unique_ptr' is 'scoped_ptr' with just another name.

No, it's much better :-) It's movable, so you can return it from functions, which is very useful for factories and all sorts of sources and sinks. They're not the same thing.

On Fri, Jul 5, 2013 at 4:24 PM, Jonathan Wakely wrote:

Did anyone ever propose shared_array? If not, how is it supposed to become part of the standard?

boost::shared_array doesn't need to be part of the standard. The extensions to boost::shared_ptr added in Boost 1.53.0 for array types (and the new boost::allocate_shared and boost::make_shared for boost::shared_ptr of array types) need to be part of the standard. :) (boost::shared_ptr has everything boost::shared_array<T> has, and more). Glen --- Software Developer at Microsoft working on distributed build systems, Contributor to Boost

Sid Sacek wrote:

Peter Dimov wrote:

...

As others already said, we only proposed shared_ptr and weak_ptr for TR1 (and then C++11).

A related question is why shared_ptr is not in C++14, and the answer here is that I proposed it just a bit too late for it to > be considered.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3640.html

So then what? C++17 ?

I could be wrong, but it seems too trivial of a modification to push off into a future C++.

There is a cool-off period while C++14 is being finalized, in which no changes are accepted. That's ISO procedure. Once C++14 becomes an official standard, it'd become possible for N3640 to be discussed and, hopefully, accepted; it would then be up to individual compiler vendors to provide an implementation. There'd be no need to actually wait for C++17 for that, but officially, yes, C++17 will be the standard in which the changes will be published (if accepted). The switch to a three-year cycle caught some of us off guard. I should have proposed shared_ptr earlier, but I didn't, so that's where we stand now. Sorry. :-)

Peter Dimov wrote:

...

Once C++14 becomes an official standard, it'd become possible for N3640 to be discussed and, hopefully, accepted;

Same goes for N3641, of course.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3641.html

I tried it out for the first time. Actually, I tried out: auto buffer = boost::make_shared< unsigned char[] >( 1234 ); which created an appropriate: shared_ptr< unsigned char [0] > I was ok with that. I then peeked into all the values stored inside the counter object and even saw the size of the array stored there. I then tried to get the buffer length (in code) but hit a brick wall. int len = buffer.length; // doesn't exist int len = buffer.length(); // doesn't exist int len = buffer.capacity; // doesn't exist int len = buffer.capacity(); // doesn't exist int len = buffer.size; // doesn't exist int len = buffer.size(); // doesn't exist Unless I'm not using the object correctly, not being able to retrieve the array's capacity is a major oversight in my opinion. As a library developer, I want to be passed buffers in this fashion, and know how much data a buffer can hold. Having to guess is not what I was hoping for. Both Java and C# allow you to do this, so why not C++ buffers. -Sid

Peter Dimov wrote:

...

...

...

Once C++14 becomes an official standard, it'd become possible for N3640 to be discussed and, hopefully, accepted;

I then tried to get the buffer length (in code) but hit a brick wall.

int len = buffer.length; // doesn't exist int len = buffer.length(); // doesn't exist int len = buffer.capacity; // doesn't exist int len = buffer.capacity(); // doesn't exist int len = buffer.size; // doesn't exist int len = buffer.size(); // doesn't exist

I thought about it a little, and I guess I could write a wrapper class around shared_ptr<> to provide buffering features.

class shared_buffer : public shared_ptr< unsigned char [] > { size_t capacity; ... }

I can't stop thinking about this. What would be the harm in providing a 'capacity()' member ? buffer = boost::make_shared< unsigned char[] >( 1234 ); int capacity = buffer.capacity(); It can be designed such that that no matter how the new 'shared_ptr' object is instantiated, the size of the array is always known, and hence the capacity should always be available. I don't know how big of a change this is to the existing classes, but it could be made into a requirement. And even if it could not be made a requirement, buffers with unknown sizes would simply return -1. That means that coders who care about the capacity() would supply it in their code, and those who don't care would use the classes in the old fashioned way. -Sid

On Monday 08 July 2013 07:22:53 you wrote:

On Sunday 07 July 2013 21:38:08 Sid Sacek wrote:

...

Peter Dimov wrote:

...

...

...

...

Once C++14 becomes an official standard, it'd become possible for N3640 to be discussed and, hopefully, accepted;

I then tried to get the buffer length (in code) but hit a brick wall.

int len = buffer.length; // doesn't exist int len = buffer.length(); // doesn't exist int len = buffer.capacity; // doesn't exist int len = buffer.capacity(); // doesn't exist int len = buffer.size; // doesn't exist int len = buffer.size(); // doesn't exist

I thought about it a little, and I guess I could write a wrapper class around shared_ptr<> to provide buffering features.

class shared_buffer : public shared_ptr< unsigned char [] > {

size_t capacity; ...

}

I can't stop thinking about this. What would be the harm in providing a 'capacity()' member ?

buffer = boost::make_shared< unsigned char[] >( 1234 );

int capacity = buffer.capacity();

It can be designed such that that no matter how the new 'shared_ptr' object is instantiated, the size of the array is always known, and hence the capacity should always be available.

I don't know how big of a change this is to the existing classes, but it could be made into a requirement. And even if it could not be made a requirement, buffers with unknown sizes would simply return -1. That means that coders who care about the capacity() would supply it in their code, and those who don't care would use the classes in the old fashioned way.

Why not simply use vector or shared_ptr< vector<...> >?

...or even shared_ptr< array<...> >?

Andrey Semashev wrote:

...

Why not simply use vector or shared_ptr< vector<...> >?

...or even shared_ptr< array<...> >?

vector<> v is not shared shared_ptr< vector<> > spv has strange syntax ( *spv )[ 123 ] = blah; shared_ptr< array< 44 > > spa has a fixed size and also strange syntax ( *spa )[ 123 ] = blah; Now watch this: int my_buffer_size = 1234; auto buffer = boost::make_shared< unsigned char[] >( my_buffer_size ); buffer[ index ] = something; int cap = buffer.capacity(); Doesn't that feel the most natural ? -Sid

On Mon, Jul 8, 2013 at 10:26 AM, Glen Fernandes wrote:

Sid Sacek wrote:

You mentioned in an earlier mail that you would prefer that size() return a value like -1 in such a case. That also feels very strange to me; i.e. returning a size_t but reserving a value like -1 to indicate unknown size.

+1, I hate magic values.

Glen Fernandes wrote:

I can see why you desire this. One concern is that boost::shared_ptr is usable without boost::make_shared. With boost::make_shared, yes, in my implementation the size is stored somewhere and changing the machinery could possibly surface that through boost::shared_ptr's interface. But what happens when boost::shared_ptr is used with operator new[]? The size is not stored in that case.

You mentioned in an earlier mail that you would prefer that size() return a value like -1 in such a case. That also feels very strange to me; i.e. returning a size_t but reserving a value like -1 to indicate unknown size.

Glen

My original idea to return -1 was not very good. Zero is much better. I'm new to 'shared_ptr< T[] >'; I don't know how long it's been out there and being used in production code, but if all the rules for shared_ptr< T[] > have not yet been set in stone, then maybe there's room for modifications. My original thought was to make 'size' a requirement for shared_ptr< T[] > since why would anyone use an array without knowing the array's bounds ? Every array has bounds and nobody would touch an array without knowing that information. And placing that information with the array itself makes the most sense to me. If it were made a requirement, then the code might look something like this, or something even more clever: shared_ptr< char[] > buffer( new char[ 10 ], 10 ); The above constructor would set the capacity value to 10. If however, the coder were to simply do this: shared_ptr< char[] > buffer( new char[ 10 ] ); It is clear that the coder has no interest in the capacity information for the buffer, and therefore no other part of that programmer's code will be looking for it either. This would be a non-breaking change, since that is the default behavior of shared_ptr, and the capacity could simply be set to 0. Let me describe how this feature could be used. Imagine your socket library filled user buffers as data arrived. For example: socket::fill_buffer( shared_ptr buffer ) { ... } my_socket-> fill_buffer( my_buffer ); In this case, your library doesn't simply want a shared buffer pointer, but it also needs to know where the buffer ends. Of course you could specify that with extra arguments, which is how it has traditionally been done in C, but with buffer objects, the buffer pointer and its size will be kept atomic. That could prevent buffer overflow bugs from occurring. If a piece of code were to then call 'fill_buffer' with an improperly initialized buffer object, the library would see a zero value for the capacity and throw an exception, or simply return from the call without doing anything. It is the contract of the library's method to be provided with a correctly initialized buffer object. This next piece of code would fail: shared_ptr< char[] > my_buffer( new char[ 10 ] ); my_socket-> fill_buffer( my_buffer ); // OOPS !! improperly initialized buffer !!! When you write this code: shared_ptr< char[] > buffer( new char[ 10 ] ); The buffer object no longer behaves like a regular shared_ptr<>, but behaves very much like an array, and for all intents and purposes, it is an array, it is a buffer! What I mean is, it is not a pointer anymore, and you can no longer do this: buffer-> ???? // what can the pointer operator be used for ???? So, in order to complete these kinds of shared-buffer (shared-array) objects, it only makes sense to tie the capacity together with the buffer pointer. That's my humble opinion. -Sid Sacek

On Jul 8, 2013, at 4:21 AM, Andrey Semashev wrote:

On Mon, Jul 8, 2013 at 11:59 AM, Sid Sacek wrote:

...

My original thought was to make 'size' a requirement for shared_ptr< T[] > since why would anyone use an array without knowing the array's bounds ? Every array has bounds and nobody would touch an array without knowing that information. And placing that information with the array itself makes the most sense to me.

+1

Size could be implied from other components around the pointer. E.g. you could have multiple buffers of the same size encapsulated in a class that maintains this invariant.

That's a very limiting view.

...

If it were made a requirement, then the code might look something like this, or something even more clever:

shared_ptr< char[] > buffer( new char[ 10 ], 10 );

The above constructor would set the capacity value to 10. If however, the coder were to simply do this:

shared_ptr< char[] > buffer( new char[ 10 ] );

It is clear that the coder has no interest in the capacity information for the buffer, and therefore no other part of that programmer's code will be looking for it either. This would be a non-breaking change, since that is the default behavior of shared_ptr, and the capacity could simply be set to 0.

That's error-prone. Why not provide (non-zero) capacity when using make_shared(), and leave the non-make_shared() usage with a zero capacity? That will encourage the use of make_shared() and avoid mistakes.

While that looks feasible, I still would prefer to use a container over a pointer in this case. Providing size manually is error prone at least.

A container is better than manually specifying the capacity, but it isn't shared.

...

Let me describe how this feature could be used. Imagine your socket library filled user buffers as data arrived. For example:

socket::fill_buffer( shared_ptr buffer ) { ... }

my_socket-> fill_buffer( my_buffer );

In this case, your library doesn't simply want a shared buffer pointer, but it also needs to know where the buffer ends. Of course you could specify that with extra arguments, which is how it has traditionally been done in C, but with buffer objects, the buffer pointer and its size will be kept atomic. That could prevent buffer overflow bugs from occurring.

With manual size provision this is still asking for trouble, IMHO. Just use a plain vector and forget about buffer overruns or insufficient buffer sizes.

Right

In fact, you don't share anything here, so by using vector you express your intent more clearly. You may argue about copying the buffer contents or asynchronous operation execution, but move semantics helps here.

Move semantics won't help with async use cases.

...

The buffer object no longer behaves like a regular shared_ptr<>, but behaves very much like an array, and for all intents and purposes, it is an array, it is a buffer! What I mean is, it is not a pointer anymore, and you can no longer do this:

buffer-> ???? // what can the pointer operator be used for ????

The operator-> would work like with any raw pointer - it would return a pointer to the first element of the array.

His point is that, without at least knowing the capacity, code called with a shared_ptr cannot use it because the extent is unknown.

Specializing the interface based on the template argument is not intuitive, IMHO.

We have that already with the subscript operator. Arguably, having only capacity() is insufficient. For cases in which the full extent is not available, size() would be wanted. Where does this end? How many other member functions are needed to cover all reasonable use cases? Perhaps the shared container idea has merit. After all, shared_ptr does ensure delete[] is called, and adding capacity() would provide all original information. ___ Rob

Andrey Semashev and Robert Stewart wrote:

...

...

...

The buffer object no longer behaves like a regular shared_ptr<>, but behaves very much like an array, and for all intents and purposes, it is an array, it is a buffer! What I mean is, it is not a pointer anymore, and you can no longer do this:

buffer-> ???? // what can the pointer operator be used for ????

The operator-> would work like with any raw pointer - it would return a pointer to the first element of the array.

His point is that, without at least knowing the capacity, code called with a shared_ptr cannot use it because the extent is > unknown.

You mean that the pointer may refer to a zero-sized array? Yes, that is UB, just as well as with the raw pointers.

What I mean is, operator->() does not exist. How can it be a pointer when it is not possible to use pointer notation? shared_ptr< char[] > buffer; buffer-> // pointer operation does not exist buffer[ n ] = m; // array indexing does exist Since you can't use pointer notation, it is not a pointer. These objects are arrays, plain and simple. -Sid Sacek

On 8 July 2013 12:20, Andrey Semashev wrote:

On Mon, Jul 8, 2013 at 3:04 PM, Sid Sacek wrote:

...

What I mean is, operator->() does not exist. How can it be a pointer when it is not possible to use pointer notation?

In that case I'd consider this a bug.

It's consistent with the interface of std::unique_ptr. You can use p.get()->x instead of p->x if you really want to use that notation to access the first element only, but how often do you really want to access the first element of an array, and p[0] isn't just as natural?

On Jul 8, 2013, at 6:39 AM, Andrey Semashev wrote:

On Mon, Jul 8, 2013 at 1:49 PM, Rob Stewart wrote:

...

On Jul 8, 2013, at 4:21 AM, Andrey Semashev wrote:

...

Size could be implied from other components around the pointer. E.g. you could have multiple buffers of the same size encapsulated in a class that maintains this invariant.

That's a very limiting view.

Why? I just suggested a use case where embedded size is not needed.

You were using it to suggest the size was not needed for any use case, or at least that's how I read it.

...

...

In fact, you don't share anything here, so by using vector you express your intent more clearly. You may argue about copying the buffer contents or asynchronous operation execution, but move semantics helps here.

Move semantics won't help with async use cases.

Why not? You can move a vector into a bind function, fill it and then move/swap out of the handler.

That depends upon the available interface, I suppose.

...

...

...

The buffer object no longer behaves like a regular shared_ptr<>, but behaves very much like an array, and for all intents and purposes, it is an array, it is a buffer! What I mean is, it is not a pointer anymore, and you can no longer do this:

buffer-> ???? // what can the pointer operator be used for ????

The operator-> would work like with any raw pointer - it would return a pointer to the first element of the array.

His point is that, without at least knowing the capacity, code called with a shared_ptr cannot use it because the extent is unknown.

You mean that the pointer may refer to a zero-sized array? Yes, that is UB, just as well as with the raw pointers.

Without the extent, one cannot safely read or write anything.

...

...

Specializing the interface based on the template argument is not intuitive, IMHO.

We have that already with the subscript operator.

That is not counterintuitive because you can apply the operator to the raw pointer.

Granted, but a deleter is counterintuitive, relative to raw pointers, too. The class name begins with "smart" for a reason. ___ Rob (Sent from my portable computation engine)

Rob Stewart wrote: ...

That's error-prone. Why not provide (non-zero) capacity when using make_shared(), and leave the non-make_shared() usage with a zero capacity?

That's not very convenient. Suppose you get a T* from some place and need to pass shared_ptr to another that requires a known size. You know the size, but you won't be able to pass it. It's also error prone. If you have an interface that takes shared_ptr and size_t, you can tell that the code is passing a size by mere visual inspection. But if the function only takes a shared_ptr, and if shared_ptr might or might not have a size, it's not immediately obvious whether you are passing a size or not. Maybe you are, maybe you aren't; we'll have to wait until runtime and see. Finally, zero is technically a legitimate value for the size, signifying an empty array, so overloading it to denote "unknown" may not be the best option.

On Mon, Jul 8, 2013 at 10:01 AM, Sid Sacek wrote:

Now watch this:

int my_buffer_size = 1234;

auto buffer = boost::make_shared< unsigned char[] >( my_buffer_size );

buffer[ index ] = something;

int cap = buffer.capacity();

Doesn't that feel the most natural ?

I agree that the syntax is simpler but that functionality exceeds that of a pointer, IMHO. Next thing you want is begin()/end() and friends and here we have a full fledged container interface. IIRC, shared_ptr was designed to be as close to raw pointers as possible, but not closer. If you want a container, use one. Containers with shared storage would be a nice addition to Boost, IMHO, but these should not abuse shared_ptr interface. I would rather expect them to be based on Boost.Intrusive and provide the natural container interface with some extensions: shared_vector< int > svi; svi.push_back(10); shared_vector< int > svi2 = svi; // creates a ref-counted alias for svi shared_vector< int > svi3 = svi.clone(); // creates a deep copy of svi shared_map< string, string > smss; smss["aaa"] = "bbb"; shared_map< string, string > smss2 = smss; // also an alias shared_map< string, string > subtree = smss.subtree("aaa"); // creates an alias to the subtree etc. The extensions are a just a thought.

On 8 July 2013 09:22, Sid Sacek wrote:

...

If you want a container, use one. Containers with shared storage would be a nice addition to Boost, IMHO, but these should not abuse shared_ptr interface.

According to what you just wrote 'shared_ptr' has already abused 'shared_ptr<>' and all of the array features should be rolled back, or be put into another class altogether.

I disagree with your implication that shared_ptr abuses anything, maybe your view is skewed by thinking it's trying to be something it isn't. shared_ptr<T> and shared_ptr are similar to T* pointing to a single object, and a T* pointing to an array, with the same limitation that the size is not part of the pointer but with the advantage that you don't have to remember whether to use delete or delete[]. Now you could argue that using pointers to refer to arrays abuses pointers, but that's how C++ works, and it does have many advantages.

In all seriousness, I'm not trying to be a thorn in anybody's side here. All I'm saying is that 'shared_ptr' looks to me like it has come 99% of the way to being a full-blown object-oriented shared-buffer. I'm saying, adding the capacity will simply bring that concept closer to completion.

Oh dear, you used the object-oriented shibboleth and now I hate the idea on principle ;-) unique_ptr doesn't store the size either, which is good as it means sizeof(unique_ptr) == sizeof(T*). When I've wanted to store the size I've either wrapped the unique_ptr (e.g. pair, size_t>) or added the size to the deleter, depending on where and when the size is needed. With shared_ptr that's even more convenient because the deleter isn't part of the type. struct wipeout { size_t m_buflen; void operator()(char* buffer) const { std::memset(buffer, 0, m_buflen); delete [] buffer; } }; typedef boost::shared_ptr WipedBuf; WipedBuf make_WipedBuf(size_t len) { return WipedBuf{ new char[len], wipeout{ len } }; } The size can be retrieved from the deleter if needed: size_t get_array_len(WipedBuf const& p) { auto d = boost::get_deleter<wipeout>(p); return d ? d->m_buflen : 0; } So it's easy enough to add the size yourself if you want it.