How to get the size of an C-array?
Hi, On TBoost.STM, I need to do something before deleting an array on each one of the elements of the array, or more precissely with the address of each one of the elements. T* ptr = new T[3]; // in another file, the number of elements to which ptr points is unknown. // before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr; As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements. The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting the pointer. Is there something in Boost or a portable way to recover the number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable? Any hint is welcome. Best, _____________________ Vicente Juan Botet Escribá
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on each one of the elements of the array, or more precissely with the address of each one of the elements.
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is unknown. // before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting the pointer. Is there something in Boost or a portable way to recover the number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable?
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this: 1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose. 2. Use a range wrapper instead of a raw pointer to the array. Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
Hi Andrey, ----- Original Message ----- From: "Andrey Semashev" <andrey.semashev@gmail.com> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 7:18 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on each one of the elements of the array, or more precissely with the address of each one of the elements.
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is unknown. // before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting the pointer. Is there something in Boost or a portable way to recover the number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable?
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
2. Use a range wrapper instead of a raw pointer to the array.
This is equivalent to say that we can not use dynamic arrays with the STM library. Before to abandoning I want to explore all the posibilities.
Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size. The drawback of using specific functions is that we will need to educate the user, and yet more important, we couldn't use generic classes that make use of these standard operators, as e.g. auto_ptr or scoped_ptr. The problem I find overloading new[] is that I dont know where my prefixed information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information. pointer returned by new[] | v | my_prefic | std prefix | user area | The parameter size_t of the new[] operator is the addition of sizes of std prefix and the user area. While writing these lines, I think that maybe the size of std prefix can be calculated requesting the creation of an array with only one element, so we can calculate the sizeof(std prefix) = requested size for new T[1] - sizeof(T) Does this make sens? Would this be portable? Anyway, thanks for your comments Andrey, regards, Vicente
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of vicente.botet Sent: Wednesday, October 14, 2009 4:54 PM To: boost@lists.boost.org Subject: Re: [boost] How to get the size of an C-array?
Hi Andrey, ----- Original Message ----- From: "Andrey Semashev" <andrey.semashev@gmail.com> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 7:18 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on each
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is
unknown.
// before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting
one of the elements of the array, or more precissely with the address of each one of the elements. the pointer. Is there something in Boost or a portable way to recover the number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable?
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
2. Use a range wrapper instead of a raw pointer to the array.
This is equivalent to say that we can not use dynamic arrays with the STM library. Before to abandoning I want to explore all the posibilities.
Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size. The drawback of using specific functions is that we will need to educate the user, and yet more important, we couldn't use generic classes that make use of these standard operators, as e.g. auto_ptr or scoped_ptr.
The problem I find overloading new[] is that I dont know where my prefixed information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information.
pointer returned by new[] | v | my_prefic | std prefix | user area |
The parameter size_t of the new[] operator is the addition of sizes of std prefix and the user area.
While writing these lines, I think that maybe the size of std prefix can be calculated requesting the creation of an array with only one element, so we can calculate the
sizeof(std prefix) = requested size for new T[1] - sizeof(T)
Does this make sens? Would this be portable?
Somehow, you need to guarantee that the memory address of the start of the memory area is aligned properly for objects of type T, otherwise all of the elements in the array will be misaligned. Different architectures often have different alignment requirements. Also, consider the following: { T v[3]; T* p=&v[0]; int s = size(p); // undefined behavior }
I'm going to take a guess. Of course, this is all contingent on whether T is a class that contains a destructor, otherwise all bets are off. -Sid Sacek struct XYZ { int i; }; struct T { XYZ xyz; ~T() {} }; void foo( void ) { T * tp = new T[ 100 ]; size_t array_size = reinterpret_cast< size_t * >( tp )[ -1 ]; }
Sid Sacek a écrit :
I'm going to take a guess. Of course, this is all contingent on whether T is a class that contains a destructor, otherwise all bets are off. -Sid Sacek
[...] T * tp = new T[ 100 ];
size_t array_size = reinterpret_cast< size_t * >( tp )[ -1 ];
I think this is not going to work if T has a stricter alignment than size_t.
Hi, ----- Original Message ----- From: "Schrader, Glenn" <gschrad@ll.mit.edu> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 11:43 PM Subject: Re: [boost] How to get the size of an C-array?
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of vicente.botet Sent: Wednesday, October 14, 2009 4:54 PM To: boost@lists.boost.org Subject: Re: [boost] How to get the size of an C-array?
Hi Andrey, ----- Original Message ----- From: "Andrey Semashev" <andrey.semashev@gmail.com> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 7:18 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on each
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is
unknown.
// before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting
one of the elements of the array, or more precissely with the address of each one of the elements. the pointer. Is there something in Boost or a portable way to recover the number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable?
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
2. Use a range wrapper instead of a raw pointer to the array.
This is equivalent to say that we can not use dynamic arrays with the STM library. Before to abandoning I want to explore all the posibilities.
Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size. The drawback of using specific functions is that we will need to educate the user, and yet more important, we couldn't use generic classes that make use of these standard operators, as e.g. auto_ptr or scoped_ptr.
The problem I find overloading new[] is that I dont know where my prefixed information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information.
pointer returned by new[] | v | my_prefic | std prefix | user area |
The parameter size_t of the new[] operator is the addition of sizes of std prefix and the user area.
While writing these lines, I think that maybe the size of std prefix can be calculated requesting the creation of an array with only one element, so we can calculate the
sizeof(std prefix) = requested size for new T[1] - sizeof(T)
Does this make sens? Would this be portable?
Somehow, you need to guarantee that the memory address of the start of the memory area is aligned properly for objects of type T, otherwise all of the elements in the array will be misaligned. Different architectures often have different alignment requirements.
Yes, I must consider alignement.
Also, consider the following: { T v[3]; T* p=&v[0]; int s = size(p); // undefined behavior }
Very good observation. Unfortunately, I don't think I could do nothing in this case. Is there a way to check if a pointer has been allocated dynimically? Vicente
Hi again, ----- Original Message ----- From: "vicente.botet" <vicente.botet@wanadoo.fr> To: <boost@lists.boost.org> Sent: Thursday, October 15, 2009 8:18 AM Subject: Re: [boost] How to get the size of an C-array?
Hi, ----- Original Message ----- From: "Schrader, Glenn" <gschrad@ll.mit.edu> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 11:43 PM Subject: Re: [boost] How to get the size of an C-array?
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of vicente.botet Sent: Wednesday, October 14, 2009 4:54 PM To: boost@lists.boost.org Subject: Re: [boost] How to get the size of an C-array?
Hi Andrey, ----- Original Message ----- From: "Andrey Semashev" <andrey.semashev@gmail.com> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 7:18 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on each
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is
unknown.
// before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting
one of the elements of the array, or more precissely with the address of each one of the elements. the pointer. Is there something in Boost or a portable way to recover the number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable?
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
2. Use a range wrapper instead of a raw pointer to the array.
This is equivalent to say that we can not use dynamic arrays with the STM library. Before to abandoning I want to explore all the posibilities.
Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size. The drawback of using specific functions is that we will need to educate the user, and yet more important, we couldn't use generic classes that make use of these standard operators, as e.g. auto_ptr or scoped_ptr.
The problem I find overloading new[] is that I dont know where my prefixed information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information.
pointer returned by new[] | v | my_prefic | std prefix | user area |
The parameter size_t of the new[] operator is the addition of sizes of std prefix and the user area.
While writing these lines, I think that maybe the size of std prefix can be calculated requesting the creation of an array with only one element, so we can calculate the
sizeof(std prefix) = requested size for new T[1] - sizeof(T)
Does this make sens? Would this be portable?
Somehow, you need to guarantee that the memory address of the start of the memory area is aligned properly for objects of type T, otherwise all of the elements in the array will be misaligned. Different architectures often have different alignment requirements.
Yes, I must consider alignement.
Also, consider the following: { T v[3]; T* p=&v[0]; int s = size(p); // undefined behavior }
Very good observation. Unfortunately, I don't think I could do nothing in this case. Is there a way to check if a pointer has been allocated dynimically?
Well I suspect the answer is no. In my opinion, this should be undefined behavior as it is the case for T v[3]; T* p=&v[0]; delete p; // undefined behavior In addition, I don't need to make public this function, as I need it only internally. Maybe I could do something similar to the boost::extent metafunction: template <typename T, std::size_t N = 0> struct number_of_elements { ... }; but number_of_elements will work as boost::extent except when the result of extent is 0. In this case the metafunction suposse that the given pointer has been allocated with the overloaded new[]. If it is not the case the result will be undefined. Do you consider that this metafunction is coherent? Can someone in this list explain why the standard don't allows to recover the number of elements allocated by new[]? What is the deep reason? Best regards, Vicente
vicente.botet wrote:
Can someone in this list explain why the standard don't allows to recover the number of elements allocated by new[]? What is the deep reason?
I suppose the reason is to not introduce a distinction between types with destructors and types without. If this was to be provided, a meta-function to tell whether a type has a trivial destructor or not would be needed as well.
Mathias Gaunard wrote:
vicente.botet wrote:
Can someone in this list explain why the standard don't allows to recover the number of elements allocated by new[]? What is the deep reason?
I suppose the reason is to not introduce a distinction between types with destructors and types without.
If this was to be provided, a meta-function to tell whether a type has a trivial destructor or not would be needed as well.
Why? From the underlying allocator point of view there's not much difference in allocating memory for PODs and non-PODs. My guess is that this is not possible in order to allow trivial implementation of new/delete allocators in terms of malloc/free. The latter don't allow to acquire the size of allocated buffer, so in order to provide this ability for new/delete it would have to add its own header to store the size, which would lead to more extensive memory consumption. It's a pity that malloc/free allocator don't allow buffer size acquisition, especially since the size is most likely stored somehow.
Andrey Semashev a écrit :
Mathias Gaunard wrote:
vicente.botet wrote:
Can someone in this list explain why the standard don't allows to recover the number of elements allocated by new[]? What is the deep reason?
I suppose the reason is to not introduce a distinction between types with destructors and types without.
If this was to be provided, a meta-function to tell whether a type has a trivial destructor or not would be needed as well.
Why? From the underlying allocator point of view there's not much difference in allocating memory for PODs and non-PODs.
No, but there is for new[] and delete[] expressions. Indeed, delete[] needs to call the destructor on each element, and for that it needs to know how many elements were constructed. If the destructor is trivial, it doesn't need to call them, so there is no need to store how much elements there are. Note that the number of elements constructed with new[] is separate in the general case from the number of elements allocated, since allocators tend to work with blocks larger than needed.
Hi Mathias, ----- Original Message ----- From: "Mathias Gaunard" <mathias.gaunard@ens-lyon.org> To: <boost@lists.boost.org> Sent: Thursday, October 15, 2009 3:52 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
Can someone in this list explain why the standard don't allows to recover the number of elements allocated by new[]? What is the deep reason?
I suppose the reason is to not introduce a distinction between types with destructors and types without.
If this was to be provided, a meta-function to tell whether a type has a trivial destructor or not would be needed as well.
Sorry but I dont understand your pourpose. What types with destructors have to be with my question. Please could you clarify? Thanks, Vicente
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of vicente.botet Sent: Thursday, October 15, 2009 3:25 AM To: boost@lists.boost.org Subject: Re: [boost] How to get the size of an C-array?
Hi again, ----- Original Message ----- From: "vicente.botet" <vicente.botet@wanadoo.fr> To: <boost@lists.boost.org> Sent: Thursday, October 15, 2009 8:18 AM Subject: Re: [boost] How to get the size of an C-array?
Hi, ----- Original Message ----- From: "Schrader, Glenn" <gschrad@ll.mit.edu> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 11:43 PM Subject: Re: [boost] How to get the size of an C-array?
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-
On Behalf Of vicente.botet Sent: Wednesday, October 14, 2009 4:54 PM To: boost@lists.boost.org Subject: Re: [boost] How to get the size of an C-array?
Hi Andrey, ----- Original Message ----- From: "Andrey Semashev" <andrey.semashev@gmail.com> To: <boost@lists.boost.org> Sent: Wednesday, October 14, 2009 7:18 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is
unknown.
// before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
The C++ compiler or the C++ standard library knows this number of elements as it needs to call the destructor of each element when deleting
each one of the elements of the array, or more precissely with the address of each one of the elements. the pointer. Is there something in Boost or a portable way to recover
number of elements of the array, maybe overloading the new [] operator and prefixing the allocate storage with some information? But what information can be stored so the implementation is portable?
There is no portable way to acquire the dynamic array length. IMO,
are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third
type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
2. Use a range wrapper instead of a raw pointer to the array.
This is equivalent to say that we can not use dynamic arrays with the STM library. Before to abandoning I want to explore all the posibilities.
Regarding the trick with the prepended length storing, it can solve
problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size. The drawback of using specific functions is that we will need to educate
user, and yet more important, we couldn't use generic classes that make use of these standard operators, as e.g. auto_ptr or scoped_ptr.
The problem I find overloading new[] is that I dont know where my
information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information.
pointer returned by new[] | v | my_prefic | std prefix | user area |
The parameter size_t of the new[] operator is the addition of sizes of std prefix and the user area.
While writing these lines, I think that maybe the size of std prefix can be calculated requesting the creation of an array with only one element, so we can calculate the
sizeof(std prefix) = requested size for new T[1] - sizeof(T)
Does this make sens? Would this be portable?
Somehow, you need to guarantee that the memory address of the start of
memory area is aligned properly for objects of type T, otherwise all of
bounces@lists.boost.org] the there party the the prefixed the the
elements in the array will be misaligned. Different architectures often have different alignment requirements.
Yes, I must consider alignement.
Also, consider the following: { T v[3]; T* p=&v[0]; int s = size(p); // undefined behavior }
Very good observation. Unfortunately, I don't think I could do nothing in this case. Is there a way to check if a pointer has been allocated dynimically?
Well I suspect the answer is no.
In my opinion, this should be undefined behavior as it is the case for
T v[3]; T* p=&v[0]; delete p; // undefined behavior
In addition, I don't need to make public this function, as I need it only internally. Maybe I could do something similar to the boost::extent metafunction:
template <typename T, std::size_t N = 0> struct number_of_elements { ... };
but number_of_elements will work as boost::extent except when the result of extent is 0. In this case the metafunction suposse that the given pointer has been allocated with the overloaded new[]. If it is not the case the result will be undefined.
Do you consider that this metafunction is coherent?
Can someone in this list explain why the standard don't allows to recover the number of elements allocated by new[]? What is the deep reason?
Best regards, Vicente
I don't know why but the same thing shows up elsewhere. For instance I don't know of a way to query the size of a memory block that was allocated with malloc() either. Even so, this isn't a major problem since its easy enough to create smart pointers that store the extra information. That also avoids burdening the infrastructure with things that many people may not care about under the "you only pay for what you use" C/C++ mantra. Tacking extra information onto the front of an array like you're doing above can be made to work. In fact, some (most??) malloc() and free() implementations manage their heap memory in a similar manner (e.g. using linked lists of allocated blocks and free blocks). You might get a sense of how portably this can be done by looking through the GNU implementation of malloc and free and see what they needed to do to support different platforms. There really isn't a safe way to do this using simple pointers so I use smart pointers quite a lot although in your case though I would also consider a std::vector. --glenn
vicente.botet wrote:
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
I'm not sure I understand you here. Are you saying that other threads may be accessing the array elements during the destruction?
Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size.
I think moving a bit closer to your use-case might help. Am I correct if I say that the type of array elements is provided by your library and this size acquisition is only needed for such arrays? If so then yes, you can override new and delete (and new[] and delete[]) for your classes to store additional info in the allocated memory.
The problem I find overloading new[] is that I dont know where my prefixed information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information.
The pointer returned by new[] defined for your class will be used to construct array elements. Thus your implementation should allocate memory of size = sizeof(info) + possible alignment gap + sizeof(elements), where the last argument is passed to your operator new[], and return the pointer to the beginning of the elements storage area (i.e. behind the info and alignment gap). The way you allocate memory is not relevant, you may use malloc or global operator new[] or something else. The additional information that the underlying allocator may or may not store before the pointer it returns to you is totally not your concern. All in all, the overall scheme is as follows: class T { struct info_t { std::size_t m_Count; }; struct alignment_gap_helper { info_t dummy1; T dummy2; }; enum _ { gap_size = sizeof(alignment_gap_helper) - sizeof(info_t) - sizeof(T) }; public: static void* operator new[] (std::size_t size) { void* p = std::malloc(sizeof(info_t) + gap_size + size); if (p) { new (p) info_t(); static_cast< info_t* >(p)->m_Count = size / sizeof(T); // Return the adjusted pointer return static_cast< char* >(p) + sizeof(info_t) + gap_size; } else throw std::bad_alloc(); } static void operator delete[] (void* p) { if (p) { // Restore the original pointer p = static_cast< char* >(p) - sizeof(info_t) - gap_size; static_cast< info_t* >(p)->~info_t(); std::free(p); } } static void* operator new (std::size_t size) { return operator new[] (size); } static void operator delete (void* p) { return operator delete[] (p); } friend std::size_t element_count_of(T const* p) { if (p) { // Restore the original pointer p = static_cast< const char* >(p) - sizeof(info_t) - gap_size; return static_cast< const info_t* >(p)->m_Count; } else return 0; } }; And you also should not forget the operator overloads with std::nothrow argument.
Hi, ----- Original Message ----- From: "Andrey Semashev" <andrey.semashev@gmail.com> To: <boost@lists.boost.org> Sent: Thursday, October 15, 2009 7:40 PM Subject: Re: [boost] How to get the size of an C-array?
vicente.botet wrote:
There is no portable way to acquire the dynamic array length. IMO, there are two practical solutions for this:
1. Put everything you need into T's destructor. If T is a third party type then a wrapper class around T can serve the purpose.
Well, the problem is that we need to do something before the destructor is called. When deleting an array of transactional objects we need to mark each one of the elements as if they were written, so the conclict detection cat detect other threads been written on a specific array element.
I'm not sure I understand you here. Are you saying that other threads may be accessing the array elements during the destruction?
Yes. This is the case I try to solve.
Regarding the trick with the prepended length storing, it can solve the problem, but it introduces pointer magic which may not be obvious for the ones reading the code (including yourself a couple months later). And, AFAIK, you won't be able to override the standard new and delete operators to behave that way, so you'll have to use dedicated functions to allocate/deallocate such arrays. Therefore I wouldn't recommend it unless there are significant reasons for it.
No, I don't want to overload the standard new and delete operators, just provide a mixing helping the user to define transactional objects. For these transactional objects, the overload of the operator new and new[] could set some information that will allow us to define a free function size taking a transactional object pointer and returning its size.
I think moving a bit closer to your use-case might help. Am I correct if I say that the type of array elements is provided by your library and this size acquisition is only needed for such arrays? If so then yes, you can override new and delete (and new[] and delete[]) for your classes to store additional info in the allocated memory.
Yes, this is the case. I pretend to pverload only the new/delete operators for specific classes.
The problem I find overloading new[] is that I dont know where my prefixed information will be placed respect to the returned pointer, as the standard, prefix it already to store its own specific information.
The pointer returned by new[] defined for your class will be used to construct array elements. Thus your implementation should allocate memory of size = sizeof(info) + possible alignment gap + sizeof(elements), where the last argument is passed to your operator new[], and return the pointer to the beginning of the elements storage area (i.e. behind the info and alignment gap).
The way you allocate memory is not relevant, you may use malloc or global operator new[] or something else. The additional information that the underlying allocator may or may not store before the pointer it returns to you is totally not your concern.
All in all, the overall scheme is as follows:
class T { struct info_t { std::size_t m_Count; };
struct alignment_gap_helper { info_t dummy1; T dummy2; };
enum _ { gap_size = sizeof(alignment_gap_helper) - sizeof(info_t) - sizeof(T) };
public:
static void* operator new[] (std::size_t size) { void* p = std::malloc(sizeof(info_t) + gap_size + size); if (p) { new (p) info_t(); static_cast< info_t* >(p)->m_Count = size / sizeof(T);
Are you sure the size given as parameter consider only the user info? Doesn't this size includes any additional part needed by the standard library to know if the the allocation corresponds to a single element or to an array and whatever is needed to call the destructors of all the array elements?
// Return the adjusted pointer return static_cast< char* >(p) + sizeof(info_t) + gap_size; } else throw std::bad_alloc(); }
static void operator delete[] (void* p) { if (p) { // Restore the original pointer p = static_cast< char* >(p) - sizeof(info_t) - gap_size; static_cast< info_t* >(p)->~info_t(); std::free(p); } }
static void* operator new (std::size_t size) { return operator new[] (size); }
static void operator delete (void* p) { return operator delete[] (p); }
friend std::size_t element_count_of(T const* p) { if (p) { // Restore the original pointer p = static_cast< const char* >(p) - sizeof(info_t) - gap_size; return static_cast< const info_t* >(p)->m_Count; } else return 0; } };
And you also should not forget the operator overloads with std::nothrow argument.
Yes. Thanks to recal this to me. I was already aware of this kind of problems. All this stuff seems correct except the way you calculate the m_Count. Thanks you Andrey to try to solve this problem and for all the details. Best, Vicente
vicente.botet wrote:
I'm not sure I understand you here. Are you saying that other threads may be accessing the array elements during the destruction?
Yes. This is the case I try to solve.
Why not locking the mutex (or whatever you use to synchronize the threads) in the destructor? It will block if another thread is working with the object.
All in all, the overall scheme is as follows:
static void* operator new[] (std::size_t size) { void* p = std::malloc(sizeof(info_t) + gap_size + size); if (p) { new (p) info_t(); static_cast< info_t* >(p)->m_Count = size / sizeof(T);
Are you sure the size given as parameter consider only the user info? Doesn't this size includes any additional part needed by the standard library to know if the the allocation corresponds to a single element or to an array and whatever is needed to call the destructors of all the array elements?
Hmm... Looks like you're right here. The Standard doesn't say what the size will be and at least gcc does add a little to the size. However, if sizeof(T) is big enough, you may not notice this extra increase in size. Well, unless you want to tweak your code for every compiler/platform to take that additional info into account, I don't see how this can be solved. You might want to use an additional argument to the operator new[] to pass the array size twice, but this essentially eliminates the advantage of using the regular operator new. Still, you will be able to use scoped_array, shared_array, etc.
vicente.botet wrote:
Hi,
On TBoost.STM, I need to do something before deleting an array on each one of the elements of the array, or more precissely with the address of each one of the elements.
T* ptr = new T[3];
// in another file, the number of elements to which ptr points is unknown. // before deleting for (size_t i=0; i< 'ptr size'; ++i) { // do something with &ptr[i] } delete [] ptr;
As the number of elemenst is unknow in the separated unit, I have no mean to iterate on ptr to get the address of the elements.
You just have to remember the size of the allocation. const ptr_size = 3; T* ptr = new T[ptr_size]; That's all there is to it! Bo Persson
participants (6)
-
Andrey Semashev -
Bo Persson -
Mathias Gaunard -
Schrader, Glenn -
Sid Sacek -
vicente.botet