Hi Andrey, Thanks for repaid response. It is freeing the memory at last decrement, but not immediately. It frees memory when all related objects’ reference count are decreased to zero. Since it’s C++, GC is not the best option all the time, It’s not preferable anyway. The idea is just delay freeing, after all related objects’ refence count is decreased to zero, and free them all. void Dispose() { if (this->_ptr != nullptr) { PTR* _ptr = this->_ptr; this->_ptr = nullptr; _ptr->~PTR(); RelRef(_ptr); } } As above, dispose method calls its pointing object’s destructor, but not free that object with free(ptr) function, This gives opportunity to that object to call other shared_ptr to call their objects’ destructors, including self. And everything a shared_ptr’s destructor does it just decreasing the count, including self-referenced or circular reference. When every count is decreased correctly, we free them all. If you have Visual Studio and run the code which detects memory leak with #ifdef _WIN32 _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF); #endif You will see no memory leak at all. As I said, It’s no easy to explain how it works, but the code speaks for itself. About GC: the same method can be used in other languages which heavily depends on GC, since we don’t have to trace the object linkage among large objects. Please run the code. I don’t know how to detect memory leak on *nix, but simply watch how many times the free(ptr) is called would explain it. void rcgc_base::Collect(std::vector<void*>& p_wilds) { if (p_wilds.size() > 0) { for (auto p = p_wilds.begin(); p != p_wilds.end(); ++p) { if (*p) { //delete p->first; //NOTICE: use free instead of delete to avoid double calling destructor free(*p); } } p_wilds.clear(); } } BTW, Collect is set to called directly after Dispose, which means, when SetAutoCollect(true), There is only necessary delay for freeing a shared_ptr managed object. Please let me know if I’m clear on this explaination. Sincerely, Yilin 发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用 发件人: Andrey Semashev via Boost<mailto:boost@lists.boost.org> 发送时间: 2020年7月31日 17:51 收件人: boost@lists.boost.org<mailto:boost@lists.boost.org> 抄送: Andrey Semashev<mailto:andrey.semashev@gmail.com> 主题: Re: [boost] Hello everybody, I have something to share about the shared_ptr class On 2020-07-31 11:52, 逸霖 杨 via Boost wrote:

Hello dear boost members,

I’m quite honored to join this mail list for the first time!

As I was told that the boost project is where the shared_ptr class originated, so I think it should be OK to talk about it here.

Here I have some ideas and code on my version of shared_ptr class.

The aim is to remove the usage of weak_ptr and fix the circular reference problem.

Please check code on https://github.com/yyl-20020115/RCGC

That’s the idea and primitive implementation. I don’t know exactly how to test it more widely among situations, please help me to find the very situations that this solution would fail, and I’ll try to fix it ASAP.

READM.md is a good start and README_ZHCN.md is better if you can read Chinese. However, I don’t think I can illustrate the algorithm very well both in English and Chinese. Well the code speaks for itself, and please read the code and run the program.

Please generate a makefile and build with CMakeLists.txt on *nix platforms if Visual Studio 2019 (where I code the project) is not available.

All ideas and discussions and help from you are all very welcome!

So, it looks like you're just doing garbage collection in a separate thread instead of freeing memory on the last reference counter decrement. Which means that your claim that memory consumption is the same as the original shared_ptr is not quite true. IMHO, GC is a too costly and unpredictable solution. I don't see myself ever wanting to have a GC, unless in a *very* localized and controlled context, like a single function cleaning up after itself. Regarding how circular dependencies are resolved, I don't quite see the solution. In the rcgc_shared_ptr::Dispose method, you immediately destroy the referenced object, not even when the reference count drops to zero, which is plain wrong because other pointers may still refer to the object. If you didn't destroy it, in the circular dependencies case the reference count would never drop to zero, which means the GC thread would never collect the object. Lastly, I question the usefulness of a shared_ptr without a weak_ptr. Note that weak_ptr is not limited to just circular references resolution, it has other use cases. Although shared_ptr has other useful features, like pointer aliasing and deleter erasure, I find myself using shared_ptr almost exclusively when I also need weak_ptr. When I don't need weak_ptr I would use intrusive_ptr. But then your rcgc_shared_ptr doesn't provide any of the additional features of shared_ptr, so there's even less reason to use it. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 2020-07-31 15:56, Andrey Semashev wrote:

Please, don't top-post. Our discussion guidelines are described here:

https://www.boost.org/community/policy.html

On 2020-07-31 13:06, 逸霖 杨 via Boost wrote:

...

Hi Andrey, Thanks for repaid response.

It is freeing the memory at last decrement, but not immediately.

It frees memory when all related objects’ reference count are decreased to zero.

Yes, it frees memory when the reference count becomes zero. But it *destroys* the object (i.e. calls its destructor) earlier, before the reference count is zero.

...

     void Dispose() {          if (this->_ptr != nullptr) {              PTR* _ptr = this->_ptr;              this->_ptr = nullptr;              _ptr->~PTR();

               ^^^ here ^^^

...

             RelRef(_ptr);          } }

...

As above,  dispose method calls its pointing object’s destructor, but not free that object with free(ptr) function,

After the destructor call, the object no longer exists, regardless of whether the memory it used to be placed in is freed or not.

Case 1:

  struct A   {     unsigned int x = 0u;

    ~A() { x = 0xBAADF00D; }   };

  rcgc_shared_ptr< A > p1;   {     rcgc_shared_ptr< A > p2(new A());     p1 = p2;     std::cout << p1->x << std::endl;   }   std::cout << p1->x << std::endl; // (1)

The above is a bug because (1) accesses the object A after its destruction. If A::x was, say a unique_ptr, it would be null or pointing to already freed memory.

Case 2 (which follows from Case 1)

  struct B;

  struct A   {     rcgc_shared_ptr< B > m_b;     std::string m_x;

    ~A() { std::cout << m_b->m_x << std::endl; }   };

  struct B   {     rcgc_shared_ptr< A > m_a;     std::string m_x;

    ~B() { std::cout << m_a->m_x << std::endl; }   };

  rcgc_shared_ptr< A > p(new A());   p->m_b.reset(new B());   p->m_b->m_a = p;

When p gets destroyed, it will destroy the object A, even though it is still referenced by B::m_a. When A::m_b is destroyed, it will call ~B, which will access A::m_x, which is already destroyed.

I should note that in the code samples above I'm ignoring the mismatch between `operator new` that is used to allocate objects and `std::free` that rcgc_shared_ptr uses to free memory. That is another bug, of course.

...

If you have Visual Studio and run the code which detects memory leak with #ifdef _WIN32      _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF); #endif

You will see no memory leak at all.

This check only shows that no memory is leaked. The problem here is not a memory leak but incorrect memory access (potential use-after-destory and use-after-free).

On 2020-07-31 19:51, 逸霖 杨 via Boost wrote:

Hi Andrey,

Sorry for Top-Post. But I searched and still don’t know what is Top-Post, pardon me, and please explain a little.

The page I linked before contained a link to Wikipedia that explains it. Here is that link: https://en.wikipedia.org/wiki/Posting_style#Top-posting

Reply about :

After the destructor call, the object no longer exists, regardless of whether the memory it used to be placed in is freed or not. ===================================== This is true as a concept, but when we overload delete operator for global or for just one class, we write “free(ptr)” inside that overloaded function (at least MS’ implementation of C++ works like this).

Right, that's how storage allocation functions work. By the time they are invoked, the object is either not yet created or is already destroyed.

================================================================== Case 1:

struct A { unsigned int x = 0u;

~A() { x = 0xBAADF00D; } };

rcgc_shared_ptr< A > p1; { rcgc_shared_ptr< A > p2(new A()); p1 = p2; std::cout << p1->x << std::endl; } std::cout << p1->x << std::endl; // (1)

The above is a bug because (1) accesses the object A after its destruction. If A::x was, say a unique_ptr, it would be null or pointing to already freed memory. ====================== Why would we assign x with a useful value in destructors ?

Does it matter? The example shows that you're accessing a destroyed object, which you confirmed in the output below.

Unlike the predicted behavior on my side, this Case1 I run in main function, results in 0 baadf00d (<<std::hex<< is added before to show hex value instead of dec value)

Case 2 (which follows from Case 1)

struct B;

struct A { rcgc_shared_ptr< B > m_b; std::string m_x;

~A() { std::cout << m_b->m_x << std::endl; } };

struct B { rcgc_shared_ptr< A > m_a; std::string m_x;

~B() { std::cout << m_a->m_x << std::endl; } };

rcgc_shared_ptr< A > p(new A()); p->m_b.reset(new B()); p->m_b->m_a = p;

When p gets destroyed, it will destroy the object A, even though it is still referenced by B::m_a. When A::m_b is destroyed, it will call ~B, which will access A::m_x, which is already destroyed. ====================================== I’m sorry that rcgc_shared_ptr does not provide reset function. I didn’t think of this yet, but will do.

The reset function has the same semantics as for shared_ptr. It doesn't change my point though.

...

If you have Visual Studio and run the code which detects memory leak with #ifdef _WIN32 _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF); #endif

You will see no memory leak at all.

This check only shows that no memory is leaked. The problem here is not a memory leak but incorrect memory access (potential use-after-destory and use-after-free). ========================================== The thing is in reality, calling destructor is not same as freeing memory.

True.

Memory is still containing valid data if no one messes with it.

Not true. In general, accessing memory after object destruction is undefined behavior. Whether that memory contents still contain the data that was last present in the destroyed object is not guaranteed.

And if the free(ptr) function is not called, no one can/should mess with it. The only thing would possible is calling the destructor for multiple times because the rcgc_shared_ptrs requests this action. But use-after-destroy can be still avoided.

To avoid use-after-destroy, just keep the good manner like

~A(){ If(this->p!=nullptr) { delete this->p; this->p = nullptr; } } And everything will be safe.

No, it won't be safe. As an example of a real practical problem with this code is that compilers are allowed to remove the assignment of nullptr to the class member because the object is destroyed and no valid code will read that member after destruction. Basically, that assignment is dead code. https://gcc.godbolt.org/z/rnncjn

And also, no one should do important work inside of destructors besides freeing and avoid double freeing, don’t they?

Does it matter what people do in destructors? Destructors are not supposed to be called on a non-existing object. (BTW yes, people do non-trivial things in destructors besides releasing resources.) You seem to be missing the fact that C++ doesn't work with just bytes in memory and has an actual object model.

BTW, is it correct that intuitive_ptr is another kind of COM IUnknown ?

I'm not sure I understand this question. Yes, you can use intrusive_ptr to reference IUnknown and derived objects.

What would you do if a reference is pointed to the object itself?

I'm not sure what use case you mean. In general, I don't have to do anything special about cycle resolution with intrusive_ptrs because either cycles are not possible in that code or they resolve automatically (e.g. there is a destroy() method that explicitly resets all pointers stored in the object before returning).

Greetings, On 31/07/2020 18:51, 逸霖 杨 via Boost wrote:

After the destructor call, the object no longer exists, regardless of whether the memory it used to be placed in is freed or not. ===================================== This is true as a concept, but when we overload delete operator for global or for just one class, we write “free(ptr)” inside that overloaded function (at least MS’ implementation of C++ works like this). Which means, it’s practically separated-able: destructor called and free() called. And if we separate the two stages, we can get something that we can not achieve before, just like get a clean reference decreasing method, and free all effected objects all together.

You seem to be missing the fact that dtors are special functions by standard, and dtors inevitably *end* the lifetime of an object of the abstract machine. I suggest you read [basic.memobj]; in particular, [basic.life] (lifetime of objects), [class.cdtor] (ctors and dtors) and [basic.stc.dynamic] (storage allocation and deallocation) (https://eel.is/c++draft/basic.memobj) (https://eel.is/c++draft/basic.life) (https://eel.is/c++draft/class.cdtor) (https://eel.is/c++draft/basic.stc.dynamic)

================================================================== Case 1:

struct A { unsigned int x = 0u;

~A() { x = 0xBAADF00D; } };

rcgc_shared_ptr< A > p1; { rcgc_shared_ptr< A > p2(new A()); p1 = p2; std::cout << p1->x << std::endl; } std::cout << p1->x << std::endl; // (1)

The above is a bug because (1) accesses the object A after its destruction. If A::x was, say a unique_ptr, it would be null or pointing to already freed memory. ====================== Why would we assign x with a useful value in destructors ?

Users will do what users do: do something you do not expect them to do

The thing is in reality, calling destructor is not same as freeing memory.

True Invoking the dtor will end the object's lifetime Freeing memory will release the allocated storage where the object lived on before it was destroyed

Memory is still containing valid data if no one messes with it Attempting to access the contents of an object after its lifetime has been ended is undefined behavior as per [basic.life.6.2], so the bytes found at the allocated storage are considered to contain invalid data, regardless of whether they contain what you expect or not during execution.

On July 31, 2020 2:21:50 PM EDT, "逸霖 杨 via Boost" <boost@lists.boost.org> wrote:

Hi Janson and Andrey,

I got your point. And I think you got mine too.

So this is so-called by-design.

Is this a good reason for rejecting an innovation?

It is absolutely a good reason. What you're doing is invalid C++, as Andrey and others have tried, repeatedly, to tell you. When you violate the rules of the language, your "innovation" is simply bad design.

I’m not sure. I don’t make the definition of C++. Also, I’m here to contribute, not to dominate.

The answers implied that calling the dtors will end the object life once for all. But is there any explicit “once for all”.

Yes. By definition of the language, there is no object until a constructor runs, and objects cease to exist when their destructor begins. The moment you invoke an object's destructor, no further access to its memory, other than its reclamation or reuse for another object, is undefined behavior. No design assuming the bits remain unchanged is valid. There's simply no other way to see this.

Upon the definition of dtors, people made code, and the new understanding of dtors would result in unstableness, which I’m quite aware of.

However, what if someone who would like to keep a simple and neat usage of the dtors to help rcgc_shared_ptr to work in the best performance just in his own code? For example, my code, or his or hers?

People are free to write code relying on undefined behavior, but such cannot be generalized and would never be accepted into Boost. (There have been a few implementation details of Boost libraries that rely on undefined behavior, but those are conditionally compiled for specific compilers which define the behavior -- thereby going beyond the C++ Standard.) Any library that depends upon users carefully designing all of their classes to avoid the bad effects of undefined behavior in the library, is unlikely to see much use.

So, would anyone be kindly to help me to understand why we can not improve it together?

I hope you understand now. (BTW, don't overquote -- retain no more of the previous messages than necessary.) -- Rob (Sent from my portable computation device.)

逸霖 杨 via Boost said: (by the date of Sun, 2 Aug 2020 00:20:13 +0000)

Again, fighting against C++’s standard is never my purpose, making life easier with a new algorithm is.

I was using boost::singleton_pool with great results for a similar purpose. It worked great. #include <boost/pool/pool_alloc.hpp> my computations got about 10% faster, because previously I was very often reallocating memory. cheers -- # Janek Kozicki http://janek.kozicki.pl/

About void Dispose() { if (this->_ptr != nullptr) { PTR* _ptr = this->_ptr; this->_ptr = nullptr; _ptr->~PTR(); RelRef(_ptr); } } Especially _ptr->~PTR(); This is not delete _ptr; Because: delete _ptr = _ptr->~PTR() + free(_ptr) We use _ptr->~PTR() to give shared_ptrs to decrease count and all descendent objects to decrease count, but not free any of them And then check if anything to be free at the end of destructor of shared_ptr。 if anything including self to be free, we free them all together then. ~rcgc_shared_ptr() { this->Dispose(); if (_ac) { Collect(); } } If _ac (auto collect) is true, we do collect here. void rcgc_base::Collect(std::vector<void*>& p_wilds) { if (p_wilds.size() > 0) { for (auto p = p_wilds.begin(); p != p_wilds.end(); ++p) { if (*p) { //delete p->first; //NOTICE: use free instead of delete to avoid double calling destructor free(*p); } } p_wilds.clear(); } } So we have options now: either collect at the end of destructor of shared_ptr, and collect everything with rc of 0; or we collect everything when we need to collect or in other dedicated thread (that’s the advantage). BR, Yilin 发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用 发件人: Andrey Semashev via Boost<mailto:boost@lists.boost.org> 发送时间: 2020年7月31日 17:51 收件人: boost@lists.boost.org<mailto:boost@lists.boost.org> 抄送: Andrey Semashev<mailto:andrey.semashev@gmail.com> 主题: Re: [boost] Hello everybody, I have something to share about the shared_ptr class On 2020-07-31 11:52, 逸霖 杨 via Boost wrote:

Hello dear boost members,

I’m quite honored to join this mail list for the first time!

As I was told that the boost project is where the shared_ptr class originated, so I think it should be OK to talk about it here.

Here I have some ideas and code on my version of shared_ptr class.

The aim is to remove the usage of weak_ptr and fix the circular reference problem.

Please check code on https://github.com/yyl-20020115/RCGC

That’s the idea and primitive implementation. I don’t know exactly how to test it more widely among situations, please help me to find the very situations that this solution would fail, and I’ll try to fix it ASAP.

READM.md is a good start and README_ZHCN.md is better if you can read Chinese. However, I don’t think I can illustrate the algorithm very well both in English and Chinese. Well the code speaks for itself, and please read the code and run the program.

Please generate a makefile and build with CMakeLists.txt on *nix platforms if Visual Studio 2019 (where I code the project) is not available.

All ideas and discussions and help from you are all very welcome!

So, it looks like you're just doing garbage collection in a separate thread instead of freeing memory on the last reference counter decrement. Which means that your claim that memory consumption is the same as the original shared_ptr is not quite true. IMHO, GC is a too costly and unpredictable solution. I don't see myself ever wanting to have a GC, unless in a *very* localized and controlled context, like a single function cleaning up after itself. Regarding how circular dependencies are resolved, I don't quite see the solution. In the rcgc_shared_ptr::Dispose method, you immediately destroy the referenced object, not even when the reference count drops to zero, which is plain wrong because other pointers may still refer to the object. If you didn't destroy it, in the circular dependencies case the reference count would never drop to zero, which means the GC thread would never collect the object. Lastly, I question the usefulness of a shared_ptr without a weak_ptr. Note that weak_ptr is not limited to just circular references resolution, it has other use cases. Although shared_ptr has other useful features, like pointer aliasing and deleter erasure, I find myself using shared_ptr almost exclusively when I also need weak_ptr. When I don't need weak_ptr I would use intrusive_ptr. But then your rcgc_shared_ptr doesn't provide any of the additional features of shared_ptr, so there's even less reason to use it. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

As far as I can tell, rcgc_shared_ptr<T> calls the destructor of T as soon as its destructor is invoked, regardless of whether other references exist. That is, int main() { rcgc_shared_ptr<X> p1( new X ); { rcgc_shared_ptr<X> p2( p1 ); } // ~X is called here // *p1 references a destroyed object here } This doesn't seem correct to me; the point of shared_ptr is to not call the destructor of the pointee as long as references (such as p1 in the code above) exist to it. ------- Yes, you’re correct. The detor of rcgc_shared_ptr<X> calls the X’s detor As soon as rcgc_shared_ptr<X>’ object p2 leaves its scope. However P1 still holds the object. So only p2’s ~X() is called, not p1’s ~X(). If both called, the same object (new X()) will be free. Although calling dtors, but it does not mean certainly, free or mess-up the object generated by new X(). Only when the reference count decreased to zero, the object generated by new X() will be free. (Here I discussed with Andrey and Janson about if it’s valid to use the data which is called detor upon) So this is not the behavior you expected as shared_ptr you know. If you write X like this class X { public: std::string text; }; It can be a disaster as text object would be freed twice or more while the detor of X called maybe twice or more. but if you write X like this class X { public: rcgc_shared_ptr<std::string> text; }; And use the text field accordingly, It will be good and correct, because rcgc_shared_ptr already managed the memory for you (including the situations that the detors of std::string Maybe called multiple times) I admit that I didn’t follow the semantics of the orginal shared_ptr. Maybe another name, just rcgc_ptr is better. It’s just smart pointer for you to manage memory allocations and deallocations. Maybe on the level of object creation/destructions is incorrect (with other way of managing pointers or objects), but rcgc_shared_ptr still cares it for you.

I��m still thinking about the use cases. Code like this If you write X like this class X { public: std::string text; }; You can not use this rcgc_shared_ptr<X> Only if the std::string��s implementation check the underlying pointer (if equals to nullptr) within ~string() when the detors called multiple times which is not true I think. So this solution is not universal �C unless everything changes accordingly. This is no problem for a new invented language or for rebuilding a GC, but for C++ may be too radical. Only someone knows what he��s doing can use this well, and yes, I haven��t give any manual yet. BTW, I updated the code on GitHub, and please pull and run. BR, Yilin