Greetings, There is more to come but I just documented the main classes of the Shifted Pointer code. To generate & look at the docs simply use: $ cd shifted_ptr/ $ svn up $ cd libs/smart_ptr/doc $ doxygen $ firefox html/index.html Some people like and some people dislike Doxygen but in my opinion it keeps the docs and the code in perfect harmony and requires no duplication of the function names and parameters. Bests, -Phil
On Fri, Apr 8, 2011 at 12:06 PM, Phil Bouchard <philippe@fornux.com> wrote:
Greetings,
There is more to come but I just documented the main classes of the Shifted Pointer code. To generate & look at the docs simply use: $ cd shifted_ptr/ $ svn up $ cd libs/smart_ptr/doc $ doxygen $ firefox html/index.html
Some people like and some people dislike Doxygen but in my opinion it keeps the docs and the code in perfect harmony and requires no duplication of the function names and parameters.
Could you also commit built html documentation? Thanks, - Jeff
On 4/8/2011 1:00 PM, Jeffrey Lee Hellrung, Jr. wrote:
On Fri, Apr 8, 2011 at 12:06 PM, Phil Bouchard<philippe@fornux.com> wrote:
Could you also commit built html documentation?
Sorry for the delay but svn decided not to like the way the lines of the html pages end, preventing me from committing the html pages. I get: svn: Inconsistent line ending style Thanks, -Phil
On 4/8/2011 1:00 PM, Jeffrey Lee Hellrung, Jr. wrote:
Could you also commit built html documentation?
You can browse the documentation here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... -Phil
On 4/8/2011 2:06 PM, Phil Bouchard wrote:
On 4/8/2011 1:00 PM, Jeffrey Lee Hellrung, Jr. wrote:
Could you also commit built html documentation?
You can browse the documentation here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
For those not aware a presentation can be found here also: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... -Phil
On 4/8/2011 2:13 PM, Phil Bouchard wrote:
On 4/8/2011 2:06 PM, Phil Bouchard wrote:
On 4/8/2011 1:00 PM, Jeffrey Lee Hellrung, Jr. wrote:
Could you also commit built html documentation?
You can browse the documentation here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
For those not aware a presentation can be found here also: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
And an overview is available here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... Please let me know if anything isn't clear enough. -Phil
On Fri, Apr 8, 2011 at 17:43, Phil Bouchard <philippe@fornux.com> wrote:
And an overview is available here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
Please let me know if anything isn't clear enough.
I'd like to see a bit more about what it means by "blocks". And I'm sure some GC fan will say that they don't have to stop the world any more. ~ Scott
On 4/8/2011 6:03 PM, Scott McMurray wrote:
On Fri, Apr 8, 2011 at 17:43, Phil Bouchard<philippe@fornux.com> wrote:
And an overview is available here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
Please let me know if anything isn't clear enough.
I'd like to see a bit more about what it means by "blocks".
And I'm sure some GC fan will say that they don't have to stop the world any more.
It seems that the new generational concurrent garbage collector still has garbage collection being triggered when the memory is exhausted. Reachable objects are then traced and copied into another segment of memory. This process is performed by a separate thread so you still need reserving a CPU if you wish the task to be unnoticed. Thanks, -Phil
On Fri, Apr 8, 2011 at 5:43 PM, Phil Bouchard <philippe@fornux.com> wrote:
On 4/8/2011 2:13 PM, Phil Bouchard wrote:
On 4/8/2011 2:06 PM, Phil Bouchard wrote:
On 4/8/2011 1:00 PM, Jeffrey Lee Hellrung, Jr. wrote:
Could you also commit built html documentation?
You can browse the documentation here:
http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
For those not aware a presentation can be found here also:
http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
And an overview is available here:
http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
Please let me know if anything isn't clear enough.
That is a good overview, but there are (obviously) some details yet to be filled in. What I'd like initially to be addressed in the documentation is: - How does the library detect unreferenced cycles? Citing relevant publications, if any, might help. - What does "shifted" mean, and what is "shifted<T>"? These are likely interrelated and addressed to some extent on the mailing list, but I'd like to see it with the rest of the documentation before I look at the library further. To me, the ppt slides don't really add any value over the overview you've recently made available. - Jeff
On 4/8/2011 11:45 PM, Jeffrey Lee Hellrung, Jr. wrote:
That is a good overview, but there are (obviously) some details yet to be filled in. What I'd like initially to be addressed in the documentation is: - How does the library detect unreferenced cycles? Citing relevant publications, if any, might help. - What does "shifted" mean, and what is "shifted<T>"? These are likely interrelated and addressed to some extent on the mailing list, but I'd like to see it with the rest of the documentation before I look at the library further. To me, the ppt slides don't really add any value over the overview you've recently made available.
I'll add this information to the rationale section shortly. Thanks for your input. -Phil
On 4/8/2011 11:45 PM, Jeffrey Lee Hellrung, Jr. wrote:
That is a good overview, but there are (obviously) some details yet to be filled in. What I'd like initially to be addressed in the documentation is: - How does the library detect unreferenced cycles? Citing relevant publications, if any, might help. - What does "shifted" mean, and what is "shifted<T>"? These are likely interrelated and addressed to some extent on the mailing list, but I'd like to see it with the rest of the documentation before I look at the library further. To me, the ppt slides don't really add any value over the overview you've recently made available.
Please see: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... Thanks, -Phil
On 4/9/2011 12:46 PM, Phil Bouchard wrote:
Please see: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
I just added a set diagram that might help: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... -Phil
Phil Bouchard wrote:
And an overview is available here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
Please let me know if anything isn't clear enough.
It's not clear how shifted_ptr detects cyclic blocks of memory with no coding overhead. The methods with which I'm familiar require (a) the ability to enumerate all pointer subobjects in an object and (b) the ability to zero these pointers so as to break the cycles. Without some sort of registry that tracks all pointer instances (overhead on construction/destruction) (a) requires cooperation from the pointee, which shifted_ptr does not seem to demand. It's also not clear how cyclic structures are detected immediately. Doing this typically either requires a scan (collection cycle) on every pointer destruction, or some sort of housekeeping on each construction and copy. It is also not clear how well does the cyclic collector work when, in parallel, other threads create and destroy references to the objects being currently scanned.
On 4/9/2011 3:48 AM, Peter Dimov wrote:
Phil Bouchard wrote:
And an overview is available here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
Please let me know if anything isn't clear enough.
It's not clear how shifted_ptr detects cyclic blocks of memory with no coding overhead. The methods with which I'm familiar require (a) the ability to enumerate all pointer subobjects in an object and (b) the ability to zero these pointers so as to break the cycles. Without some sort of registry that tracks all pointer instances (overhead on construction/destruction) (a) requires cooperation from the pointee, which shifted_ptr does not seem to demand.
It stacks up all allocations made on the heap to later track within which memory block a shifted_ptr belong to. This is what pool::top() defined in "sh_owned_base_nt.hpp" does. Once a pointer from the stack is reached then the set counter is shared will all of the newly allocated blocks of memory.
It's also not clear how cyclic structures are detected immediately. Doing this typically either requires a scan (collection cycle) on every pointer destruction, or some sort of housekeeping on each construction and copy.
I haven't added diagrams yet but the text of the rationale can be found here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
It is also not clear how well does the cyclic collector work when, in parallel, other threads create and destroy references to the objects being currently scanned.
The pools are designed to be used by a multithreaded environment and so is the reference counter base class but the manipulation of set haven't been tested yet. Perhaps a mutex can be added to the set counter if an atomic mechanism isn't found. Thanks, -Phil
Phil Bouchard wrote:
http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
"If it is found to be living on the heap then a more complicated mechanism is involved to track the last memory block that was allocated in the same thread, find its address and stack up the set it will be part of until the first shifted_ptr<T> found on the stack is or data segment is initialized, where the new set counter will be shared with all of the newly allocated blocks of memory referencing each other." This doesn't seem very robust. If you have struct X { std::vector< shifted_ptr<A> > v_; }; struct Y { shifted_ptr<A> p_; }; and then do something like X * px = new X; // heap allocation #1 px->v_.reserve( 5 ); // heap allocation #2 Y * py = new Y; // heap allocation #3 py->p_.reset( new A ); // heap allocation #4 px->v_.push_back( py->p_ ); // no heap allocation I see no way for the last shifted_ptr instance (px->v_[0]) to be associated with heap allocation #1 or #2, where it belongs.
On 4/9/2011 12:55 PM, Peter Dimov wrote:
Phil Bouchard wrote:
http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
"If it is found to be living on the heap then a more complicated mechanism is involved to track the last memory block that was allocated in the same thread, find its address and stack up the set it will be part of until the first shifted_ptr<T> found on the stack is or data segment is initialized, where the new set counter will be shared with all of the newly allocated blocks of memory referencing each other."
This doesn't seem very robust. If you have
struct X { std::vector< shifted_ptr<A> > v_; };
struct Y { shifted_ptr<A> p_; };
and then do something like
X * px = new X; // heap allocation #1 px->v_.reserve( 5 ); // heap allocation #2 Y * py = new Y; // heap allocation #3 py->p_.reset( new A ); // heap allocation #4 px->v_.push_back( py->p_ ); // no heap allocation
I see no way for the last shifted_ptr instance (px->v_[0]) to be associated with heap allocation #1 or #2, where it belongs.
(If the allocation isn't made with "new shifted<>" then the member pointers will be treated as if they were living on the stack. This is because "new shifted<>" uses its own pool and only memory blocks allocated within this pool are recognized to be living on the heap.) If for example you used: struct X { std::vector< shifted_ptr<A>, shifted_allocator< shifted_ptr<A> > > v_; }; struct Y { shifted_ptr<A> p_; }; shifted_ptr<X> px = new shifted<X>(); // heap allocation #1 px->v_.reserve( 5 ); // heap allocation #2 shifted_ptr<Y> py = new shifted<Y>(); // heap allocation #3 py->p_.reset( new shifted<A>() ); // heap allocation #4 px->v_.push_back( py->p_ ); // no heap allocation #5 Then we have 2 sets created by #1 and #3 that are unified by the last operation #5. The last operation constructs a shifted_ptr living in set 1 with another shifted_ptr living in set 2. Set 1 and 2 are therefore merged. -Phil
On 4/9/2011 1:27 PM, Phil Bouchard wrote:
py->p_.reset( new shifted<A>() ); // heap allocation #4
In fact reset() should be an affectator: #include <iostream> #include <boost/shifted_ptr.hpp> #include <boost/shifted_allocator.hpp> using namespace std; using namespace boost; struct A { shifted_ptr<A> p; ~A() { std::cout << __FUNCTION__ << std::endl; } }; struct X { std::vector< shifted_ptr<A>, shifted_allocator< shifted_ptr<A> > > v_; ~X() { std::cout << __FUNCTION__ << std::endl; } }; struct Y { shifted_ptr<A> p_; ~Y() { std::cout << __FUNCTION__ << std::endl; } }; int main() { shifted_ptr<X> px = new shifted<X>(); // heap allocation #1 px->v_.reserve( 5 ); // heap allocation #2 shifted_ptr<Y> py = new shifted<Y>(); // heap allocation #3 py->p_ = new shifted<A>(); // heap allocation #4 px->v_.push_back( py->p_ ); // no heap allocation #5 } Outputs: ~Y ~A ~X Thanks, -Phil
Phil Bouchard wrote:
In fact reset() should be an affectator:
I never heard the word "affectator" before. Neither do any of the English -> German dictionaries I have consulted. I noticed that you also use the words "affected" and "reaffected" in your documentation, in places where I can't make any sense of them. So I looked up a "affecter" in a French -> English dictionary. Besides "to affect", it also suggested "to allocate" and "to appoint" as translations. These seem to make more sense in the current context. Regards, Thomas
On 4/10/2011 2:48 PM, Thomas Klimpel wrote:
Phil Bouchard wrote:
In fact reset() should be an affectator:
I never heard the word "affectator" before. Neither do any of the English -> German dictionaries I have consulted. I noticed that you also use the words "affected" and "reaffected" in your documentation, in places where I can't make any sense of them. So I looked up a "affecter" in a French -> English dictionary. Besides "to affect", it also suggested "to allocate" and "to appoint" as translations. These seem to make more sense in the current context.
reset() should be an "assignment"... -Phil
On 4/9/2011 3:48 AM, Peter Dimov wrote:
Phil Bouchard wrote:
And an overview is available here: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
Please let me know if anything isn't clear enough.
It's not clear how shifted_ptr detects cyclic blocks of memory with no coding overhead. The methods with which I'm familiar require (a) the ability to enumerate all pointer subobjects in an object and (b) the ability to zero these pointers so as to break the cycles. Without some sort of registry that tracks all pointer instances (overhead on construction/destruction) (a) requires cooperation from the pointee, which shifted_ptr does not seem to demand.
It's also not clear how cyclic structures are detected immediately. Doing this typically either requires a scan (collection cycle) on every pointer destruction, or some sort of housekeeping on each construction and copy.
It is also not clear how well does the cyclic collector work when, in parallel, other threads create and destroy references to the objects being currently scanned.
We clearly see in the following diagram the mechanism behind the detection of cyclic structures: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... -Phil
On 4/10/2011 4:13 PM, Phil Bouchard wrote:
We clearly see in the following diagram the mechanism behind the detection of cyclic structures: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc...
And we see here how the unification of 2 sets works: http://www.fornux.com/personal/philippe/devel/shifted_ptr/libs/smart_ptr/doc... Please let me know if I am missing anything. Thanks, -Phil
participants (5)
-
Jeffrey Lee Hellrung, Jr. -
Peter Dimov -
Phil Bouchard -
Scott McMurray -
Thomas Klimpel