On 2016-05-29 00:25, Chris Glover wrote:

...

https://github.com/yet-another-user/pimpl

I would personally use this, I am sure of that, as I type this in by hand using unique_ptr all the time.

Glad to hear that... and IMO the proposed design does have advantage over unique_ptr-based pimpl as IMO unique_ptr hardly has any advantage over the raw pointer -- even the destructor has to be explicit non-default and non-inlined. When the proposed design IMO cuts down on implementation minutia.

One addition I would like to see is a version where the heap allocation is avoided by storing an internal buffer in the base class.

I feel that the proposed policy/manager-based design allows us to supply a manager as per your requirements. I personally have never used such design but I am certainly eager to see it implemented, tested and incorporated... if pimpl gets its "foot in the Boost door" so to speak.

On 2016-06-01 10:25, Howard Hinnant wrote:

On May 28, 2016, at 10:35 PM, Vladimir Batov wrote:

...

and IMO the proposed design does have advantage over unique_ptr-based pimpl as IMO unique_ptr hardly has any advantage over the raw pointer -- even the destructor has to be explicit non-default and non-inlined.

<nitpick> You keep saying this and it is like fingernails on a chalkboard to me. The correct statement is not that far off of what you’re saying, and does not invalidate your point.

The unique_ptr-based pimpl has to have an outlined destructor, but it can (and should) be defaulted:

Book::~Book() = default;

Howard, my most humble apologies. English is not my first and its subtleties stubbornly escape me. The least of all I sought to degrade std::unique_ptr. Truly. Unreservedly. I cringed myself when I read what I wrote because I left out an important part -- "in pimpl-related context". And I keep saying those things not because I am trying to degrade unique_ptr but because IMO unique_ptr was developed for a very different purpose. It fits the purpose perfectly... it's just not a good fit for the pimpl idiom... something Sutter suggests in his GotWs over and over again. As for Book::~Book() = default; I am afraid I have to disagree. I suspect that won't work. Because the compiler will try to inline ~Book() (i.e. call unique_ptrBook::implementation destructor) and for that Book::implementation needs to be complete. So, ~Book() even with an empty body needs to be explicit in the implementation file. Am I right?

On 2016-06-01 11:08, Howard Hinnant wrote:

... It has to be outlined, in the source.cpp: ... Book::~Book() = default; Book::Book() = default;

Uh, yes, indeed. I got it now. I'll adjust the wording in the docs. Somewhat off the topic. From efficiency point of view the above is identical to Book::~Book() {} or compiler writers do more magic with "=default"? I like "=default" as I feel it's cleaner language-wise. What weighty reason can I give to a beginner to prefer "=default" over {}... apart from "I am older, musclier and your boss" :-)

On 2016-06-01 11:57, Howard Hinnant wrote:

On May 31, 2016, at 9:51 PM, Vladimir Batov

...

From efficiency point of view the above is identical to

Book::~Book() {}

or compiler writers do more magic with "=default"? I like "=default" as I feel it's cleaner language-wise. What weighty reason can I give to a beginner to prefer "=default" over {}... apart from "I am older, musclier and your boss" :-)

That’s a good question. It doesn’t turn the destructor from non-trivial to trivial. It does not change the noexcept or constexpr status. The only thing it really does is say: I want default semantics here. Maintenance programmers should be more leery of adding superfluous stuff to your destructor.

Good point. Indeed, it's just too easy/tempting to stick something between the braces. :-)

Peter Dimov writes:

Howard Hinnant wrote:

...

It has to be outlined, in the source.cpp:

class Book { struct implementation; std::unique_ptr<implementation> impl_; public: ~Book(); Book(); };

You could probably do

// hpp class Book { struct implementation; std::unique_ptr impl_; public: Book(); };

// cpp Book::Book(): impl_( new implementation, checked_delete<implementation> ) {}

Sorry if this is a silly question, but wouldn't this double the size of member variable impl_ (two pointers), just to save some typing? To me it doesn't look like a good tradeoff. I'm quietly following this thread, and I still don't get why std::unique_ptr is not good enough for heap-allocated pimpl.

On 05/30/2016 10:12 AM, Gavin Lambert wrote:

On 29/05/2016 02:25, Chris Glover wrote:

...

One addition I would like to see is a version where the heap allocation is avoided by storing an internal buffer in the base class. Yes, this means one needs to keep the size in sync between the impl and the header, but this can be an important optimization sometimes.

Ideally the header should store a max size / capacity -- construction succeeds as long as the "real" impl is equal to or smaller than this. This allows a bit of flexibility for different layouts used by different compilers, or for "expansion room" without breaking consumers.

Though you don't want to leave too much wiggle room. Memory is cheap but cache is less so.

I am personally very open to that idea. Although I would probably suggest using custom allocator instead. It can be (almost?) as quick, easily fit to the current design and would not irritate pimpl purists :-) ... Having said that I feel that discussing in depth another pimpl::manager is somewhat premature... although I am hoping we'll bet to that point. In that light I feel that our experience with boost::convert was quite positive: 1) reviewed; 2) accepted in principle; 3) extended, improved by a group of enthusiasts who knew that their effort won't be wasted/dismissed -- that's important IMO as that's the only reward we can offer.

On 05/30/2016 10:08 AM, Gavin Lambert wrote:

On 28/05/2016 17:41, Vladimir Batov wrote:

...

I feel that pimpl and its deployment pattern need to be codified, described, "standardized" and in our toolboxes to stop/avoid everyone re-inventing the pimpl and making the same mistakes. IMO Boost is best positioned for that.

Do you think we might review what I've got and/or maybe collectively come up with something better... It's no MPL or Hana but as std::unique_ptr it's one of the first "little things" I personally reach out for in my everyday work. IMO having pimpl in Boost would save quite a few hours of frustration for many.

I like the idea, certainly. My main concerns about Boost-ification of this are:

1. The way the docs are structured suggest that the "natural" implementation is the shared one and the unique implementation is an extension. Standard C++ language and performance guidelines suggest the reverse should be preferred (or as Chris suggested, one that avoids heap allocation entirely). (This is mostly just a doc issue; the actual implementation seems neutral.)

As you point out the described is "just a doc issue". In that light I do not quite understand how it can be a "main concern". I'll certainly be re-working docs when/if the time comes. At this point IMO we should decide if we want pimpl in Boost and if the suggested design could be used as the starting point.

2. This introduces a symbol (pimpl) into the global namespace, which is probably against Boost guidelines.

Again, I feel you are rushing things. Adapting doccs and putting it into Boost namespace are minor issues to be addressed in due course.

But putting it into the boost namespace doesn't seem like a good solution either as usage requires explicit template specialisation, which is more clunky if the template to be specialised is in a different namespace.

Indeed. Visual Studio did not have that problem for quite some time. GCC had that "clunkiness" a few years ago. Now I try with gcc-4.8: namespace boost { template<class user_type> struct pimpl {...}; } struct Shared : public boost::pimpl<Shared>::shared { ... }; Works fine.

(Though even being in the global namespace doesn't avoid this clunkiness, if the user classes are themselves in a non-global namespace.)

That's something I do not understand. I have plenty of classes like: template<> struct pimplchart::panel::leg::implementation.

On 05/30/2016 11:02 AM, Gavin Lambert wrote:

On 30/05/2016 12:29, Vladimir Batov wrote:

...

Again, I feel you are rushing things. Adapting doccs and putting it into Boost namespace are minor issues to be addressed in due course.

It would be minor if your entire design wasn't based around template specialisation. Since it is, however, it becomes a major consideration since it directly impacts the primary API of the library.

...

On 05/30/2016 10:08 AM, Gavin Lambert wrote:

...

(Though even being in the global namespace doesn't avoid this clunkiness, if the user classes are themselves in a non-global namespace.)

That's something I do not understand. I have plenty of classes like:

template<> struct pimplchart::panel::leg::implementation.

This can't exist inside a "namespace chart { namespace panel {" block.

You are right. It can't be inside a "namespace chart { namespace panel {" block... and I personally never expected it to be. After all that's a specialization of (assume Boost) namespace boost { template<typename user_type> pimpl { struct implementation; } }

If you have the convention that these blocks span the file, it means you either have to put such specialisations at the top or bottom of the file outside the namespace block (which may break locality) or you have to close the namespace blocks and reopen them later (which is ugly).

(Imagine the case where you want to define some detail classes that only exist in the implementation *first*, then the pimpl definition, then the public interface. The first and last should be in the local namespace [or child thereof]; the second can't be. And they have to be defined in that order due to the dependencies.)

Granted implementation files don't *have* to have namespace blocks -- they can use using directives or explicit naming instead; but the latter is ugly and the former risks accidentally defining global symbols, so it's generally safer to use namespace blocks instead.

Yes, I understand. You have your personal programming style and you feel the suggested pimpl does not fit that style as well as you'd like it to. I respect that. But I do not see it as a principal objection... is it a principal objection?

Gavin, Apologies for top-posting... I feel that you make very ambitious generalizations about how users will feel and how ugly in your view the code will be of those who might decide to use the proposed pimpl. I hear your view and I respect it. I just do not happen to share it. No drama. we are all different. So, in your view, should we try and have pimpl in Boost? I think we should as it'd be useful IMO. I proposed the one I use. I am not married to it. If you have a design which makes you happy, let's consider it... Otherwise, we won't get far on criticism only without alternatives to consider. Don't you think? On 05/30/2016 12:32 PM, Gavin Lambert wrote:

On 30/05/2016 13:20, Vladimir Batov wrote:

...

You are right. It can't be inside a "namespace chart { namespace panel {" block... and I personally never expected it to be. After all that's a specialization of (assume Boost)

namespace boost { template<typename user_type> pimpl { struct implementation; } }

Right. And to users, this is going to feel like defining their types (at least the private members, possibly even some method implementations) inside the boost namespace -- and that feeling isn't entirely wrong.

...

...

If you have the convention that these blocks span the file, it means you either have to put such specialisations at the top or bottom of the file outside the namespace block (which may break locality) or you have to close the namespace blocks and reopen them later (which is ugly).

(Imagine the case where you want to define some detail classes that only exist in the implementation *first*, then the pimpl definition, then the public interface. The first and last should be in the local namespace [or child thereof]; the second can't be. And they have to be defined in that order due to the dependencies.)

Granted implementation files don't *have* to have namespace blocks -- they can use using directives or explicit naming instead; but the latter is ugly and the former risks accidentally defining global symbols, so it's generally safer to use namespace blocks instead.

Yes, I understand. You have your personal programming style and you feel the suggested pimpl does not fit that style as well as you'd like it to. I respect that. But I do not see it as a principal objection... is it a principal objection?

I don't think it's a personal programming style issue. I think it's a natural consequence of writing code in the non-global namespace.

I suppose you could consider writing code not in the global namespace as a personal style choice, but at least for library code (which is the most likely to want to use pimpl style so they can avoid ABI breakage with applications) this is the encouraged style. Certainly most (all?) of STL and Boost is itself not in the global namespace.

So (at least in my opinion), the primary users of Pimpl will probably be library developers who want to write code in custom namespaces. But they can't use Pimpl without breaking out of their namespace and writing some code in the global/boost namespace instead. This is a code uglification that seems like it should be a principal objection -- if that doesn't count, what does?

(Some app authors will want to use Pimpl as well of course -- which has the same issues if the namespaces don't match.)

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 2016-05-30 14:18, Gavin Lambert wrote:

On 30/05/2016 15:16, Vladimir Batov wrote:

...

So, in your view, should we try and have pimpl in Boost? I think we should as it'd be useful IMO...

As I said at the start, yes, I think this has potential usefulness and it would be nice to get something like it into Boost.

Excellent.

... As it's the template specialisation which causes most of my concern, my main query is whether it's really necessary or if it could be structured differently to avoid this instead.

... Taking the Book example from the docs... ... we get something like this:

namespace library { namespace detail { ... } }

namespace boost { template<> struct pimpllibrary::Book::implementation { implementation(string const& the_title, string const& the_author) : title(the_title), author(the_author), price(0) {} ... bool check_isbn_10_digit() { ... } ... string title; string author; int price; }; }

namespace library { Book::Book(string const& title, string const& author) : pimpl_type(title, author) {}

string const& Book::author() const { return (*this)->author; } ... } ...

Yes, I understand. That's why I said before and still insist now that that's style... because I personally do not like namespace library { Book::Book(string const& title, string const& author) : pimpl_type(title, author) {} } as you move your code to the right and IMO just waste the space (you fill it with blanks instead of the code). So, I write template<> struct boost::pimpllibrary::Book::implementation { .... } library::Book::Book(string const& title, string const& author) : pimpl_type(title, author) { ... } Then, you might say you do not like fully qualified library::Book::Book(), etc... which in my view is, well, style... for me the "like" and "do not like" are usually the style triggers.

On 05/31/2016 12:56 AM, Rob Stewart wrote:

On May 30, 2016 1:42:39 AM EDT, Vladimir Batov wrote:

...

On 2016-05-30 14:18, Gavin Lambert wrote:

...

it would be nice to get something like it into Boost. Excellent. I, too, think that something that simplifies The Pimpl Idiom could be a good addition to Boost.

Thank you, Robert, for chiming in. Much appreciated. Maybe this time we'll be able to get enough momentum and to actually get something tangible out of it.

Adding The Pimpl Idiom to a class means splitting the implements across two class: one public and one private. Even then, the private class is, normally nested in the public class so they are tightly related. With your library, that split is carried further by splitting the code across two distinct namespaces. That is, indeed, a style issue, since the code still does the same work. Nevertheless it seems wrong to write the implementation details of one's own class in the (proposed) boost namespace.

Yes, I hear you. And, yes, from a certain angle that might seem wrong... On the other hand, from a certain angle anything might seem wrong... :-) Is it a serious design flaw? I personally do not think so. More so, I personally see it from a different angle. Namely, I read the code below as "Book is declared as a pimpl; then I define the implementation of that pimpl-Book". Seems sensible and even natural: struct Book : public pimpl<Book>::shared {}; template<> pimpl<Book>::implementation {}; So, when it is simply read (rather than mechanically dissected) it does not seem that wrong at all... to me :-) The "boost::" prefix may or may not be explicitly present... It is up to coder's preferences and style.

I've not given any thought to alternatives, so I can only criticize your solution at present.

Thank you for mentioning it. I'll hijack your prop to re-iterate that I very much hope we all can keep the discussion *constructive* and ultimately moving forward. That IMO means picking the best available solution... and "no solution" is not one of the choices. I am sure we'll never get anything that everyone likes. However, a solution is immensely better than no solution. IMO it's important that we get *something* in... The seed that the library will grow, mature, improve from. We have plenty of libs that went through several incompatible changes... It did not make them worse. It made them better. I am not sure if all those improvements (for everyone's benefits) were possible if those libs did not have the visibility, exposure and scrutiny which come with the Boost tag.

On 05/31/2016 09:45 AM, Gavin Lambert wrote:

On 31/05/2016 09:37, Vladimir Batov wrote:

...

Yes, I hear you. And, yes, from a certain angle that might seem wrong... On the other hand, from a certain angle anything might seem wrong... :-) Is it a serious design flaw? I personally do not think so. More so, I personally see it from a different angle. Namely, I read the code below as "Book is declared as a pimpl; then I define the implementation of that pimpl-Book". Seems sensible and even natural:

struct Book : public pimpl<Book>::shared {};

template<> pimpl<Book>::implementation {};

So, when it is simply read (rather than mechanically dissected) it does not seem that wrong at all... to me :-)

The "boost::" prefix may or may not be explicitly present... It is up to coder's preferences and style.

The trouble is that the syntax you've written above is not legal C++. In order to provide the implementation of the impl class (assuming that pimpl is in the boost namespace), it *must* be done thusly:

// close any other namespace blocks first namespace boost { template<> struct pimplother::ns::Book::implementation { ... }; }

In particular it is *not* legal to use a boost:: prefix (or even to leave it unprefixed in the presence of a "using namespace boost") -- you have to put it in a namespace block. Otherwise you are declaring a new type in a different namespace, not specialising the existing template.

This is the part that makes it feel like you're actually implementing your class (or at least its private members, anyway) in the boost namespace, and that feels wrong (unless you're a boost library author of course).

OK. Clearly in my code it is not in the boost namespace for obvious reasons... Now, we can over-dramatize things indefinitely but what ultimately matters to me is if the thing works for me. If you insist that the world will end if pimpl is not in the boost namespace, then I'll probably disagree. For those who'll go shirtless unless a shirt has the Gucci tag or it must be inside the boost namespace I do the following (just tested with gcc-4.8): template<class user_type> struct pimpl // It's outside boost! We all gonna die. { } namespace boost { // Hmm, we might probably survive after all. template<class user_type> using pimpl = ::pimpl; } // Look, ma! It all is in the boost namespace. It feels so good. struct Base : public boost::pimpl<Base>::shared { ... }; template<> struct boost::pimpl<Book>::implementation {...};

...

Thank you for mentioning it. I'll hijack your prop to re-iterate that I very much hope we all can keep the discussion *constructive* and ultimately moving forward. That IMO means picking the best available solution... and "no solution" is not one of the choices. I am sure we'll never get anything that everyone likes. However, a solution is immensely better than no solution. IMO it's important that we get *something* in... The seed that the library will grow, mature, improve from. We have plenty of libs that went through several incompatible changes... It did not make them worse. It made them better. I am not sure if all those improvements (for everyone's benefits) were possible if those libs did not have the visibility, exposure and scrutiny which come with the Boost tag.

I agree -- but using template specialisation or not is a fundamental design decision.

I personally love those "I agree -- but". If "we agree", we move forward. If "we agree but", we stay where we were and keep arguing. That is, it's essentially "I disagree", isn't? :-) From my experience "template specialisation" is essential to create a *unique type*.

It's not possible to incrementally improve it later, only to tear it out and replace it with something different.

If it has to be "tear and replace", so be it. People did that for Boost libs and the Standard. We all adjusted and moved on.

My point is that if we can figure out what that something different should be, then now is the best time to make that change, before it gets users whose code would break from changes.

I hear you. Not to be rude but I am still waiting for "that something different" to consider. Without it it's a one-way endless and unfair game -- you have all the fun criticizing, I sweat trying to make you happy. As for the "best time", then I feel you tend to over-dramatize unnecessarily. If we offer something to wide audience (as part of Boost), that wider audience will evaluate the design with everything else following. If that means the current-design acceptance, good. If that means coming up with something better, even better. Seems like win-win for everyone. Don't you agree?

On 2016-05-30 12:36, Emil Dotchevski wrote:

My 2c:

In foo.h, instead of

class foo { class pimpl pimpl * p_; public: foo(); ~foo(); void do_something(); };

It's better to do it the C way: simply leave foo incomplete:

struct foo; foo * create_foo(); void destroy_foo( foo * ); void do_something( foo * );

The above is much improved using shared_ptr:

struct foo; shared_ptr<foo> create_foo(); void do_something( foo * );

If you are saying that pimpl is conceptually simple, then I agree. std::unique_ptr is conceptually simple as well. If you are saying that the snippets you provided are sufficient, then I have to disagree. Say, pimpl in the context of polymorphic hierarchies or value pimpls come to mind.

On 2016-05-31 13:41, Emil Dotchevski wrote:

On Sun, May 29, 2016 at 10:47 PM, Vladimir Batov < Vladimir.Batov@constrainttec.com> wrote:

...

On 2016-05-30 12:36, Emil Dotchevski wrote:

...

It's better to do it the C way: simply leave foo incomplete:

struct foo; foo * create_foo(); void destroy_foo( foo * ); void do_something( foo * );

The above is much improved using shared_ptr:

struct foo; shared_ptr<foo> create_foo(); void do_something( foo * );

... Polymorphism -- the snippets I provided are sufficient, virtual function calls work just fine. For example, in the CPP file you could define do_something (see above declaration) like so:

void do_something( foo * p ) { p->do_something(); //virtual } ...

Indeed. You right... Once we take the methods out to be free-standing functions, there is nothing left of the interface/proxy class but shared_ptr<foo>... I have to admit it never occurred to me... :-) Not sure if I am ready to do things your way :-) but you are most certainly correct. Thank you.

Wow I must admit this surprises me, the resistance I usually get is much stronger. :)

Then I will offer up one point of resistance. I sometimes don't pimpl an entire object. This might be because I have templates in the interface or because I need the functions to inline. Your solution doesn't allow me to do that. It also forces a dynamic allocation which we already discussed could be avoided by using an internal buffer. Other than those thigns, I like this approach. -- chris

I agree that being able to un-pimple a subset of the members in the face of profiler evidence that inlining is necessary is important. I'm not sure why do you say that "my approach" doesn't support that. One possibility is to use inheritance in the cpp file:

header:

struct foo { int critical_; }; shared_ptr<foo> create_foo(); inline void use_foo_critical( foo * p ) { ++p->critical_; } void use_foo_not_so_critical( foo * );

Yes, it seems I missed this. Thanks!

OTOH do note that in the header we may leave shared_ptr incomplete and still declare a shared_ptr factory function. This leads to extremely lean header files:

namespace boost { template <class> class shared_ptr; } struct foo; boost::shared_ptr<foo> create_foo(); void use_foo( foo * );

Emil

You're making a very compelling argument. Actually strong enough that it makes me want the uniform call syntax when I was previously on the fence about that feature. -- chris

On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski wrote:

On Tue, May 31, 2016 at 2:29 PM, Chris Glover wrote:

...

You're making a very compelling argument. Actually strong enough that it makes me want the uniform call syntax when I was previously on the fence about that feature.

I'm generally not in favor of adding stuff to C++. What's the upside in this case? To be able to say p.do_something() instead of do_something(p), because the latter offends Java programmers? :)

The upside is not writing some calls one way and others the other way on the same object, and having to remember which is which. ___ Rob (Sent from my portable computation engine)

On 2016-06-01 07:58, Emil Dotchevski wrote:

On Tue, May 31, 2016 at 2:48 PM, Rob Stewart wrote:

...

The upside is not writing some calls one way and others the other way on the same object, and having to remember which is which.

So, don't use the dot syntax. :)

Indeed. If fact, with "your" approach we can't "use the dot syntax"... And that's the "problem". :-) It's not a criticism. It's merely a fair observation. I myself 've come to C++ with a considerable C experience and I do not "cringe" seeing "your" free-function-based API. Others might object stronger as we are entering the "style area" where people fight tooth and nail over nothing. :-) Even I must say that from the "purist" point of view forcing free-function API kinda violates the very basic OO paradigm -- the association between data and behavior. From practical point of view it's "meh, big deal". Please do not get me wrong. I am not criticizing your approach. In fact, to me it feels surprisingly potent just using the tools we've had "forever". I feel that it is unlikely to "fly" with "general programming population". :-)

On 2016-06-01 09:07, Emil Dotchevski wrote:

... That said, the most important feature of object-oriented design is data encapsulation, ...

I think it needs corrected. I am pretty sure it is "data and behavior encapsulation"... and the "behavior" is something that you take out to be free-standing functions and, actually, violate OO... just saying... not that I lose sleep over it. :-)

...

... I feel that it is unlikely to "fly" with "general programming population". :-)

Yes, I'm aware of that. It doesn't mean they have a point though. :)

I just can't support including yet another can of worms in the already complicated name lookup/implicit instantiation/overload resolution process.

Look who is an orthodox zealot now!.. :-) So, even with understanding that "general public" is unlikely to do it your way you are not to give them anything else... Just saying... Not judging... :-)

A compromise would be to allow the definition of non-friend members, like this:

struct foo; void foo::do_something();

Indeed, that'd be handy to support your pimpl style. I just suspect it violates encapsulation and, well, it's just not in the language.

This language change seems a lot safer than messing with the overload resolution. Perhaps I'm wrong. Either way, I don't consider "but I want to use the dot syntax" a very compelling argument.

I do not feel it is as dumb as "but I want to use the dot syntax". I am beginning to feel that 1) you break the "data+behavior" encapsulation (by taking the methods out) and we reap the immediate benefits of reducing pimpl to mere shared_ptr; exciting; 2) now if I want to bring that "data+behavior" association back, I'll use notional (rather than actual) association wrapping it all in a namespace; why not; 3) after reading the thread I am under impression that as the deployments are getting more complex I need to introduce accessibility (private, public) qualifiers... and the impact of #1 is becoming more apparent as (due to language limitations you might say) we have to re-introduce the broken "data+behavior" association via "friends". That is we seem to be drifting back where we started... but in an unconventional way... which I am not sure will be eagerly embraced. :-)

On Tue, May 31, 2016 at 4:05 PM, rstewart wrote:

...

On Tue, May 31, 2016 at 2:48 PM, Rob Stewart wrote:

...

On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski < emildotchevski@gmail.com> wrote:

...

On Tue, May 31, 2016 at 2:29 PM, Chris Glover wrote:

...

You're making a very compelling argument. Actually strong enough

...

...

...

...

it makes me want the uniform call syntax when I was previously on

"Emil Dotchevski" wrote: that the

...

...

...

...

fence about that feature.

I'm generally not in favor of adding stuff to C++. What's the upside in this case? To be able to say p.do_something() instead of do_something(p), because the latter offends Java programmers? :)

The upside is not writing some calls one way and others the other way on the same object, and having to remember which is which.

So, don't use the dot syntax. :)

Not all functions can be non-members.

Do you mean e.g. virtual functions? They can be hidden behind free function wrappers. Emil

On 1/06/2016 11:05, rstewart wrote:

...

...

The upside is not writing some calls one way and others the other way on the same object, and having to remember which is which.

So, don't use the dot syntax. :)

Not all functions can be non-members.

Operators can be; so the only types of functions I'm aware of that must be members are the constructors (regular, copy, and move). But constructors simply don't exist with this design, since you use factory methods in place of regular constructors, and copy/move construction is not possible with an incomplete type (which is all that external code will ever see). Moving is not really an issue -- you can move the shared_ptr instead. The same applies to shallow copying (copying the pointer rather than the underlying object). If you want to be able to deep copy the object then this requires an explicit API (typically a "shared_ptr<T> clone(T*)" factory method). This is the reverse of traditional C++ objects which are copyable by default -- these are copyable only where explicitly permitted. I'm sure there will be people who argue that this is a good thing and those that argue that it isn't. :) I've used a similar coding style when writing object-oriented C code, although I admit it would feel weirder doing it in C++ (although this has some natural benefits, such as overloading).

On 1/06/2016 11:55, Emil Dotchevski wrote:

Not in the public interface, but they would exist in the private implementation. Encapsulation is great, all that goodness with constructors establishing invariants, member functions maintaining invariants, exceptions enforcing postconditions -- it's all good, but it is now an implementation detail; all that code lives in the CPP file.

Of course.

...

Moving is not really an issue -- you can move the shared_ptr instead. The same applies to shallow copying (copying the pointer rather than the underlying object).

Yes, but that's different. It's more precise to say that one can't move or copy an incomplete type.

I did say that.

On May 31, 2016 5:10:35 PM EDT, Emil Dotchevski wrote:

I'm not sure why do you say that "my approach" doesn't support that. One possibility is to use inheritance in the cpp file:

header:

struct foo { int critical_; }; shared_ptr<foo> create_foo(); inline void use_foo_critical( foo * p ) { ++p->critical_; } void use_foo_not_so_critical( foo * )

Were you leaving encapsulation as an exercise for the reader? class foo { public: use_critical () { ++critical_; } protected: int critical_; }; Your foo_ can still access critical_. ___ Rob (Sent from my portable computation engine)

"Emil Dotchevski" wrote:

On Tue, May 31, 2016 at 2:45 PM, Rob Stewart wrote:

...

On May 31, 2016 5:10:35 PM EDT, Emil Dotchevski wrote:

...

header:

struct foo { int critical_; }; shared_ptr<foo> create_foo(); inline void use_foo_critical( foo * p ) { ++p->critical_; } void use_foo_not_so_critical( foo * )

Were you leaving encapsulation as an exercise for the reader?

class foo { public: use_critical () { ++critical_; } protected: int critical_; };

Your foo_ can still access critical_.

Right, though my preference is as follows.

//Public interface:

namespace boost { template <class> class shared_ptr; } class foo; boost::shared_ptr<foo> create_foo(); void use_foo_critical( foo * ); void use_foo_not_so_critical( foo * );

//Implementation details:

class foo { foo( foo const & ); foo & operator=( foo const & ); int critical_; friend void use_foo_critical( foo * ); protected: foo(); ~foo(); }; inline void use_foo_critical( foo * p ) { ++p->critical_; }

That doesn't provide inline access to those with access only to the header.

On May 31, 2016 7:13:53 PM EDT, Emil Dotchevski wrote:

On Tue, May 31, 2016 at 4:07 PM, rstewart wrote:

...

"Emil Dotchevski" wrote:

...

On Tue, May 31, 2016 at 2:45 PM, Rob Stewart wrote:

...

On May 31, 2016 5:10:35 PM EDT, Emil Dotchevski < emildotchevski@gmail.com> wrote:

...

header:

struct foo { int critical_; }; shared_ptr<foo> create_foo(); inline void use_foo_critical( foo * p ) { ++p->critical_; } void use_foo_not_so_critical( foo * )

Were you leaving encapsulation as an exercise for the reader?

class foo { public: use_critical () { ++critical_; } protected: int critical_; };

Your foo_ can still access critical_.

Right, though my preference is as follows.

//Public interface:

namespace boost { template <class> class shared_ptr; }

You can't do the same thing with std::shared_ptr. Besides, the user of your foo will need the definition of the same shared_ptr as does create_foo(), do you save anything by avoiding the include directive?

...

...

class foo; boost::shared_ptr<foo> create_foo(); void use_foo_critical( foo * ); void use_foo_not_so_critical( foo * );

//Implementation details:

class foo { foo( foo const & ); foo & operator=( foo const & ); int critical_; friend void use_foo_critical( foo * ); protected: foo(); ~foo(); }; inline void use_foo_critical( foo * p ) { ++p->critical_; }

That doesn't provide inline access to those with access only to the header.

I meant that whole thing being in the header, which is required for inline access (within the CPP file inline is next to pointless since the compiler can easily inline any function.) The point I'm making with "//implementation details" is that the fact that use_foo_critical is defined inline, as well as the exact definition of foo itself, is not part of the interface and thus subject to change.

As you guessed, I assumed that "Implementation details" referred to code in a separate source file. Unlike a detail namespace, a mere comment separates users from what you consider implementation details. The compiler can't help you, though I suppose you can implement, in the header, only what needs exposure for performance reasons. ___ Rob (Sent from my portable computation engine)

On May 31, 2016 5:56:29 PM EDT, Josh Juran wrote:

On May 31, 2016, at 5:10 PM, Emil Dotchevski wrote:

...

On Tue, May 31, 2016 at 12:09 PM, Chris Glover wrote:

...

I sometimes don't pimpl an entire object. This might be because I have templates in the interface or because I need the functions to inline. Your solution doesn't allow me to do that.

One possibility is to use inheritance in the cpp file:

header:

struct foo { int critical_; };

cpp:

namespace { struct foo_: foo { int not_so_critiral_; }; }

Since this uses a derived class instead of a pointer, I've been calling this the "dimpl" idiom. :-)

I like the name. It's a nice play on Herb's "Pimpl". ___ Rob (Sent from my portable computation engine)

On Tue, May 31, 2016 at 4:21 PM, Vladimir Batov < Vladimir.Batov@constrainttec.com> wrote:

On 2016-06-01 07:10, Emil Dotchevski wrote:

...

... header:

struct foo { int critical_; }; shared_ptr<foo> create_foo(); inline void use_foo_critical( foo * p ) { ++p->critical_; } void use_foo_not_so_critical( foo * );

cpp:

namespace { struct foo_: foo { int not_so_critiral_; }; }

shared_ptr<foo> create_foo() { return shared_ptr<foo>(new foo_); }

void use_foo_not_so_critical( foo * p ) { foo_ * q=static_cast(p); ++q->not_so_critical_; } ...

Hmm, we probably need to adjust your example (admittedly written in a hurry) -- "foo" needs to be virtual (given we wanted efficiency),

You mean polymorphic? It could be.

static_cast kills all C++ advances in type-safety...

The use of static_cast in this case is perfectly safe, since there is no way to get to undefined behavior using the public interface only. When I say "there is no way" I mean that static type checking isn't butchered at all.

So, probably it should better be

struct foo_; struct foo { int critical_; shared_ptr; }; shared_ptr<foo> create_foo();

I disagree, I don't see the benefit of exposing foo_ in the header. Emil

On 1/06/2016 09:12, charleyb123 . wrote:

This is why I'm such a fan of CRTP (curiously-recurring-template-pattern), where it should work for this PIMPL design with no virtual and no overhead.

I've been quiet on this thread, but:

*- I like the PIMPL pattern very much *- I like your approach *- I want no overhead *- I vote CRTP *- CRTP also allows template-member-functions (requested by Chris, also desired by me) *- I'm generally disappointed with namespaces

This already uses CRTP, eg: struct T : public pimpl<T>::shared {...}; Is there some other application of this that you actually meant instead?

On 2016-06-01 11:30, charleyb123 . wrote:

... In short, what I like:

*- some kind of "std_pimpl<>" provides a standardized interface, and "plumbing"/implementation. *- CRTP in the "std_impl<>" allows for a zero-cost compile-time implementation-override. *- If necessary, implementation can permit "std_pimpl<>" to be parameterized with merely a "Foo" declaration.

Seems like the proposed pimpl has it all from the list. If not, could you please give an example of what you want but the proposed pimpl falls short. I'll try and see if I can accommodate your case.

On 2016-06-01 05:03, Emil Dotchevski wrote:

... The problem from language design point of view is that in C++ the "dot syntax" can only be used with member functions, but member functions have access to the private data, which means that they must participate in the type definitions (encapsulation), which means they can't be called without the type being complete. The C++ solution to this problem is to use only abstract types with no data members in header files, but that has virtual function call overhead compared to the C-style approach.

It would have been useful for C++ to allow the declaration of non-friend "member" functions outside of the type definition. This would require no change in syntax.

Interesting... and liberating... and dangerous... :-) Aren't you suggesting to actually officially support encapsulation violation? Say, by your book I can write a new free-standing "member" function to your class and cause quite a bit of mischief in there.

On 1/06/2016 11:48, Emil Dotchevski wrote:

No, I said non-friend. Look, there is no semantic difference between allowing non-friend member functions to be added without changes to the type definition on one hand, and allowing non-friend free functions to be added without changes to the type definition on the other hand. Neither breaks the encapsulation; the difference is entirely syntactic.

This is basically what .NET extension methods do -- they're static functions outside of a class declared with some initial parameter of a particular type. They can be invoked either as static methods passing the parameters exactly as declared, or as if they were member functions where the initial parameter is removed from the argument list and placed before the dot instead. eg. public static int Sum(this IEnumerable<int> list), which can be called just as list.Sum() on any IEnumerable<int>. It's purely syntactic but allows for some highly readable code. (Although it's easy to get carried away sometimes.)

On Tue, May 31, 2016 at 6:02 PM, Vladimir Batov < Vladimir.Batov@constrainttec.com> wrote:

On 2016-06-01 09:48, Emil Dotchevski wrote:

...

On Tue, May 31, 2016 at 4:42 PM, Vladimir Batov < Vladimir.Batov@constrainttec.com> wrote:

On 2016-06-01 05:03, Emil Dotchevski wrote:

...

...

...

The problem from language design point of view is that in C++ the "dot syntax" can only be used with member functions, but member functions have access to the private data, which means that they must participate in the type definitions (encapsulation), which means they can't be called without the type being complete. The C++ solution to this problem is to use only abstract types with no data members in header files, but that has virtual function call overhead compared to the C-style approach.

It would have been useful for C++ to allow the declaration of non-friend "member" functions outside of the type definition. This would require no change in syntax.

Interesting... and liberating... and dangerous... :-) Aren't you suggesting to actually officially support encapsulation violation?

No, I said non-friend. Look, there is no semantic difference between allowing non-friend member functions to be added without changes to the type definition on one hand, and allowing non-friend free functions to be added without changes to the type definition on the other hand. Neither breaks the encapsulation; the difference is entirely syntactic.

Say, by your book I can write a new free-standing "member" function to

...

your class and cause quite a bit of mischief in there.

Not without having access to the private/protected stuff, which you wouldn't have.

That's where I think communication breaks. "Friend" to me is essentially a "member" (just declared outside) as it has access to "private".

So, I interpret "non-friend member" as "non-member member" which my "computer does not compute" :-)

What I mean by non-friend member is something that is currently not supported in C++, which is defined/declared/called using the member function syntax, but has no access to the private or protected members of the class. So: struct foo; //incomplete void foo::do_something(); would be semantically the same as: struct foo; //incomplete void do_something( foo * ); the only difference being in syntax: the former would be callable using the dot syntax. Emil

Vladimir Batov wrote:

On 2016-06-01 05:03, Emil Dotchevski wrote:

...

It would have been useful for C++ to allow the declaration of non-friend "member" functions outside of the type definition. This would require no change in syntax.

Interesting... and liberating... and dangerous... :-) Aren't you suggesting to actually officially support encapsulation violation?

These are called "extension methods" and they don't violate encapsulation because they aren't "friends". http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0079r0.pdf

On 1/06/2016 12:35, Vladimir Batov wrote:

I am not sure if "to allow free function invocation syntax to invoke member functions and vice-versa" is exactly what Emil wanted. Quoting from the top -- the "declaration of non-friend "member" functions outside of the type definition". I read it as Emil wants it to be a "member" but declared "outside".

That's what extension methods are though, as I mentioned in my other post. They're free functions declared outside of the class (and thus don't have access to private/protected members) that take the class as their initial (or possibly only) parameter. What makes them actual extension methods is that the language allows them to be called as if they were members -- ie. at the compiler level, it finds this: a.f(b); and if it fails to find a "real" member f on a, but it can find a compatible free function, then it will pretend that the code had said this instead: f(a, b); This allows external code to define additional methods that aren't actually part of a class but can be called as if they were, which enables some quite nice things. For example, imagine a library that could add string algorithms directly to std::string (as far as it appears from the call syntax). Though the real power typically comes from defining extensions for concepts rather than concrete types, though both are useful. (An odd side effect of this, which is both good and bad, is that it also allows well-formed "member" calls on nullptr with a well-defined result.)

On Tue, May 31, 2016 at 7:28 PM, Vladimir Batov < Vladimir.Batov@constrainttec.com> wrote:

On 2016-06-01 11:40, Gavin Lambert wrote:

...

On 1/06/2016 12:35, Vladimir Batov wrote:

...

I am not sure if "to allow free function invocation syntax to invoke member functions and vice-versa" is exactly what Emil wanted. Quoting from the top -- the "declaration of non-friend "member" functions outside of the type definition". I read it as Emil wants it to be a "member" but declared "outside".

That's what extension methods are though, as I mentioned in my other post.

Even though Emil did clarify that he indeed meant "extension methods" functionality, I really feel it begs for a correction as I feel we are dangerously mixing the terms.

Yes I stand corrected. Different syntax, same idea. I still don't think that it's that important of a feature, since my premise is that there are no benefits to the member function call syntax. Emil

On 2016-06-01 20:40, Domagoj Saric wrote:

* +1 for 'standardizing' pimpling (and bringing it up yet again;)

Good.

* -1 for 'breaking namespace usage' (forcing implementation in the global namespace) ...

I believe that's been addressed... From the user perspectives anyway. Your mileage may vary.

* -1 for forcing heap usage...

This is incorrect. The proposed design is policy-based, i.e. configurable/customizeable. Indeed, I personally only implemented heap-based policies that I care about. Care to provide a stack-based policy based on your implementation? That already was one the requests on the thread.

And no, allocators won't ('fully') do it. You still have extra indirection and even more code complexity (allocators which use stack in the current function? ugh...)...

I would not be so rush dismissing allocators. There must be reasons why we have them in the Standard.

* -1 for the 'pointer/value semantics' 'kludge'

Not sure what you mean by that. Anyway, I'll provide some history. As you are aware that pimpl stands for "pointer to implementation". Stress is on "pointer". Kludge or not from the set-go very early implementations of the idiom had pointer-semantics. In fact, they *were* pointers (opaque pointers throughout, say, X Window System). Then, when we had a pimpl-related discussion on the list many moons ago "value semantics" were requested and been provided. Now it's implemented as a manager. If you do not need/like it, you simply do not use it. You provide your own stack-based manager (incidentally with value-semantics) and use it. Does it sound acceptable?

IMO this is completely misplaced, I can't see it serving as anything else but replacing/wrapping the standard arrow/-> (smart)pointer syntax with the dot syntax. If the particular class objects are always to be heap allocated simply use them through a pointer...

I am under impression you've disposed of the pivotal pimpl property -- implementation hiding. Looking at your implementation seems to confirm that view... I could be wrong. If my understanding and reading of your code are correct, then, yes, indeed with no implementation-hiding goal all that code is "completely misplaced" and "serving as anything else but" and unnecessary. I agree.

* -1 for paying for shared_ptr for inherently shared classes of objects Use boost::intrusive_ptr for those (in fact make the 'actual' factory function, that is implemented on the .cpp side, private and have it return a naked pointer, so that it can return it through a register across the ABI, and then wrap on the interface side in an inline member that returns a smart_ptr)...

You might be right and boost::intrusive_ptr is far superior to std::shared_ptr. I personally never liked the intrusiveness of boost::intrusive_ptr. And I suspect that that requirement of boost::intrusive_ptr won't sit well with at least some users as well. As the intrusive_ptr docs state: "As a general rule, if it isn't obvious whether intrusive_ptr better fits your needs than shared_ptr, try a shared_ptr-based design first". And it is not obvious to me at the moment. I believe shared_ptr is as efficient as boost::intrusive_ptr when created with make_shared or allocate_shared (used in the proposal). Well, apart from the 2-pointers' foot-print... which IMO is the proper way to go.

Domagoj, Apologies for top posting. I started writing a point-by-point reply just to realize what it was unlikely to have any impact. The way we view the pimpl idiom, its deployment, applicability scope, acceptable design and overhead are drastically different. You do not seem to like the proposed implementation of "policies", you do not like allocators, you do not like pointer/value semantics, you do not like deployment of std::shared_ptr. And that's fine. I had another look at your implementation and indeed it does seem to hide the implementation. Apologies. It does not seem to provide or to be extendible to support pointer-semantics or polymorphic hierarchies or variable-size structures/records which you seem to dismiss but which are important to me. Therefore, in order to move forward, one possibility is that In addition to the existing managers you write an additional policy/manager as you see it with no allocators, shared_ptr, etc. as part of the proposed policy-based extendible framework. That way people will have a choice of policies which is a good thing IMO as the end product will be useful for us all. Alternatively, you might like to explore the possibility of proposing your design, say, on this list. Your design might well be better and more successful. IMO any outcome is good as I think it'd be useful to have a pimpl entry in Boost. Does it sound reasonable? On 06/06/2016 07:12 AM, Domagoj Saric wrote:

On 1.6.2016. 14:05, Vladimir Batov wrote:

...

...

* -1 for 'breaking namespace usage' (forcing implementation in the global namespace) ...

I believe that's been addressed... From the user perspectives anyway. Your mileage may vary.

Could you please point me to the post(s) that address this? AFAICT you've mostly just dismissed the criticisms as subjective (which I described as an extreme position WRT what can be named a matter of style, especially when more conventional 'styles', that do not pollute the global namespace, have been presented)...

...

...

* -1 for forcing heap usage...

This is incorrect. The proposed design is policy-based, i.e. configurable/customizeable. Indeed, I personally only implemented heap-based policies that I care about. Care to provide a stack-based policy based on your implementation? That already was one the requests on the thread.

OK, I'm sorry, I somehow missed this possibility. I didn't see it offered and the code seemed a bit convoluted to deduce how configurable it actually is - a major culprit being the fact that you've made the individual policies/'managers' members of the master class template. These should be separate, decoupled classes that can live in separate headers and be included only when desired...

...

...

And no, allocators won't ('fully') do it. You still have extra indirection and even more code complexity (allocators which use stack in the current function? ugh...)...

I would not be so rush dismissing allocators. There must be reasons why we have them in the Standard.

Whatever the reasons why we have them, the points raised still hold...

...

...

* -1 for the 'pointer/value semantics' 'kludge'

Not sure what you mean by that. Anyway, I'll provide some history. As you are aware that pimpl stands for "pointer to implementation". Stress is on "pointer". Kludge or not from the set-go very early implementations of the idiom had pointer-semantics. In fact, they *were* pointers (opaque pointers throughout, say, X Window System). Then, when we had a pimpl-related discussion on the list many moons ago "value semantics" were requested and been provided. Now it's implemented as a manager. If you do not need/like it, you simply do not use it. You provide your own stack-based manager (incidentally with value-semantics) and use it. Does it sound acceptable?

No :)

The opaque pointers were/are used as pointers, not through adapters that made them look like references.

I'm aware of the history and how, once you already had an API used by other code, the 'easiest' way to hide the implementation was to 'store' it in a pointer and forward all the calls (requiring little or no change in 'client' code). However, just because that's the 'historical evolutionary path' or that it maps to 'pimpl', a 'cool tongue-in-cheek name' isn't an argument that that's the way it should be (the 'is-ought' distinction):

* Supporting automatic-storage allocation automatically 'already' doesn't square with the pimpl moniker. I'd rather lose or 'misuse' the established idiom name than efficiency (and simpler user code).

* 'Pointer-semantics'/'shallow-copy' types are..um...'unusual' (like having a pointer with the dot syntax).

* If all of a type's implementation is hidden in a pointer-to-impl it means it is always dynamically allocated so just make it so. Instead of class interface { public: void foo(); private: impl * pimpl; };

just do

class interface { public: void foo();

static smart_ptr<interface> create();

protected: interface() = default; ~interface(); };

// in .cpp

class impl : interface { ... };

void interface::foo() { static_cast( *this ).... }

...this is also slightly more efficient as you eliminate one pointer indirection...

...

...

IMO this is completely misplaced, I can't see it serving as anything else but replacing/wrapping the standard arrow/-> (smart)pointer syntax with the dot syntax. If the particular class objects are always to be heap allocated simply use them through a pointer...

I am under impression you've disposed of the pivotal pimpl property -- implementation hiding. Looking at your implementation seems to confirm that view... I could be wrong. If my understanding and reading of your code are correct, then, yes, indeed with no implementation-hiding goal all that code is "completely misplaced" and "serving as anything else but" and unnecessary. I agree.

Even if my code is convoluted it still a priori cannot be the way you understood it - what would it be for then (a pimpl that doesn't pimpl the impl)? :)

My 'pimpl impl' does hide the implementation, in fact it hides even the pimpl's private/implementation side parts from the interface/client side (that's why it has two headers, pimpl.hpp included on the interface side and impl.hpp included on the implementation side) unlike you implementation that has everything, and all the policies/managers, in one header...

...

...

* -1 for paying for shared_ptr for inherently shared classes of objects Use boost::intrusive_ptr for those (in fact make the 'actual' factory function, that is implemented on the .cpp side, private and have it return a naked pointer, so that it can return it through a register across the ABI, and then wrap on the interface side in an inline member that returns a smart_ptr)...

You might be right and boost::intrusive_ptr is far superior to std::shared_ptr.

It is different, not necessarily superior.

...

I personally never liked the intrusiveness of boost::intrusive_ptr.

That's kind of like saying I never liked the sharedness of shared_ptr :)

...

And I suspect that that requirement of boost::intrusive_ptr won't sit well with at least some users as well.

a) It is/would be an implementation detail? b) Why?

...

As the intrusive_ptr docs state: "As a general rule, if it isn't obvious whether intrusive_ptr better fits your needs than shared_ptr, try a shared_ptr-based design first".

That just follows from the 'managed' ('configurability discourages use') mindset behind the whole smart_ptr library.

...

And it is not obvious to me at the moment.

If a type is always allocated on the heap and inherently has shared ownership semantics (e.g. any type deriving from enable_shared_from_this) then 'obviously' reference counting has already 'intruded' it 'by definition'...

...

I believe shared_ptr is as efficient as boost::intrusive_ptr when created with make_shared or allocate_shared (used in the proposal).

Instead of believing (WRT 'things of this world') you may be better served by checking the evidence (e.g. codegen, binary size etc.). intrusive_ptr won't force an indirect deleter, RTTI, aditional pointer and a 'control object' 'on you'...

...

Well, apart from the 2-pointers' foot-print... which IMO is the proper way to go.

Why is that the way to go?

Andrzej, Thank you for your input. It's appreciated. Please find my replies below. On 2016-06-06 18:45, Andrzej Krzemienski wrote:

Hi Vladimir, I am following the discussion about Pimpl here, and it seems to me I understand the idea less and less.

In my view the idea has not changed at all. Not even a little. As I see it the pimpl idea is still 1) to separate interface from the implementation and 2) to hide the implementation. And I personally did not see this thread diverging from these two principal pimpl properties. IMO the discussion revolved around the implementation. Say, Emil (as I understand) was advocating taking the interface separation part to the "extreme" -- eliminating the interface/proxy object altogether and providing the interface via public functions. Domagoj suggested his own alternative implementation also.

My impression is (but I might be wrong here) that people use Pimp for slightly different things and that one solution might not suit all the needs.

Well, I feel that people use pimpl to do #1 (separate) and #2 (hide). If they use pimpl to do something else, then they might be trying the wrong tool. Still, people requirements and priorities are different so, indeed, "one solution might not suit all the needs". That's why the proposal has, in fact, two solutions -- with pointer and value-semantics. More so, this time the proposal is policy-based and extendible. I do not claim to cover every-man requirements and needs. I feel, the design is extendible and sufficiently flexible to accommodate new requirements and needs. That is why I am trying to encourage Domagoj to incorporate his stack-based policy/manager. That way we'd cover all pimpl requirements/expectations that I am aware of so far.

Let me list two use cases that I consider significantly different:

1a. I am selling my own complicated library, and I want to make sure users will never see the source code. I can afford to add some run-time penalty because this is a high-level library and the cost of additional indirection is negligible compared to what the library is doing.

1b. I am a programmer and I use a third party library called LIB1 but I wrap it into a class C because I do not want the users of class C to have any knowledge of LIB1, because next month I want to replace library LIB1 with another library LIB2, and I want to do it seamlessly, without forcing the users of C to recompile anything. Again, the cost of additional indirection is negligible, because what LIB1 and LIB2 do is orders of magnitude more expensive.

2. I want to apply Pimpl to every second class in my program because my code is not performance-critical and I would rather have fast compile-times than fast run-times. A valid trade-off for non-critical parts of the program.

The question is, which of the two (or any other) use cases does your library address?

All three cases seem to be the standard cases for pimpl idiom application as in every case you want separate+hide. I believe the proposed pimpl handles all three. You can deploy pimpl<Foo>::shared or pimpl<Foo>>::unique depending on your circumstances. Better still, if, say, Domagoj decides to contribute his stack-based policy implementation, then, say, pimpl<Foo>::stack might be deployed in applications with stricter efficiency/memory requirements.

Also, I remember that last time the library promoted the use of shared ownership (altering the state of one object immediately alters the states of other objects). The examples from the new dos still seem to imply that. Is it necessary? If for some reason I need the shared ownership I could implement it atop of Pimpl classes using a shared_ptr.

Yes, indeed, I think it is probably a valid point. The thing is I initially wrote it for myself. Probably due to my application-domain specifics I still use pimpl::shared exclusively. So, for me value-pimpl wrapped in shared_ptr seemed like unnecessary inconvenient complexity. If someone else with different priorities decides to submit his value-semantics pimpl and it gets accepted, I'll use it happily as you describe.

But I might be wrong about my understanding of the library. I am really missing the introduction, that would outline the scope of the problems you are addressing, some high level design decisions, and the intended usage.

I tend to agree with you and "some high level design decisions, and the intended usage" would be beneficial as I feel pimpl can be easily mis-used. I would most likely go down that road if pimpl gets into Boost. As it is now I am far from certain if I can justify any additional effort documentation-wise.

For now, the initial example encourages me to use shared semantics, and I feel it is encouraging me to do the wrong thing.

I am not sure I can agree with that. The proposal has two behavioral facets -- pointer-semantics and value-semantics. I had to start the description with one or the other. I happened to choose the pointer-semantics as it's easier to describe as it is shared_ptr-based. After that, again, value-semantics implementation is easier to describe as its underlying implementation mimics shared_ptr's incomplete-type-management functionality. Still, I think you are right sensing that I feel more for pointer-semantics :-) but I would not interpret it as an encouragement to do the wrong thing. :-) Again, I do agree that the documentation needs more "do it", "don't do it", "best practices", etc. I just do not feel like investing more of my time due to high probability of it all wasted.

On Wed, Jun 8, 2016 at 2:29 PM Rainer Deyke wrote:

3. I want a class to have value semantics, but use a copy-on-write implementation behind the scenes.

Pimpl is not about semantics, it's about reducing physical coupling. It's not about the pointer pointing at the private implementation object (which, being private, is an implementation detail anyway), it's about the private implementation *type* being left incomplete. Emil

On Thu, Jun 9, 2016 at 5:19 AM, Rainer Deyke wrote:

On 08.06.2016 23:57, Emil Dotchevski wrote:

...

On Wed, Jun 8, 2016 at 2:29 PM Rainer Deyke wrote:

3. I want a class to have value semantics, but use a copy-on-write

...

implementation behind the scenes.

Pimpl is not about semantics, it's about reducing physical coupling. It's not about the pointer pointing at the private implementation object (which, being private, is an implementation detail anyway), it's about the private implementation *type* being left incomplete.

Pimpl is a technique where the implementation of a class is moved to a separate implementation class (the "impl"), to which the interface class maintains a pointer (the "p").

The "p" stands for "private" not for pointer. The defining property of the pimpl idiom is that the private implementation type is left incomplete in the interface: the pointer is opaque. For example, if you implement copy-on-write using a pointer to a type that is defined in the interface, you're not using pimpl. So, logical design (semantics), including copy semantics, is orthogonal to pimpl. Emil

On 06/10/2016 04:59 AM, Emil Dotchevski wrote:

...

... The defining property of the pimpl idiom is that the private implementation type is left incomplete in the interface: the pointer is opaque...

Emil

Just happened to re-read Gamma's "Design Patterns". The Bridge section... which is the Pimpl... There the "hide the implementation" is mentioned in the Applicability section, i.e. what the pattern is good for... i.e. it's not the pattern's property... it's "only" its applicability... in the context of C++. As for copy-on-write, then Gamma et al explicitly mention it... in the Proxy pattern... which they position separately from Bridge... they do not even mention it in the Related Section... which to me is a bit too subtle... but admittedly subtlety is not my strong suit.