Header file granularity policy?

newer
boost::bind and member template...

older
Compile 32 bits on 64 arch how-to ?

Scott Meyers

19 Aug 2006 19 Aug '06

5:49 a.m.

Some Boost libraries, such as SmartPtr and Variant, offer a single header with declarations for all library functionality as well as a set of headers for subsets of functionality. For example, smart_ptr.hpp declares all of shared_ptr, shared_array, weak_ptr, scoped_ptr, scoped_array, and intrusive_ptr. If I don't want all these declarations, I can use more specific headers, e.g., shared_ptr.hpp or scoped_array.hpp. Such libraries allow me to choose between the convenience of a single comprehensive header or the precision of multiple targeted headers. Other libraries seem to offer only the collection of targeted headers. For example, FileSystem offers five headers, but there doesn't seem to be a comprehensive filesystem.hpp. Similarly, Lambda seems to offer only targeted headers; there doesn't seem to be a single "declare everything in Lambda" header. (There is a lambda.hpp, but it's not comprehensive, e.g., it doesn't declare Lambda's bind.) I find that this kind of inconsistency among libraries degrades my experience as a Boost user, especially when I'm first learning to use a library. More than once I've spent time puzzling over a compilation error, only to discover that my mistake was not including a needed header. (I seem to make this error most commonly when using Lambda.) Is there some kind of Boost convention or policy on whether a library should offer a comprehensive "declare everything in the library" header in addition to more targeted headers? Thanks, Scott

Show replies by date

Jeff Garland

19 Aug 19 Aug

6:39 a.m.

Scott Meyers wrote:

...

Some Boost libraries, such as SmartPtr and Variant, offer a single header with declarations for all library functionality as well as a set of headers for subsets of functionality. For example, smart_ptr.hpp declares all of shared_ptr, shared_array, weak_ptr, scoped_ptr, scoped_array, and intrusive_ptr. If I don't want all these declarations, I can use more specific headers, e.g., shared_ptr.hpp or scoped_array.hpp. Such libraries allow me to choose between the convenience of a single comprehensive header or the precision of multiple targeted headers.

Other libraries seem to offer only the collection of targeted headers. For example, FileSystem offers five headers, but there doesn't seem to be a comprehensive filesystem.hpp.

I think you missed the boost/filesystem.hpp -- I believe it includes "everything" except the fstream.hpp.

...

Similarly, Lambda seems to offer only targeted headers; there doesn't seem to be a single "declare everything in Lambda" header. (There is a lambda.hpp, but it's not comprehensive, e.g., it doesn't declare Lambda's bind.)

Ok.

...

I find that this kind of inconsistency among libraries degrades my experience as a Boost user, especially when I'm first learning to use a library. More than once I've spent time puzzling over a compilation error, only to discover that my mistake was not including a needed header. (I seem to make this error most commonly when using Lambda.)

Yeah, that's happened to me before too.

...

Is there some kind of Boost convention or policy on whether a library should offer a comprehensive "declare everything in the library" header in addition to more targeted headers?

There are header and directory policies: http://www.boost.org/more/header.htm http://www.boost.org/more/lib_guide.htm#Directory_structure Unfortunately, it doesn't mention these issues. Overall I'd say that the trend is toward granular headers for the usual dependency / design issues with a single all-inclusive header...but, as you point out it's not 100% consistent. There are special cases -- like libraries that provide serialization support normally don't put this in the 'all-inclusive' header because they don't want to drag in all of the serialization stuff unless it's really needed. Or the fstream case in filesystem which avoid the iostream headers unless you want to use the fstream with filesystem::path. I think it would be good to improve on this. Care to take a crack at writing some guidelines for the docs? Jeff

Edward Diener

12:33 p.m.

Jeff Garland wrote:

...

Scott Meyers wrote:

...
Some Boost libraries, such as SmartPtr and Variant, offer a single header with declarations for all library functionality as well as a set of headers for subsets of functionality. For example, smart_ptr.hpp declares all of shared_ptr, shared_array, weak_ptr, scoped_ptr, scoped_array, and intrusive_ptr. If I don't want all these declarations, I can use more specific headers, e.g., shared_ptr.hpp or scoped_array.hpp. Such libraries allow me to choose between the convenience of a single comprehensive header or the precision of multiple targeted headers.

Other libraries seem to offer only the collection of targeted headers. For example, FileSystem offers five headers, but there doesn't seem to be a comprehensive filesystem.hpp.

I think you missed the boost/filesystem.hpp -- I believe it includes "everything" except the fstream.hpp.

...
Similarly, Lambda seems to offer only targeted headers; there doesn't seem to be a single "declare everything in Lambda" header. (There is a lambda.hpp, but it's not comprehensive, e.g., it doesn't declare Lambda's bind.)

Ok.

...
I find that this kind of inconsistency among libraries degrades my experience as a Boost user, especially when I'm first learning to use a library. More than once I've spent time puzzling over a compilation error, only to discover that my mistake was not including a needed header. (I seem to make this error most commonly when using Lambda.)

Yeah, that's happened to me before too.

...
Is there some kind of Boost convention or policy on whether a library should offer a comprehensive "declare everything in the library" header in addition to more targeted headers?

There are header and directory policies:

http://www.boost.org/more/header.htm http://www.boost.org/more/lib_guide.htm#Directory_structure

Unfortunately, it doesn't mention these issues. Overall I'd say that the trend is toward granular headers for the usual dependency / design issues with a single all-inclusive header...but, as you point out it's not 100% consistent. There are special cases -- like libraries that provide serialization support normally don't put this in the 'all-inclusive' header because they don't want to drag in all of the serialization stuff unless it's really needed. Or the fstream case in filesystem which avoid the iostream headers unless you want to use the fstream with filesystem::path.

I think it would be good to improve on this. Care to take a crack at writing some guidelines for the docs?

Each library can offer an all-enclusive header file, with a proviso that the particular header is overkill if one only wants to do a subset of what the library offers at any one time. From a user's point of view I think this would be good for all libraries to do. Some programmers do not care, at least initially, that all headers might be included, even if it increases compile time, as long as they can get their code to compile/link. Only after they become accustomed to a particular library do they care about limiting the headers being included for the particular usage at hand.

Robert Ramey

3:26 p.m.

Jeff Garland wrote:

...

...
Is there some kind of Boost convention or policy on whether a library should offer a comprehensive "declare everything in the library" header in addition to more targeted headers?

Personally, I avoid the "comprehensive" headers. I personally feel uncomfortable with the idea if including unused stuff as a side effect. This is especially true now that this extra stuff may include templates. This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat. So, I think the issue would be best addressed by discouraging the creation and usage of "comprehensive" headers. Of course, for me that is not a big issue as I avoid their usage in any case. Now, that does raise the issue that sometimes a compilation error arise for the lack of a specific header and that the error message is non-obvious. I'm not sure, but perhaps this can be alleviated by more/better usage of concept checking in library headers.

...

There are header and directory policies:

http://www.boost.org/more/header.htm

LOL - amazing that I had never seen this before. I think this needs a little updating. It seems to me that the creation of something like #include would be preferred to #include unless boost/furball.hpp is merely a "comprehensive" library. I suppose this is how the guidlines should be enhanced. Robert Ramey

Jeff Flinn

4:47 p.m.

"Robert Ramey" wrote in message news:ec7aiq$mni$1@sea.gmane.org...

...

Jeff Garland wrote:

...
...
Is there some kind of Boost convention or policy on whether a library should offer a comprehensive "declare everything in the library" header in addition to more targeted headers?

Personally, I avoid the "comprehensive" headers. I personally feel uncomfortable with the idea if including unused stuff as a side effect. This is especially true now that this extra stuff may include templates.

This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat.

So, I think the issue would be best addressed by discouraging the creation and usage of "comprehensive" headers. Of course, for me that is not a big issue as I avoid their usage in any case.

IMO, lack of comprehnsive headers make those libraries more difficult to learn, and more intimidating than need be. Typically, when learning a new library, many coding shortcuts are taken in the context of a simple test program that would not be dreamt of in the context of a complete application. This is all done in an effort to focus on mastering the main points of the library. Later the other details are addressed when one is confident of the main points of the library. Optimizing compilation time by identifying the less encompassing header files falls in this stage. I certainly appreciated this approach when first learnig the function library on VC6(which required the compatible approach at that time). Later to improve compile times I switched from the encompassing function.hpp to the specific funttion0.hpp ... function3.hpp. Jeff Flinn

Jeff Garland

20 Aug 20 Aug

2:28 p.m.

Robert Ramey wrote:

...

Jeff Garland wrote:

...
...
Is there some kind of Boost convention or policy on whether a library should offer a comprehensive "declare everything in the library" header in addition to more targeted headers?

Personally, I avoid the "comprehensive" headers. I personally feel uncomfortable with the idea if including unused stuff as a side effect. This is especially true now that this extra stuff may include templates.

Hmm, most template code amounts to nothing other than a slightly slower compile if you don't instantiate it. That's certainly the case for date-time.

...

This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat.

I think a few years ago I would have agreed with this, but I think the "size" issues just don't apply that much on modern machines.

...

So, I think the issue would be best addressed by discouraging the creation and usage of "comprehensive" headers. Of course, for me that is not a big issue as I avoid their usage in any case.

...

Now, that does raise the issue that sometimes a compilation error arise for the lack of a specific header and that the error message is non-obvious. I'm not sure, but perhaps this can be alleviated by more/better usage of concept checking in library headers.

I don't see how... I think the ultimate cost is minimal -- a simple statement in the docs explaining the costs in the docs and the same as a comment in the comprehensive header. As long as the fine grain headers are available those can continue to be used. Jeff

David Abrahams

9:54 p.m.

Jeff Garland writes:

...

...
This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat.

I think a few years ago I would have agreed with this, but I think the "size" issues just don't apply that much on modern machines.

In general, Jeff, I agree with you, but in this case I have to agree with Robert. Because of the way the serialization library is constructed, including another archive can really result in the generation of substantial code. It's not just "size" but actual size. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Robert Ramey

21 Aug 21 Aug

12:57 a.m.

David Abrahams wrote:

...

Jeff Garland writes:

...
...
This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat.

I think a few years ago I would have agreed with this, but I think the "size" issues just don't apply that much on modern machines.

In general, Jeff, I agree with you, but in this case I have to agree with Robert. Because of the way the serialization library is constructed, including another archive can really result in the generation of substantial code. It's not just "size" but actual size.

Actually, the serialization is an unusual example in this regard so maybe its not the best example for discussion. But using this logic, wouldn't we want to have a header stl.hpp which includes all the stl libraries or all the stream libraries. Again, where is the line drawn when deciding wath to collect. Regardless of what boost does there are always gong to be a lot of headers which don't fit in some group. Finally, there is a big issue for. If I inherit a piece of code and for some reason I need to investigate how it works, I now have to go through a lot more code to decide what is relevant. I don't like that. Smaller programs are better programs, and "comprehensive" headers make programs bigger while making them look smaller. I especially don't like this. Robert Ramey

Jeff Garland

1:45 a.m.

Robert Ramey wrote:

...

David Abrahams wrote:

...
Jeff Garland writes:

...
...
This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat. I think a few years ago I would have agreed with this, but I think the "size" issues just don't apply that much on modern machines. In general, Jeff, I agree with you, but in this case I have to agree with Robert. Because of the way the serialization library is constructed, including another archive can really result in the generation of substantial code. It's not just "size" but actual size.

Sure, but that's optimization. There's learning how to write a serialization program and then there's learning how to writing an optimized serialization program. Here's some code from my new serialization tutorial: //this is expensive see granular_headers (hyperlink) for details #include

...

Actually, the serialization is an unusual example in this regard so maybe its not the best example for discussion.

I expect that this is the reason we don't really have a policy -- because there are little nuances for each library. Turns out date_time doesn't have an all-inclusive header either. My reason was the usual cost of inclusion -- why should I include the timezone header if I'm only using the date. But for my standard proposal there is only one header: datetime.

...

But using this logic, wouldn't we want to have a header stl.hpp which includes all the stl libraries or all the stream libraries. Again, where is the line drawn when deciding wath to collect. Regardless of what boost does there are always gong to be a lot of headers which don't fit in some group.

Or maybe boost.hpp. Now there's an extreme for you since boost is bigger than the standard library. So I guess there is some limit.

...

Finally, there is a big issue for. If I inherit a piece of code and for some reason I need to investigate how it works, I now have to go through a lot more code to decide what is relevant. I don't like that. Smaller programs are better programs, and "comprehensive" headers make programs bigger while making them look smaller. I especially don't like this.

Sure smaller is better, but I don't agree that a comprehensive program makes the program bigger. Sure it makes it harder to narrow down where something is, but that's why we have grep. Well, sounds like it's back on someone like Scott to sit back and weigh all the tradeoffs to figure out how this should be. Jeff

David Abrahams

2:55 a.m.

Jeff Garland writes:

...

Robert Ramey wrote:

...
David Abrahams wrote:

...
Jeff Garland writes:

...
...
This is one reason that the serialization library has no such "comprehensive" header defined. The other reason is that it's not clear how much such a header might include - all possible archives? all stl serialization. etc. Finally, at least in the serialization library, the mere inclusion of a header can result in the instantiation of code which is never called - thereby adding to code bloat. I think a few years ago I would have agreed with this, but I think the "size" issues just don't apply that much on modern machines. In general, Jeff, I agree with you, but in this case I have to agree with Robert. Because of the way the serialization library is constructed, including another archive can really result in the generation of substantial code. It's not just "size" but actual size.

Sure, but that's optimization. There's learning how to write a serialization program and then there's learning how to writing an optimized serialization program.

Agreed. I was only responding to your statement that "size" issues just don't apply that much on modern machines In this case they do. It's often appropriate to provide comprehensive headers, but that does eventually break down when libraries become really large (e.g. MPL) or have unusual characteristics (e.g. serialization). It's not a foregone conclusion that in the serialization case, we'd be doing beginners any favors in the long run by providing the comprehensive header. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Moshe Matitya

7:38 a.m.

...

Robert Ramey wrote:

But using this logic, wouldn't we want to have a header stl.hpp which includes all the stl libraries or all the stream libraries.

Absolutely. And in fact, the GNU C++ Library contains exactly that: A header , which #include's all the standard STL headers. Moshe

Scott Meyers

19 Aug 19 Aug

4:57 p.m.

Jeff Garland wrote:

...

I think you missed the boost/filesystem.hpp -- I believe it includes "everything" except the fstream.hpp.

There doesn't seem to be a boost/filesystem.hpp or boost/filesystem/filesystem.hpp in my 1.33.1 distribution. In fact, not anywhere in the distribution. Are you sure it's really there for you?

...

I think it would be good to improve on this. Care to take a crack at writing some guidelines for the docs?

I think library authors should at least be strongly encouraged to create a comprehensive header that typically does little more than include all the headers offered by the library. This may have to be adapted to make sense for some libraries, but the idea should be that if I want to start playing around with the Fridget library, I should be able to include fridget.hpp and get everything I might want to use. This enables an exploratory style of familiarizing oneself with a new library and imposes no cost on people who prefer to use the more granular headers. Scott

Rene Rivera

5:07 p.m.

Scott Meyers wrote:

...

Jeff Garland wrote:

...
I think you missed the boost/filesystem.hpp -- I believe it includes "everything" except the fstream.hpp.

There doesn't seem to be a boost/filesystem.hpp or boost/filesystem/filesystem.hpp in my 1.33.1 distribution. In fact, not anywhere in the distribution. Are you sure it's really there for you?

It's a new header http://boost.cvs.sourceforge.net/boost/boost/boost/filesystem.hpp. So it's going to show up in the 1.34 release. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo

Scott Meyers

5:20 p.m.

Rene Rivera wrote:

...

It's a new header http://boost.cvs.sourceforge.net/boost/boost/boost/filesystem.hpp. So it's going to show up in the 1.34 release.

Is there information somewhere on the status and likely release timeframe for 1.34? Scott

Jeff Garland

20 Aug 20 Aug

2:09 p.m.

Scott Meyers wrote:

...

Rene Rivera wrote:

...
It's a new header http://boost.cvs.sourceforge.net/boost/boost/boost/filesystem.hpp. So it's going to show up in the 1.34 release.

Yep, sorry I looked at the cvs version...

...

Is there information somewhere on the status and likely release timeframe for 1.34?

ASAP -- whenever the last regression tests get fixed, etc. It's been dragging on for months now so I suppose some people may be starting to doubt it will ever happen. Jeff

Dave Steffen

21 Aug 21 Aug

3:27 p.m.

Jeff Garland writes:

...

Scott Meyers wrote: [...]

...

...
I find that this kind of inconsistency among libraries degrades my experience as a Boost user, especially when I'm first learning to use a library. More than once I've spent time puzzling over a compilation error, only to discover that my mistake was not including a needed header. (I seem to make this error most commonly when using Lambda.)

Yeah, that's happened to me before too.

Me too (never used Lambda, but uBlas has, or at least had, similar problems); however, I attributed my problem not to an inconsistent Boost policy, but to a deficiency in the documentation, which (sometimes) tells you everything you might ever want to know except which headers to include. :-) ---------------------------------------------------------------------- Dave Steffen, Ph.D. Software Engineer IV Disobey this command! Numerica Corporation - Douglas Hofstadter dgsteffen at numerica dot us

6661

Age (days ago)

6663

Last active (days ago)

List overview

Download

15 comments

9 participants

participants (9)

Dave Steffen
David Abrahams
Edward Diener
Jeff Flinn
Jeff Garland
Moshe Matitya
Rene Rivera
Robert Ramey
Scott Meyers