[explore] Library Proposal: Container Streaming
Jared McIntyre, Danny Havenith, and myself, with support from Jeff Garland have developed a library to stream standard containers to output streams. We would like to submit this for review. The requirements for this library were gathered at the BoostCon 2007 Library in a Week sessions. An initial review was held and further requirements were gathered at BoostCon 2008. Everything can be found in the boost vault <http://www.boostpro.com/vault/> . Here is a direct link to explore.zip <http://www.boostpro.com/vault/index.php?action=downloadfile&filename=ex plore.zip&directory=&> . Some quick examples: #include <boost/explore.hpp> # include ... // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); std::cout << vi; // prints [1, 2, 3] // lets do some containers in containers std::vector<std::vector<int> > vvi; vvi.push_back(vi); vvi.push_back(vi); std::cout << vvi; // prints [[1, 2, 3], [1, 2, 3]] // associative containers std::map<std::string, int> si_map; si_map["hello"] = 1; si_map["world"] = 2; std::cout << si_map; // prints [hello:1, world:2] // containers of complex types std::list<date> dl; // date from boost::gregorian dl.push_back(date(2007, Jan, 1)); dl.push_back(date(2007, Jan, 3)); std::cout << dl; // prints [2007-Jan-1, 2007-Jan-3] // how about some boost container types: boost::array<std::string, 2> sa2; sa2[0] = "one"; sa2[1] = "two"; std::cout << sa2; // prints [one, two] Jeffrey Faust
Jeffrey Faust wrote:
Jared McIntyre, Danny Havenith, and myself, with support from Jeff Garland have developed a library to stream standard containers to output streams. We would like to submit this for review. The requirements for this library were gathered at the BoostCon 2007 Library in a Week sessions. An initial review was held and further requirements were gathered at BoostCon 2008.
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing ambiguity.) As a technical question, how are the operators found? They can't be added to std (adding to std is undefined behavior), and there's otherwise no one namespace to put them in that will allow them to be found for all element types (at least with my understanding of ADL). By the way, "Explore" is a very uninformative name. --Jeffrey Bosboom
Jeffrey Bosboom wrote:
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing ambiguity.)
We have no plans to implement input operators. You're correct that there is ambiguity in the form of the stream. There is also ambiguity in the items that are streamed. In general, containers of strings would be problematic.
As a technical question, how are the operators found? They can't be added to std (adding to std is undefined behavior), and there's otherwise no one namespace to put them in that will allow them to be found for all element types (at least with my understanding of ADL).
The operators are added to namespace std, and they are found through ADL. I don't know how to make this work otherwise. I think that as long as all the compilers supported by boost also behave correctly when extending std, we're safe.
By the way, "Explore" is a very uninformative name.
I agree. It came about during BoostCon 2007 when we had some grander ideas about what this library would be. It was to be a way to 'explore' your data, and there were some initial template metaprogramming approaches. The library is ended up in a much more modest form, but the name has remained. Any ideas for a better name? 'container_streaming' is a bit wordy, and even misleading due to the lack of input operators. -- Jeff Faust
Jeffrey Faust wrote:
Jeffrey Bosboom wrote:
As a technical question, how are the operators found? They can't be added to std (adding to std is undefined behavior), and there's otherwise no one namespace to put them in that will allow them to be found for all element types (at least with my understanding of ADL).
The operators are added to namespace std, and they are found through ADL. I don't know how to make this work otherwise.
You could put them in the global namespace? I think that would conform better to the letter of the rules, without really being any safer in practice. John Bytheway
On Mon, Nov 30, 2009 at 4:37 PM, John Bytheway <jbytheway+boost@gmail.com<jbytheway%2Bboost@gmail.com>
wrote:
Jeffrey Faust wrote:
Jeffrey Bosboom wrote:
As a technical question, how are the operators found? They can't be added to std (adding to std is undefined behavior), and there's otherwise no one namespace to put them in that will allow them to be found for all element types (at least with my understanding of ADL).
The operators are added to namespace std, and they are found through ADL. I don't know how to make this work otherwise.
You could put them in the global namespace? I think that would conform better to the letter of the rules, without really being any safer in practice.
I'd rather put them in a library specific namespace in that case, and require the programmer to put a using declaration in the translation unit.
Am Monday 30 November 2009 23:42:34 schrieb Zachary Turner:
You could put them in the global namespace? I think that would conform better to the letter of the rules, without really being any safer in practice.
I'd rather put them in a library specific namespace in that case, and require the programmer to put a using declaration in the translation unit.
precedent: std::rel_ops
On Tue, Dec 1, 2009 at 00:11, Stefan Strasser <strasser@uni-bremen.de> wrote:
Am Monday 30 November 2009 23:42:34 schrieb Zachary Turner:
You could put them in the global namespace? I think that would conform better to the letter of the rules, without really being any safer in practice.
I'd rather put them in a library specific namespace in that case, and require the programmer to put a using declaration in the translation unit.
precedent: std::rel_ops _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
This reminds me of the repr() function in Python, http://docs.python.org/library/functions.html#repr. Maybe Boost.Represent would make sense as name for the project, as it shows a representation of the data. Jeroen Habraken
Jeroen Habraken wrote:
This reminds me of the repr() function in Python, http://docs.python.org/library/functions.html#repr. Maybe Boost.Represent would make sense as name for the project, as it shows a representation of the data.
That's a good idea. I like it more than 'explore'. Still, to me it sounds like such a big word for something so basic in use. Unless you are extending it, there's literally no interface to the library other than operator<<. So I was thinking some more about a better name. What do you think of Boost.Out? -- Jeff Faust
Jeffrey Faust wrote:
Jeroen Habraken wrote:
This reminds me of the repr() function in Python, http://docs.python.org/library/functions.html#repr. Maybe Boost.Represent would make sense as name for the project, as it shows a representation of the data.
That's a good idea. I like it more than 'explore'. Still, to me it sounds like such a big word for something so basic in use. Unless you are extending it, there's literally no interface to the library other than operator<<.
So I was thinking some more about a better name. What do you think of Boost.Out?
I agree that 'represent' is too vague a word (I don't think it's any better than 'explore', really). I like the bluntness of Out, but I expect Out to be more general than just standard containers. (It makes me think of a library that was described as "the opposite of Spirit" for output formatting, although its name is nonobvious enough that I can't recall it.) How about container_output? It's not as succinct as 'represent' or 'out', but it's a more descriptive of what the library actually is, and with utility libraries, it's more about making people aware of the utility than selling them on using it. --Jeffrey Bosboom
Jeffrey Faust wrote;
Jeroen Habraken wrote:
This reminds me of the repr() function in Python, http://docs.python.org/library/functions.html#repr. Maybe Boost.Represent would make sense as name for the project, as it shows a representation of the data.
What do you think of Boost.Out?
PrettyPrint? _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
Jeffrey Faust wrote:
Jeroen Habraken wrote:
This reminds me of the repr() function in Python, http://docs.python.org/library/functions.html#repr. Maybe Boost.Represent would make sense as name for the project, as it shows a representation of the data.
That's a good idea. I like it more than 'explore'. Still, to me it sounds like such a big word for something so basic in use. Unless you are extending it, there's literally no interface to the library other than operator<<.
So I was thinking some more about a better name. What do you think of Boost.Out?
What's wrong with Boost.Repr? Out doesn't seem to quite capture it. When I use operator<< to put something into a string, which I might later send out to the user, I don't think of it going out right then. Depends on what the stream is. Repr captures what it does better, i.e. you get a representation of what the data is. Patrick
Patrick Horgan wrote:
What's wrong with Boost.Repr? Out doesn't seem to quite capture it. When I use operator<< to put something into a string, which I might later send out to the user, I don't think of it going out right then. Depends on what the stream is. Repr captures what it does better, i.e. you get a representation of what the data is.
I'm going to collect these naming recommendations and settle that after the group decided if this would be a useful library or not. I'll follow up later. Thanks, -- Jeff Faust
Zachary Turner wrote:
John Bytheway wrote:
You could put them in the global namespace? I think that would conform better to the letter of the rules, without really being any safer in practice. I'd rather put them in a library specific namespace in that case, and require the programmer to put a using declaration in the translation unit.
Below is an example of why that won't work for all cases. The only way for variant::operator<< to access operator<<(vector) is to rely on ADL and put the operator<< in the std namespace. #include <boost/variant.hpp> #include <iostream> #include <vector> namespace explore { template<typename T> std::ostream& operator<<(std::ostream& ostr, const std::vector<T>& v) { // ... return ostr; } } int main() { using namespace explore; boost::variant<std::vector<int> > v; std::cout << v << std::endl; } -- Jeff Faust
Zachary Turner wrote:
On Mon, Nov 30, 2009 at 4:37 PM, John Bytheway <jbytheway+boost@gmail.com<jbytheway%2Bboost@gmail.com>
wrote:
Jeffrey Bosboom wrote:
As a technical question, how are the operators found? They can't be added to std (adding to std is undefined behavior), and there's otherwise no one namespace to put them in that will allow them to be found for all element types (at least with my understanding of ADL). The operators are added to namespace std, and they are found through ADL. I don't know how to make this work otherwise. You could put them in the global namespace? I think that would conform better to the letter of the rules, without really being any safer in
Jeffrey Faust wrote: practice.
I'd rather put them in a library specific namespace in that case, and require the programmer to put a using declaration in the translation unit.
That's the way Boost.Assign does it, and is the way to go IMHO.
AMDG Mathias Gaunard wrote:
Zachary Turner wrote:
I'd rather put them in a library specific namespace in that case, and require the programmer to put a using declaration in the translation unit.
That's the way Boost.Assign does it, and is the way to go IMHO.
Boost.Assign doesn't need ADL. In Christ, Steven Watanabe
Jeffrey Faust wrote:
Jeffrey Bosboom wrote:
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing
[snip]
By the way, "Explore" is a very uninformative name.
I agree. It came about during BoostCon 2007 when we had some grander ideas about what this library would be. It was to be a way to 'explore' your data, and there were some initial template metaprogramming approaches. The library is ended up in a much more modest form, but the name has remained.
Any ideas for a better name? 'container_streaming' is a bit wordy, and even misleading due to the lack of input operators.
Boost.PrettyPrint, of which the container support is only a part? (the name might sound a bit silly, but has precedence) / Johan
Jeffrey Bosboom wrote:
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing ambiguity.)
how would this be different than just writing a special archive class and using boost seriallization? Robert Ramey
Robert Ramey wrote:
Jeffrey Bosboom wrote:
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing ambiguity.)
how would this be different than just writing a special archive class and using boost seriallization?
Robert Ramey
It didn't require learning to use Boost.Serialization. Also, in the specific case I needed this for, I was running code on a cluster where Boost was not available, so it was lighter-weight (although obviously far less featureful). --Jeffrey Bosboom
Jeffrey Bosboom wrote:
Robert Ramey wrote:
Jeffrey Bosboom wrote:
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing ambiguity.)
how would this be different than just writing a special archive class and using boost seriallization?
Robert Ramey
It didn't require learning to use Boost.Serialization.
Here is what you're example looks like with boost serialization #include <iostream> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp> #include <boost/serialization/list.hpp> #include <boost/serialization/string.hpp> //#include <boost/date_time.hpp> #include <boost/archive/text_oarchive.hpp> int main(int argc, char * argv[]){ boost::archive::text_oarchive output_log(std::cout); // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); output_log << vi; // lets do some containers in containers std::vector<std::vector<int> > vvi; vvi.push_back(vi); vvi.push_back(vi); output_log << vvi; // prints [[1, 2, 3], [1, 2, 3]] // associative containers std::map<std::string, int> si_map; si_map["hello"] = 1; si_map["world"] = 2; output_log << si_map; // prints [hello:1, world:2] #if 0 // containers of complex types std::list<date> dl; // date from boost::gregorian dl.push_back(date(2007, Jan, 1)); dl.push_back(date(2007, Jan, 3)); output_log << dl; // prints [2007-Jan-1, 2007-Jan-3] #endif // how about some boost container types: boost::array<std::string, 2> sa2; sa2[0] = "one"; sa2[1] = "two"; output_log << sa2; // prints [one, two] } I excluded data/time because I didn't have it compiled on my machine. I don't see it as being any more difficult to use/learn than the library you propose. If I were to do this, I would likely make a special "human readable" text archive. This would be a tiny fraction of the effort already expended and produce a result with much more features.
Also, in the specific case I needed this for, I was running code on a cluster where Boost was not available, so it was lighter-weight (although obviously far less featureful).
I don't see how one is going to run boost libraries in an environment were boost is not available. Sorry, I'm just not getting this. Robert Ramey
Robert Ramey wrote:
Robert Ramey
It didn't require learning to use Boost.Serialization.
Here is what you're example looks like with boost serialization
#include <iostream> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp> #include <boost/serialization/list.hpp> #include <boost/serialization/string.hpp> //#include <boost/date_time.hpp> #include <boost/archive/text_oarchive.hpp> int main(int argc, char * argv[]){ boost::archive::text_oarchive output_log(std::cout); // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); output_log << vi;
While this is possible, with a custom archive class, I hope you'll agree that it will be rather heavyweight solution, while a simple output of output of std::vector can be implement in a 3-line template function. - Volodya
Vladimir Prus wrote:
Robert Ramey wrote:
Robert Ramey
It didn't require learning to use Boost.Serialization.
Here is what you're example looks like with boost serialization
#include <iostream> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp> #include <boost/serialization/list.hpp> #include <boost/serialization/string.hpp> //#include <boost/date_time.hpp> #include <boost/archive/text_oarchive.hpp> int main(int argc, char * argv[]){ boost::archive::text_oarchive output_log(std::cout); // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); output_log << vi;
While this is possible, with a custom archive class, I hope you'll agree that it will be rather heavyweight solution,
Hmmm - I realize that the serialization library has a lot of template code. But to me the issue is conceptual "heaviness". In both libraries, the idea is to wrap extra information about types so that they can easily be displayed on a stream. In either library you just write #include ... ... output_stream << data I just don't see the value in re-inventing of a whole new wheel. By the time such a library would pass the boost review process, I'm sure it would be just as heavy weight (or more) than boost serialization. One would have a whole new interface to to something that is less capable. Already the serialization library has an xml_archive as well as text archive.
while a simple output of output of std::vector can be implement in a 3-line template function.
I didn't have to write even one line to template code to implement the example. Robert Ramey
Robert Ramey wrote:
While this is possible, with a custom archive class, I hope you'll agree that it will be rather heavyweight solution,
Hmmm - I realize that the serialization library has a lot of template code. But to me the issue is conceptual "heaviness".
In both libraries, the idea is to wrap extra information about types so that they can easily be displayed on a stream. In either library you just write
#include ... ...
output_stream << data
I just don't see the value in re-inventing of a whole new wheel.
It's an altogether different kind of wheel. Like, I am sure that if somebody suggested to attach 15' car wheels, complete with suspension and studded tires, to your office chair, you would not be exactly thrilled.
By the time such a library would pass the boost review process, I'm sure it would be just as heavy weight (or more) than boost serialization.
This seems fairly unlikely.
One would have a whole new interface to to something that is less capable.
Already the serialization library has an xml_archive as well as text archive.
while a simple output of output of std::vector can be implement in a 3-line template function.
I didn't have to write even one line to template code to implement the example.
I though boost.serialization was written by you? And that code still has to be parsed by the compiler. - Volodya
On Dec 1, 2009, at 12:47 PM, Vladimir Prus wrote:
Robert Ramey wrote:
I didn't have to write even one line to template code to implement the example.
I though boost.serialization was written by you?
The example is quite distinct from the library being used... In other words, there is a special quality of Boost.Serialization that is of importance: it exists. So, we are allowed to use it, and by using it, we have to write very little code ourselves, beside a number of include statements. Great!
And that code still has to be parsed by the compiler.
This is a different issue. Is that your problem, that you think compilers have a problem with some complex constructs inside the Boost.Serialization library? Does the compiler fail? Consume too much resources? Too much time? /David
Responding to two e-mails from David:
And that code still has to be parsed by the compiler.
This is a different issue. Is that your problem, that you think compilers have a problem with some complex constructs inside the Boost.Serialization library? Does the compiler fail? Consume too much resources? Too much time?
And:
Vladimir's claim is that his solution is lighter in two aspects (I think):
1. Easier to use
2. Not dependent on Boost
I hope we all realize that, no, #1 is not the case, and if so, one can create a simple wrapper as the one above to make it easier. So, what about #2? Well, as a proposed Boost library, it is no longer that important.
I agree with Volodya here. In my environment at least, we would not be willing to include all of the serialization headers just to be able to print a vector to the screen. We'd write yet another custom vector printing function instead (which we have). The reason is that we have very large binaries, and if we included the serialization library into many of our compilation units, it would have a non-trivial effect on compilation time. C++ compile times are a concern to many of us, I think - hence the invention of languages such as Go (without the exclamation point). Besides, I like the Explore library as is (other than the name, sorry). Jeremy Pack
On Dec 1, 2009, at 1:21 PM, Jeremy Pack wrote:
Responding to two e-mails from David:
And that code still has to be parsed by the compiler.
This is a different issue. Is that your problem, that you think compilers have a problem with some complex constructs inside the Boost.Serialization library? Does the compiler fail? Consume too much resources? Too much time?
And:
Vladimir's claim is that his solution is lighter in two aspects (I think):
1. Easier to use
2. Not dependent on Boost
I hope we all realize that, no, #1 is not the case, and if so, one can create a simple wrapper as the one above to make it easier. So, what about #2? Well, as a proposed Boost library, it is no longer that important.
I agree with Volodya here. In my environment at least, we would not be willing to include all of the serialization headers just to be able to print a vector to the screen. We'd write yet another custom vector printing function instead (which we have). The reason is that we have very large binaries, and if we included the serialization library into many of our compilation units, it would have a non-trivial effect on compilation time. C++ compile times are a concern to many of us, I think - hence the invention of languages such as Go (without the exclamation point).
Besides, I like the Explore library as is (other than the name, sorry).
So, we have two issues: 1. Compilation time. Have you a guesstimate as to much time is spent on parsing, compiling and linking (see #2 below) Boost.Serialization for a case with N simple container outputs? 2. The extra size on executables (or libraries) given the linked-in; to give you some insight into this, when compiling the examples provided in this thread, the code size increased by 28k when linking the (MT version) of Boost.Serialization with the executable. I assume the issue of "easy to use" is a non-issue, right, i.e., that the examples provided show that one can use very few lexemes in outputting containers using Boost.Serialization? I still believe it is a false notion that Boost.Serialization would add that much more than any non-trivial proposal in terms of compilation time or size of output. If I see some figures regarding #1 and #2 above that scare me - even for dreaded embedded targets :-) - OR a much simpler way to output containers than the boost::archive::text_oarchive(std::cout) << v; in any reasonable typedef'd or wrapped version (e.g., "arch(std::cout) << v"), then I will gladly stand behind a new outputter in our joyous family. /David
David Bergman wrote:
On Dec 1, 2009, at 1:21 PM, Jeremy Pack wrote:
So, we have two issues:
1. Compilation time. Have you a guesstimate as to much time is spent on parsing, compiling and linking (see #2 below) Boost.Serialization for a case with N simple container outputs?
FYI - I ran the indicated test. Time to make changes in code - about a minute Time to compile link and run - undetectable. This might be in part due the the fact that I use boost serialization dll so it already has most of the code pre-compiled. Oh - and BTW it compiled, linked and ran the first time
2. The extra size on executables (or libraries) given the linked-in; to give you some insight into this, when compiling the examples provided in this thread, the code size increased by 28k when linking the (MT version) of Boost.Serialization with the executable.
I still believe it is a false notion that Boost.Serialization would add that much more than any non-trivial proposal in terms of compilation time or size of output.
Again, I think the boost serialization library - and boost in general - - and templated code in general - gets a bad wrap for being "heavy weight". With a little bit of thoughtful design and usage - I never have this problem. Robert Ramey
the whole list is up in arms about printing a vector... Zitat von Jeremy Pack <rostovpack@gmail.com>:
I agree with Volodya here. In my environment at least, we would not be willing to include all of the serialization headers just to be able to print a vector to the screen.
in my environment, I wouldn`t even be willing to look if there`s a boost library that prints a vector for me. is printing containers all it does? then it`s argueable just filling an omission of the STL container library, and could be part of Boost.Container, which is already on the review schedule. when both libraries are accepted seperately it would be like boost had Boost.Variant and Boost.OutputVariant, DateTime/OutputDateTime, ... until then, STL can handle most cases: copy(V.begin(), V.end(), ostream_iterator<int>(cout, ","));
Here is what you're example looks like with boost serialization
link an external library for printing a vector?
Yes, too much time and resources. When building debug variant, an program that does nothing but creates std::vector<int> is a 53K object file. When I add serialization, I get: [...] - a 387K object file
that doesn't mean anything, does it? boost.serialization creates huge call stacks and instantiates dozends if not hundreds of trivial functions, that are all removed in a non-debug build.
Zitat von Robert Ramey <ramey@rrsd.com>:
strasser@uni-bremen.de wrote:
link an external library for printing a vector?
aaaa... try printing a vector without linking the C runtime library.
you know the difference. I don't understand why you even brought up using your library for that. if I may link to a document written by a certain library author: http://www.boost.org/doc/libs/1_40_0/libs/serialization/doc/rationale.html#a... archives are not streams, and serialization is not streaming. boost.serialization was never intended to replace all operator<<()'s in the world, was it?
strasser@uni-bremen.de wrote:
the whole list is up in arms about printing a vector...
Zitat von Jeremy Pack <rostovpack@gmail.com>:
I agree with Volodya here. In my environment at least, we would not be willing to include all of the serialization headers just to be able to print a vector to the screen.
in my environment, I wouldn`t even be willing to look if there`s a boost library that prints a vector for me. is printing containers all it does? then it`s argueable just filling an omission of the STL container library, and could be part of Boost.Container, which is already on the review schedule. when both libraries are accepted seperately it would be like boost had Boost.Variant and Boost.OutputVariant, DateTime/OutputDateTime, ...
until then, STL can handle most cases: copy(V.begin(), V.end(), ostream_iterator<int>(cout, ","));
Not very pretty with that trailing ",", IMHO of course. Regardless of the approach taken, I think this would be a useful facility. I've created a similar, albeit much simplified facility for two separate projects in the last 5 years. There've been several more projects that could have used the more general facilities that this lib offers. Jeff
Zitat von Jeff Flinn <TriumphSprint2000@hotmail.com>:
until then, STL can handle most cases: copy(V.begin(), V.end(), ostream_iterator<int>(cout, ","));
Not very pretty with that trailing ",", IMHO of course.
Regardless of the approach taken, I think this would be a useful facility.
agreed. in every other instance, streaming operators are part of the boost library that contains the streamed type. using namespace boost::container;
Hi! Are there some example code for the find_regex() or find_all_regex() string_algo function? Thanks! Claudio La Rosa
2009/12/1 <strasser@uni-bremen.de>:
is printing containers all it does? then it`s argueable just filling an omission of the STL container library, and could be part of Boost.Container, which is already on the review schedule.
Agreed... I think this functionality fits nicely inside of Boost.Container.
David Bergman wrote:
On Dec 1, 2009, at 12:47 PM, Vladimir Prus wrote:
Robert Ramey wrote:
I didn't have to write even one line to template code to implement the example.
I though boost.serialization was written by you?
The example is quite distinct from the library being used...
In other words, there is a special quality of Boost.Serialization that is of importance: it exists. So, we are allowed to use it, and by using it, we have to write very little code ourselves, beside a number of include statements. Great!
If this new proposed library is accepted, you get to write just as little code, if not less.
And that code still has to be parsed by the compiler.
This is a different issue. Is that your problem, that you think compilers have a problem with some complex constructs inside the Boost.Serialization library? Does the compiler fail? Consume too much resources? Too much time?
Yes, too much time and resources. When building debug variant, an program that does nothing but creates std::vector<int> is a 53K object file. When I add serialization, I get: - a useless warning shown below, and no, I am not willing to apply const_cast or declare things const just because Boost.Serialization has strange position about what const really means - a 387K object file When doing a release build, the source with serialization takes 1.3 second to compile, while the empty one takes 0.3 second. It does not seem like this overhead is justifiable for a trivial task of just getting my std::vector printed. - Volodya ../../../boost/mpl/print.hpp: In instantiation of ‘boost::mpl::print<boost::serialization::STATIC_WARNING_LINE<98> >’: ../../../boost/serialization/static_warning.hpp:92: instantiated from ‘boost::serialization::static_warning_test<false, 98>’ ../../../boost/archive/detail/check.hpp:98: instantiated from ‘void boost::archive::detail::check_object_tracking() [with T = std::vector<int, std::allocator<int> >]’ ../../../boost/archive/detail/oserializer.hpp:313: instantiated from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::invoke(Archive&, T&) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/detail/oserializer.hpp:525: instantiated from ‘void boost::archive::save(Archive&, T&) [with Archive = boost::archive::text_oarchive, T = std::vector<int, std::allocator<int> >]’ ../../../boost/archive/detail/common_oarchive.hpp:69: instantiated from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/basic_text_oarchive.hpp:80: instantiated from ‘void boost::archive::basic_text_oarchive<Archive>::save_override(T&, int) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/detail/interface_oarchive.hpp:64: instantiated from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ d.cpp:13: instantiated from here ../../../boost/mpl/print.hpp:55: warning: comparison between signed and unsigned integer expressions
On Dec 1, 2009, at 1:57 PM, Vladimir Prus wrote:
David Bergman wrote:
On Dec 1, 2009, at 12:47 PM, Vladimir Prus wrote:
Robert Ramey wrote:
I didn't have to write even one line to template code to implement the example.
I though boost.serialization was written by you?
The example is quite distinct from the library being used...
In other words, there is a special quality of Boost.Serialization that is of importance: it exists. So, we are allowed to use it, and by using it, we have to write very little code ourselves, beside a number of include statements. Great!
If this new proposed library is accepted, you get to write just as little code, if not less.
There are not that many lexemes left, especially if one does a "typedef boost::archive::text_oarchive arch;", i.e., it is hard to beat arch(std::cout) << v; or (by using my suggested wrapper function) output(v); Granted, you would have to define such typedef (one line) or wrapper (five lines) somewhere, but the main concern is the various locations of invocation, no?
And that code still has to be parsed by the compiler.
This is a different issue. Is that your problem, that you think compilers have a problem with some complex constructs inside the Boost.Serialization library? Does the compiler fail? Consume too much resources? Too much time?
Yes, too much time and resources.
How much time? I am not asking to be cute, but just curious as to how much extra building time is needed for Boost.Serialization. I have used it heavily in a lot of projects, and have not been disturbed by it. When you say resources, do you mean that compiler (or linker) use a lot of memory to handle Boost.Serialization?
When building debug variant, an program that does nothing but creates std::vector<int> is a 53K object file. When I add serialization, I get:
- a useless warning shown below, and no, I am not willing to apply const_cast or declare things const just because Boost.Serialization has strange position about what const really means
I did not follow you here; could you explain what that strange position is?
- a 387K object file
Let me check this. I am using -O2 on a Snow Leopard box with Apple's GCC 4.2.1, with the demo file. With *no* inclusion of Boost.Serialization headers nor reference to that library, I get a 10kB executable (basically a simple Hello World app with some vector manipulations). I append the source code at the end of this post for self-containment purposes :-) Let us introduce Boost.Serialization, and see what happens with a varying number of such invocation locations (log << vi) in our single demo file, compiling to an executable: no Boost.Serialization: 10kB N=0 (i.e., just including Boost.Serialization headers and linking with library): 27kB N=1: 51kB N=2: 55kB N=10: 55kB N=100: 55kB N=500: 65kB N=1000: 73kB N=4000: 118k Performing a mental linear regression, and using N=1 as a fixed constant, we get a linear function in extra (disk) space of 41kB + N * 20B Note that this extra space is relative code containing no Boost references at all. So, basically 20 bytes for each such invocation location. If you think that is strenuous, you could always wrap the invocation in a function (and force it to not be inlined), in which case you get a constant space addition of 41kB. NOTA BENE: more than a few hundred invocation locations in a project seems simply weird.
When doing a release build, the source with serialization takes 1.3 second to compile, while the empty one takes 0.3 second.
Here you have a better case, methinks. Let us do the similar tests as above, but now for compilation+linking time (linking time is much lower than compilation time...), where I measure the user + system time spent, which should be fairer for extrapolations to bigger builds (and my concurrent threads running on my box...): no Boost.Serialization: 0.45s N=0: 1.59s N=1: 1.59s N=2: 1.62s N=10: 1.62s N=100: 1.85s N=500: 2.68s N=1000: 4.02s N=4000: 13.52s Again fixating the constant at N=1, and performing some mental linear regression again, we get a linear function in extra build time of 1.14s + N * 3.4ms The biggie here is of course the constant, of an extra 1.14s, as you also reported. But note that it is relative *no* Boost use at all. When I wrap the output in the template function 'output' shown in the bottom of this post, and force that function not to be inlined (actually using a function pointer), I get N=1: 1.64s N=1000: 2.11s N=4000: 4.15s Using mental linear regression again, we get: 1.19s + N * 1.2ms NOTA BENE: I used no precompiled headers or such "cheating" which is the norm for bigger projects :-)
It does not seem like this overhead is justifiable for a trivial task of just getting my std::vector printed.
I agree, after measuring and being somewhat surprised by that large constant. But, again, it is relative no Boost use at all, and no precompiled headers or other compiler/linker tricks. In my regular code, the addition of Boost.Serialization does not add much compilation time. After all, it is just a few ms per invocation location.
- Volodya
../../../boost/mpl/print.hpp: In instantiation of ‘boost::mpl::print<boost::serialization::STATIC_WARNING_LINE<98> >’: ../../../boost/serialization/static_warning.hpp:92: instantiated from ‘boost::serialization::static_warning_test<false, 98>’ ../../../boost/archive/detail/check.hpp:98: instantiated from ‘void boost::archive::detail::check_object_tracking() [with T = std::vector<int, std::allocator<int> >]’ ../../../boost/archive/detail/oserializer.hpp:313: instantiated from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::invoke(Archive&, T&) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/detail/oserializer.hpp:525: instantiated from ‘void boost::archive::save(Archive&, T&) [with Archive = boost::archive::text_oarchive, T = std::vector<int, std::allocator<int> >]’ ../../../boost/archive/detail/common_oarchive.hpp:69: instantiated from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/basic_text_oarchive.hpp:80: instantiated from ‘void boost::archive::basic_text_oarchive<Archive>::save_override(T&, int) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/detail/interface_oarchive.hpp:64: instantiated from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ d.cpp:13: instantiated from here ../../../boost/mpl/print.hpp:55: warning: comparison between signed and unsigned integer expressions <d.cpp><d_empty.cpp>_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I have not seen this before. /David -------------- My test code: #include <iostream> #include <vector> #ifndef SKIP_SER #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp> #include <boost/serialization/list.hpp> #include <boost/serialization/string.hpp> //#include <boost/date_time.hpp> #include <boost/archive/text_oarchive.hpp> typedef boost::archive::text_oarchive arch; template<typename Cont> std::ostream& output(Cont const& cont, std::ostream & stream = std::cout) { boost::archive::text_oarchive(stream) << cont; return stream; } #endif int main(int argc, char * argv[]) { #ifndef SKIP_SER boost::archive::text_oarchive output_log(std::cout); #endif // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); #ifdef SKIP_SER std::cout << "Hello!"; #endif // A Ruby script of mine generates a number of invocations as this (I did not want to use Boost.PP for this) output_log << vi; }
Another thing that explore provides beyond the basic 'cout << c' is a set of extendable manipulators. These behave like existing std manipulators to modify the formatting of the output. Love them or hate them, they are familiar to a large base of developers. In particular, the cols and item_width manipulators together provide a simple but powerful way to stream numeric data. Can either serialization or karma handle this as easily? -- Jeff Faust #include <boost/explore/map.hpp> #include <boost/explore/vector.hpp> #include <iostream> #include <string> using namespace std; using namespace boost::explore; int main() { map<int, string> mis; cout << mis << endl; // [] mis[1] = "Hello"; cout << mis << endl; // [1:Hello] cout << quote_strings << mis << endl; // [1:"Hello"] vector<int> vi; for( int i = 1; i <= 9; ++i ) vi.push_back(i*2); cout << vi << endl; // [2, 4, 6, 8, 10, 12, 14, 16, 18] cout << delimiters("", " ", ""); cout << vi << endl; // 2 4 6 8 10 12 14 16 18 cout << cols(3) << vi << endl; // 2 4 6 // 8 10 12 // 14 16 18 cout << item_width(3) << vi << endl; // 2 4 6 // 8 10 12 // 14 16 18 }
On Tue, Dec 1, 2009 at 5:47 PM, Jeffrey Faust <jeff@opticalres.com> wrote:
Another thing that explore provides beyond the basic 'cout << c' is a set of extendable manipulators. These behave like existing std manipulators to modify the formatting of the output. Love them or hate them, they are familiar to a large base of developers.
In particular, the cols and item_width manipulators together provide a simple but powerful way to stream numeric data.
Can either serialization or karma handle this as easily?
Serialization cannot (as base, you could make a new archive format though, but that would be a bit of work). Karma can handle this easily, it is designed for it.
David Bergman wrote:
And that code still has to be parsed by the compiler.
This is a different issue. Is that your problem, that you think compilers have a problem with some complex constructs inside the Boost.Serialization library? Does the compiler fail? Consume too much resources? Too much time?
Yes, too much time and resources.
How much time? I am not asking to be cute, but just curious as to how much extra building time is needed for Boost.Serialization. I have used it heavily in a lot of projects, and have not been disturbed by it.
When you say resources, do you mean that compiler (or linker) use a lot of memory to handle Boost.Serialization?
When building debug variant, an program that does nothing but creates std::vector<int> is a 53K object file. When I add serialization, I get:
- a useless warning shown below, and no, I am not willing to apply const_cast or declare things const just because Boost.Serialization has strange position about what const really means
I did not follow you here; could you explain what that strange position is?
It's strange that you used Boost.Serialization in a lot of project without running into that. See http://www.boost.org/doc/libs/1_40_0/libs/serialization/doc/rationale.html#t... Note that 1.41 appears to have removed that bit of docs, and replaced assertion with the warning I have posted.
- a 387K object file
Let me check this. I am using -O2
That's not debug build, for sure.
on a Snow Leopard box with Apple's GCC 4.2.1, with the demo file. With *no* inclusion of Boost.Serialization headers nor reference to that library, I get a 10kB executable (basically a simple Hello World app with some vector manipulations). I append the source code at the end of this post for self-containment purposes :-)
Let us introduce Boost.Serialization, and see what happens with a varying number of such invocation locations (log << vi) in our single demo file, compiling to an executable:
no Boost.Serialization: 10kB N=0 (i.e., just including Boost.Serialization headers and linking with library): 27kB N=1: 51kB N=2: 55kB N=10: 55kB N=100: 55kB N=500: 65kB N=1000: 73kB N=4000: 118k
Performing a mental linear regression, and using N=1 as a fixed constant, we get a linear function in extra (disk) space of
41kB + N * 20B
Note that this extra space is relative code containing no Boost references at all.
So, basically 20 bytes for each such invocation location. If you think that is strenuous, you could always wrap the invocation in a function (and force it to not be inlined), in which case you get a constant space addition of 41kB.
First, I explicitly did not measure per-invocation overhead, as it does not seem to be of much importance for a specific application. I don't plan to have debug output all over. Second, it's not clear to me why you think that wrapping invocation in a function will entirely remove the 20B per call overhead. You need some instructions to call a function. Third, I explicitly measured debug build because it is the variant whose size is most affected by dependencies, and the size of debug builds obviously matters, as anybody who ever used the GDB debugger can attest. Fourth, I explicitly did not measure binary size. If we go that route, we'll have to factor in the size of libboost_serialization.so. And if we do, we'd have to count relocations that it contains and effect of those relocations on application startup time..
NOTA BENE: more than a few hundred invocation locations in a project seems simply weird.
When doing a release build, the source with serialization takes 1.3 second to compile, while the empty one takes 0.3 second.
Here you have a better case, methinks. Let us do the similar tests as above, but now for compilation+linking time (linking time is much lower than compilation time...), where I measure the user + system time spent, which should be fairer for extrapolations to bigger builds (and my concurrent threads running on my box...):
no Boost.Serialization: 0.45s N=0: 1.59s N=1: 1.59s N=2: 1.62s N=10: 1.62s N=100: 1.85s N=500: 2.68s N=1000: 4.02s N=4000: 13.52s
Again fixating the constant at N=1, and performing some mental linear regression again, we get a linear function in extra build time of
1.14s + N * 3.4ms
The biggie here is of course the constant, of an extra 1.14s, as you also reported. But note that it is relative *no* Boost use at all.
When I wrap the output in the template function 'output' shown in the bottom of this post, and force that function not to be inlined (actually using a function pointer), I get
N=1: 1.64s N=1000: 2.11s N=4000: 4.15s
Using mental linear regression again, we get:
1.19s + N * 1.2ms
NOTA BENE: I used no precompiled headers or such "cheating" which is the norm for bigger projects :-)
It does not seem like this overhead is justifiable for a trivial task of just getting my std::vector printed.
I agree, after measuring and being somewhat surprised by that large constant. But, again, it is relative no Boost use at all, and no precompiled headers or other compiler/linker tricks. In my regular code, the addition of Boost.Serialization does not add much compilation time. After all, it is just a few ms per invocation location.
I guess the above discussion has clearly demonstrated one thing. Namely, there's no set of established guidelines how to measure the cost of a dependency library -- especially a highly templated library. There are different option regarding: - What to measure: size or compilation time, or startup time, or debugger startup time - What build variant(s) to use - Should overhead for first use, or for each successive invocation be measured - What is the baseline? Empty file? File not using Boost? - If we have the numbers of everything above, what to make of them. Definitely, case studies regarding specific libraries might give interesting results.
../../../boost/mpl/print.hpp: In instantiation of ‘boost::mpl::print<boost::serialization::STATIC_WARNING_LINE<98> >’: ../../../boost/serialization/static_warning.hpp:92: instantiated from ‘boost::serialization::static_warning_test<false, 98>’ ../../../boost/archive/detail/check.hpp:98: instantiated from ‘void boost::archive::detail::check_object_tracking() [with T = std::vector<int, std::allocator<int>
]’ ../../../boost/archive/detail/oserializer.hpp:313: instantiated from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::invoke(Archive&, T&) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/detail/oserializer.hpp:525: instantiated from ‘void boost::archive::save(Archive&, T&) [with Archive = boost::archive::text_oarchive, T = std::vector<int, std::allocator<int> >]’ ../../../boost/archive/detail/common_oarchive.hpp:69: instantiated from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/basic_text_oarchive.hpp:80: instantiated from ‘void boost::archive::basic_text_oarchive<Archive>::save_override(T&, int) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ ../../../boost/archive/detail/interface_oarchive.hpp:64: instantiated from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = std::vector<int, std::allocator<int> >, Archive = boost::archive::text_oarchive]’ d.cpp:13: instantiated from here ../../../boost/mpl/print.hpp:55: warning: comparison between signed and unsigned integer expressions <d.cpp><d_empty.cpp>_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I have not seen this before.
Presumably, because in 1.40 this would have been a static assertion. - Volodya
Just a quick note regarding debug compilation: On Wed, Dec 2, 2009 at 8:06 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
- a 387K object file
Let me check this. I am using -O2
That's not debug build, for sure.
The -O2 setting only controls optimization, the -g flag is for enabling debug. You can have debug builds and turn on optimization at the same time, check for the presence of the -g flag (presence = debug) With kind regards, Mikael Olenfalk
Mikael Olenfalk wrote:
Just a quick note regarding debug compilation:
On Wed, Dec 2, 2009 at 8:06 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
- a 387K object file
Let me check this. I am using -O2
That's not debug build, for sure.
The -O2 setting only controls optimization, the -g flag is for enabling debug. You can have debug builds and turn on optimization at the same time, check for the presence of the -g flag (presence = debug)
I know fairly well what -O2 and -g do, and I stand by my earlier statement. -O2 makes debugging experience sufficiently painful that nobody will ever want to use that in debug builds, except in very unusual circumstances -- like tracking down a bug that only occurs at -O2 in the first place. - Volodya
Vladimir Prus wrote:
-O2 makes debugging experience sufficiently painful that nobody will ever want to use that in debug builds, except in very unusual circumstances
I routinely build with -O3 -g or -Os -g. Maybe I'm crazy to do so, but words like "nobody", "never" and "nowhere" are just asking for some weirdo like me to contradict them.... Phil.
Phil Endecott wrote:
I routinely build with -O3 -g or -Os -g. Maybe I'm crazy to do so, but words like "nobody", "never" and "nowhere" are just asking for some weirdo like me to contradict them....
Just throwing an actual use case of having -O2 and -g : when using Boost.Graph disabling optimizations makes the code too slow to be usable, even during a debug session. -- Maxime
Phil Endecott wrote:
Vladimir Prus wrote:
-O2 makes debugging experience sufficiently painful that nobody will ever want to use that in debug builds, except in very unusual circumstances
I routinely build with -O3 -g
And is that the set of options you actually use when you wish to debug something that is know to be buggy? Or are those the set of options that permit you to get somewhat usable backtrace "in the field"? - Volodya
On Dec 2, 2009, at 10:34 AM, Vladimir Prus wrote:
Phil Endecott wrote:
Vladimir Prus wrote:
-O2 makes debugging experience sufficiently painful that nobody will ever want to use that in debug builds, except in very unusual circumstances
I routinely build with -O3 -g
And is that the set of options you actually use when you wish to debug something that is know to be buggy? Or are those the set of options that permit you to get somewhat usable backtrace "in the field"?
Wow, I am often surprised by the tangential discussions that pop up here :-) Yes, it is sometimes proper to use both -g and -O2 (or -O3); I do it at times, but not as often as only with -g. BUT, this was not about "debug" vs "release" build. I know that you created a build without any optimization and with debug symbols (-g) when you got your big output (300k, right?) I just wanted to find out what the extra luggage of Boost.Serialization would be in a more relevant scenario, of a release build, i.e., I never stated that I created a "debug" build. And regarding the linear space function (of some 40kB + N * 20 B) becoming a constant (40kB) when using a non-inline function wrapping the serialization: yes, I am quite aware that even an innocuous function call does require a few bytes (especially after having implemented a lot of compilers and abstract machines myself), did I state otherwise? This was about the *extra* space (or time) needed for Boost.Serialization, and I did not regard those function invocations to have anything to do with Boost.Serialization per se, so those bytes did not go into that account. Obviously, not all of those 20B could be attributable to Boost.Serialization, but at least it presented a decent upper limit on the linear function. /David
On Wed, Dec 2, 2009 at 9:02 AM, David Bergman <David.Bergman@bergmangupta.com> wrote:
On Dec 2, 2009, at 10:34 AM, Vladimir Prus wrote:
Phil Endecott wrote:
Vladimir Prus wrote:
-O2 makes debugging experience sufficiently painful that nobody will ever want to use that in debug builds, except in very unusual circumstances
I routinely build with -O3 -g
And is that the set of options you actually use when you wish to debug something that is know to be buggy? Or are those the set of options that permit you to get somewhat usable backtrace "in the field"?
Wow, I am often surprised by the tangential discussions that pop up here :-)
Yes, it is sometimes proper to use both -g and -O2 (or -O3); I do it at times, but not as often as only with -g.
BUT, this was not about "debug" vs "release" build. I know that you created a build without any optimization and with debug symbols (-g) when you got your big output (300k, right?) I just wanted to find out what the extra luggage of Boost.Serialization would be in a more relevant scenario, of a release build, i.e., I never stated that I created a "debug" build.
I do the same for Visual Studio, I have debugging turned on in both Debug and Release, thankfully the debug data is stored in a separate file and not in the application file so when I distribute then I do not have to include that usually massive debug file. Is there not a way to make it an external information file in GCC as in VS?
2009/12/2 OvermindDL1 <overminddl1@gmail.com>:
On Wed, Dec 2, 2009 at 9:02 AM, David Bergman I do the same for Visual Studio, I have debugging turned on in both Debug and Release, thankfully the debug data is stored in a separate file and not in the application file so when I distribute then I do not have to include that usually massive debug file. Is there not a way to make it an external information file in GCC as in VS?
There is a way. From the manpage for "strip": ---- 1. Link the executable as normal. Assuming that is is called "foo" then... 2. Run "objcopy --only-keep-debug foo foo.dbg" to create a file containing the debugging info. 3. Run "objcopy --strip-debug foo" to create a stripped executable. 4. Run "objcopy --add-gnu-debuglink=foo.dbg foo" to add a link to the debugging info into the stripped executable. Note---the choice of ".dbg" as an extension for the debug info file is arbitrary. Also the "--only-keep-debug" step is optional. You could instead do this: 1. Link the executable as normal. 2. Copy "foo" to "foo.full" 3. Run "strip --strip-debug foo" 4. Run "objcopy --add-gnu-debuglink=foo.full foo" i.e., the file pointed to by the --add-gnu-debuglink can be the full executable. It does not have to be a file created by the --only-keep-debug switch.
Mikael Olenfalk wrote:
On Wed, Dec 2, 2009 at 8:06 AM, Vladimir Prus <vladimir@codesourcery.com> wrote:
Let me check this. I am using -O2
That's not debug build, for sure.
The -O2 setting only controls optimization, the -g flag is for enabling debug. You can have debug builds and turn on optimization at the same time, check for the presence of the -g flag (presence = debug)
-g (equivalent to -g2) inserts debugging symbols, useful for debugging, but also for other purposes. That can be done for optimized and non-optimized builds. -O2 activates optimizations. The notion of a debug or a release build is not intrinsic in GCC. The usual notion of a debug build implies no optimizations and the inclusion of debugging symbols. The usual notion of a release build implies optimizations and no debugging symbols. Combinations are possible (I usually include symbols in my release builds so core dumps and logged stack traces are more helpful to diagnose failures). The standard refers to the manifest constant, NDEBUG, to control the behavior of assert. This manifest constant is usually co-opted to indicate a release build and another, such as _DEBUG or DEBUG, is used to indicate a debug build. Based upon the presence of those manifest constants, debugging only code may be included. To compare the debug or optimized build effects of the proposed library and of Boost.Serialization requires agreeing on the options used, not just bandying about the terms "debug" and "release." (Never mind the need to agree on whether to examine the effects on debug or release builds.) _____ Rob Stewart robert.stewart@sig.com Software Engineer, Core Software using std::disclaimer; Susquehanna International Group, LLP http://www.sig.com IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
On Dec 1, 2009, at 12:43 PM, Robert Ramey wrote:
Vladimir Prus wrote:
Robert Ramey wrote:
Robert Ramey
It didn't require learning to use Boost.Serialization.
Here is what you're example looks like with boost serialization
#include <iostream> #include <boost/serialization/vector.hpp> #include <boost/serialization/map.hpp> #include <boost/serialization/list.hpp> #include <boost/serialization/string.hpp> //#include <boost/date_time.hpp> #include <boost/archive/text_oarchive.hpp> int main(int argc, char * argv[]){ boost::archive::text_oarchive output_log(std::cout); // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); output_log << vi;
While this is possible, with a custom archive class, I hope you'll agree that it will be rather heavyweight solution,
Hmmm - I realize that the serialization library has a lot of template code. But to me the issue is conceptual "heaviness".
The only line that might look heavy to anyone is boost::archive::text_oarchive output_log(std::cout); So, perhaps Vladimir should create an alias typedef boost::archive::text_oarchive arch; and then: arch(std::cout) << v; // v is a standard vector for a simple one-shot archive... I cannot see anything heavy at all, but that might be a disease related to having spent 23 years with C++ and having implemented CFront alternatives in the old days :-( Perhaps even using a simple wrapper function would make Vladimir less stressed? template<typename Cont> std::ostream& output(Cont const& cont, std::ostream & stream = std::cout) { boost::archive::text_oarchive(stream) << cont; return stream; } and then use output(v); or output(v, std::cerr);
In both libraries, the idea is to wrap extra information about types so that they can easily be displayed on a stream. In either library you just write
#include ... ...
output_stream << data
I just don't see the value in re-inventing of a whole new wheel.
Vladimir's claim is that his solution is lighter in two aspects (I think): 1. Easier to use 2. Not dependent on Boost I hope we all realize that, no, #1 is not the case, and if so, one can create a simple wrapper as the one above to make it easier. So, what about #2? Well, as a proposed Boost library, it is no longer that important.
By the time such a library would pass the boost review process, I'm sure it would be just as heavy weight (or more) than boost serialization. One would have a whole new interface to to something that is less capable.
Already the serialization library has an xml_archive as well as text archive.
while a simple output of output of std::vector can be implement in a 3-line template function.
I didn't have to write even one line to template code to implement the example.
This is a point of confusion for Vladimir, and I will comment on it in a more recent post. /David
Robert Ramey wrote:
Jeffrey Bosboom wrote:
Robert Ramey wrote:
Jeffrey Bosboom wrote:
I've written simple versions of streaming operators (both output and input) for the standard containers as a simple, low-barrier-to-entry form of serialization. Do you plan to provide the input operators too? (I understand this would involve some restrictions on the streamed forms of the container elements to prevent parsing ambiguity.) how would this be different than just writing a special archive class and using boost seriallization?
Robert Ramey It didn't require learning to use Boost.Serialization.
Here is what you're example looks like with boost serialization [snip] I don't see it as being any more difficult to use/learn than the library you propose.
It isn't really "more difficult", but in terms of the time required to write two template functions that contain a for loop each to serialize a std::vector<T> versus the time to read how to use Boost.Serialization, the template functions win.
Also, in the specific case I needed this for, I was running code on a cluster where Boost was not available, so it was lighter-weight (although obviously far less featureful).
I don't see how one is going to run boost libraries in an environment were boost is not available.
explore (or whatever it's called) wouldn't need to depend on any other part of Boost, so I can just copy the header files. This is opposed to Serialization, which depends on other parts of Boost (I think, not exactly sure) and needs to be built and linked against. In any case, if the input operators are not being provided, I wouldn't be able to use explore for serialization anyway. --Jeffrey Bosboom
Jeffrey Faust wrote:
Jared McIntyre, Danny Havenith, and myself, with support from Jeff Garland have developed a library to stream standard containers to output streams. We would like to submit this for review. The requirements for this library were gathered at the BoostCon 2007 Library in a Week sessions. An initial review was held and further requirements were gathered at BoostCon 2008.
I've been trying to think of when I might use such a library. Without the symmetry (input as well as output) I can think of few instances where this functionality might come in handy. Other than potential debug output, what else do you envision? What am I missing? I'm also finding little improvement over the following example based on your samples. Here I am using Karma to produce the same output as Explore. It is a bit more verbose; however, it is infinitely more flexible. Best regards - michael ------------------------------------------------------------------- #include <boost/config/warning_disable.hpp> #include <iostream> #include <vector> #include <map> #include <list> #include <boost/array.hpp> #include <boost/date_time/gregorian/gregorian.hpp> #include <boost/fusion/include/std_pair.hpp> #include <boost/spirit/include/karma.hpp> #include <boost/spirit/include/karma_format.hpp> #include <boost/spirit/include/karma_stream.hpp> using boost::spirit::stream; using boost::spirit::karma::format; int main() { // simple vector example std::vector<int> vi; vi.push_back(1); vi.push_back(2); vi.push_back(3); std::cout << format('[' << (stream % ", " ) << ']' , vi ); // prints [1, 2, 3] // lets do some containers in containers std::vector<std::vector<int> > vvi; vvi.push_back(vi); vvi.push_back(vi); std::cout << format('[' << *( '[' << (stream % ", " ) << ']' ) << ']' , vvi); // prints [[1, 2, 3][1, 2, 3]] // associative containers std::map<std::string, int> si_map; si_map["hello"] = 1; si_map["world"] = 2; std::cout << format('[' << ( (stream << ':' << stream) % ", " ) << ']' , si_map); // prints [hello:1, world:2] // containers of complex types using namespace boost::gregorian; std::list<date> dl; // date from boost::gregorian dl.push_back(date(2007, Jan, 1)); dl.push_back(date(2007, Jan, 3)); std::cout << format('[' << stream % ", " << ']' , dl); // prints [2007-Jan-1, 2007-Jan-3] // how about some boost container types: boost::array<std::string, 2> sa2; sa2[0] = "one"; sa2[1] = "two"; std::cout << format( '[' << stream % ", " << ']' , sa2); // prints [one, two] } -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
Michael Caisse wrote:
I've been trying to think of when I might use such a library. Without the symmetry (input as well as output) I can think of few instances where this functionality might come in handy. Other than potential debug output, what else do you envision? What am I missing?
Yes, I mostly think of this for debug output. But that does come up a lot. Similar to debug output would be testcase output. Something the customer still does not see, but is certainly useful. An example of something in production code might be printing a container out as an html table.
I'm also finding little improvement over the following example based on your samples. Here I am using Karma to produce the same output as Explore. It is a bit more verbose; however, it is infinitely more flexible.
In BoostCon 2007, we wanted something that worked like 'print(c)' where 'c' is any container. Other languages have this capability. We did consider using Karma during that conference and somebody presented this. It was decided to not be simple enough. The code samples you provided are compelling, but it's not as dead simple as 'print(c)' or the more C++ oriented 'cout << c'. -- Jeff Faust
Michael Caisse wrote:
I've been trying to think of when I might use such a library. Without the symmetry (input as well as output) I can think of few instances where this functionality might come in handy. Other than potential debug output, what else do you envision? What am I missing?
Yes, I mostly think of this for debug output. But that does come up a lot. Similar to debug output would be testcase output. Something the customer still does not see, but is certainly useful. An example of something in production code might be printing a container out as an html table.
What about: os << karma::format_delimited( "<ol>" << *verbatim["<li>" << stream << "</li>"] << "</ol>", // format description '\n', // delimiter c // data ) << std::endl; (ok, not a table, but a list, but you get the idea) where c could be any container type holding any data type. And with some preparation it is easily possible to create a special html framework allowing to write this as: html::ol[*html::li[stream]] or html::table[*html::row[stream]]
I'm also finding little improvement over the following example based on your samples. Here I am using Karma to produce the same output as Explore. It is a bit more verbose; however, it is infinitely more flexible.
In BoostCon 2007, we wanted something that worked like 'print(c)' where 'c' is any container. Other languages have this capability. We did consider using Karma during that conference and somebody presented this. It was decided to not be simple enough. The code samples you provided are compelling, but it's not as dead simple as 'print(c)' or the more C++ oriented 'cout << c'.
'cout << c' just doesn't work because of the need to use UB (extending namespace std). So you will end up writing cout << explore::format(c); which would have the Karma equivalent of (Boost V1.41): cout << karma::format(*stream, c); or even (post V1.41): cout << karma::format(auto_, c); or cout << karma::format(c); Regards Hartmut
Michael Caisse wrote:
I've been trying to think of when I might use such a library. Without the symmetry (input as well as output) I can think of few instances where this functionality might come in handy. Other than potential debug output, what else do you envision? What am I missing?
I'm also finding little improvement over the following example based on your samples. Here I am using Karma to produce the same output as Explore. It is a bit more verbose; however, it is infinitely more flexible. ... // lets do some containers in containers std::vector<std::vector<int> > vvi; vvi.push_back(vi); vvi.push_back(vi); std::cout << format('[' << *( '[' << (stream % ", " ) << ']' ) << ']' , vvi);
I surely would not like to write the above any time I need to dump std::vector for debugging. And I suspect that the effect on compilation time would also be somewhat different. In other words, I think there's definitely a place for an inflexible library that just works. - Volodya
Vladimir Prus wrote:
Michael Caisse wrote:
I've been trying to think of when I might use such a library. Without the symmetry (input as well as output) I can think of few instances where this functionality might come in handy. Other than potential debug output, what else do you envision? What am I missing?
I'm also finding little improvement over the following example based on your samples. Here I am using Karma to produce the same output as Explore. It is a bit more verbose; however, it is infinitely more flexible.
...
// lets do some containers in containers std::vector<std::vector<int> > vvi; vvi.push_back(vi); vvi.push_back(vi); std::cout << format('[' << *( '[' << (stream % ", " ) << ']' ) << ']' , vvi);
I surely would not like to write the above any time I need to dump std::vector for debugging. And I suspect that the effect on compilation time would also be somewhat different. In other words, I think there's definitely a place for an inflexible library that just works.
- Volodya
Volodya - Good point. I wouldn't. I was just trying to create the same format. I would instead write: std::cout << format( stream % ',', vvi); My experience with inflexible libraries is that they quickly fall apart with anything except trivial examples and the canned sample/example code. What happens when I have the following? struct my_struct{ int foo; int bar; }; I assume there needs to be an operator<<( std::ostream& s, my_struct& v ). Simply adapting my_struct to a fusion sequence will make it "just work" with Karma. Pre-Karma I used Phoenix to write: std::for_each( v.begin(), v.end(), std::cout << arg1 << ',' ); That isn't so bad either. Does making a library that allows me to write std::cout << v; for the few real-life (my real-life (o; ) cases where the contents of v can just output without massaging the values make sense? Not to me at least. Not when there are such powerful tools already in the boost toolbox that solve both the simple case and allow me to handle the complex situations. michael -- ---------------------------------- Michael Caisse Object Modeling Designs www.objectmodelingdesigns.com
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Jeffrey Faust Sent: Monday, November 30, 2009 3:59 PM To: boost@lists.boost.org Subject: [boost] [explore] Library Proposal: Container Streaming
Jared McIntyre, Danny Havenith, and myself, with support from Jeff Garland have developed a library to stream standard containers to output streams. We would like to submit this for review. The requirements for this library were gathered at the BoostCon 2007 Library in a Week sessions. An initial review was held and further requirements were gathered at BoostCon 2008.
Long overdue! We all needed it yesterday ;-) Paul PS Boost.Show ? PPS a simpler pairs and tuples 'show-ers' would be good too? Vectors of vectors? --- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
Paul A. Bristow wrote:
PPS a simpler pairs and tuples 'show-ers' would be good too? didn't Fusion already have operator<< for tuples like structure ?
Vectors of vectors? It seems it is handled already as the operator<< of vector will call << on its contents and then display vector itself.
Joel Falcou wrote:
Paul A. Bristow wrote:
PPS a simpler pairs and tuples 'show-ers' would be good too? didn't Fusion already have operator<< for tuples like structure ?
Vectors of vectors? It seems it is handled already as the operator<< of vector will call << on its contents and then display vector itself.
We handle std::pair and containers in containers. tuple already has streaming. See the boost file tuple/tuple_io.hpp. -- Jeff Faust
participants (27)
-
Claudio La Rosa -
David Bergman -
Duncan Smith -
Hartmut Kaiser -
Jeff Flinn -
Jeffrey Bosboom -
Jeffrey Faust -
Jeremy Pack -
Jeroen Habraken -
Joel Falcou -
Johan Nilsson -
John Bytheway -
Mathias Gaunard -
Maxime van Noppen -
Michael Caisse -
Mikael Olenfalk -
OvermindDL1 -
Patrick Horgan -
Paul A. Bristow -
Phil Endecott -
Robert Ramey -
Stefan Strasser -
Steven Watanabe -
Stewart, Robert -
strasser＠uni-bremen.de
-
Vladimir Prus -
Zachary Turner