
All - We are extending the IOStreams review for one week until 9/11/04. While we have had many good reviews, the library is quite large and we would like to see addtional discussion and review. I've included the library information for your reference. Thanks, Jeff ----- Today (August 28th, 2004) is the start of the formal review of the Iostreams library by Jonathan Turkanis. I will be serving as review manager. Note that this is a somewhat unusual situation in that we have several libraries that overlap in the same area, so comments related to the MoreIo overlap are needed. As usual, please state in review comments how you reviewed the library and whether the you think the library should be accepted into Boost. Further guidelines for writing reviews can be found on the website at: http://www.boost.org/more/formal_review_process.htm#Comments ********************************************** Library Synopsis ********************************************** The Iostreams Library serves two main purposes: * To allow the easy creation of standard C++ stream and stream buffer classes for new data sources and sinks. * To provide a convenient interface for defining i/o filters and attaching them to standard streams and stream buffers. The library focuses on freeing users from writing boiler plate code and allowing them instead to create highly reusable components. In addition to providing an abstract framework the library provides a number of concrete filters, sources and sinks which serve as example applications of the library but are also useful in their own right. These include components for accessing memory-mapped files, for file access via operating system file descriptors, for code conversion, for text filtering with regular expressions, for line-ending conversion and for compression and decompression in the zlib, gzip and bzip2 formats. The latest package can be found at the following locations: http://home.comcast.net/~jturkanis/iostreams/

"Bronek Kozicki" wrote:
Good point. It looks like File Descriptor Resource that deals with OS handles should, in principle, be able to deal with socket handles as well. However, socket handles are not available in current implementation, as far as I can see reading fdstream.cpp. There is a comment that non-blocking mode is curently not supported and you would imagine it would be even more important issue with socket handles. Tony

"Tony Juricic" <tonygeek@yahoo.com> wrote in message news:cheogr$v4s$1@sea.gmane.org...
Right. The current file_descriptor resources are only useful for files. The main reason is that they use only one buffer for both input and output. A secondary issue is that they assume the file descriptors are opened in blocking mode. See my reply to Bronek for information.
Tony
Jonathan

"Bronek Kozicki" <brok@rubikon.pl> wrote in message news:413AD72B.1010802@rubikon.pl...
Yes. I have done this. I wanted to included an example, but there would have been too much extra code to include. There will be better support for sockets if the library is extended to handle non-blocking, asynchronous and/or multiplexed i/o. For now, you can write a socket_resource with read and write members which are wrappers around api calls. You must be careful to meet the current specifications of read and write, according to which returning read fewer characters than requested signals EOF, and writing fewer than requested is an error. In my view, one of the major open questions about the library design is whether and how to allow a filter or resource to process fewer characters than requested without indicating and error or EOF. Please see 'Planned Changes'-->Item 4 (http://tinyurl.com/6rtkz) and 'Fututer Directions' (http://tinyurl.com/6r8p2). I'm very eager to hear how people think these issues should be handled. Best Regards, Jonathan

Jonathan Turkanis wrote:
maybe you should publish this example anyway?
Please see 'Planned Changes'-->Item 4 (http://tinyurl.com/6rtkz) and 'Fututer Directions' (http://tinyurl.com/6r8p2).
I WILL take a look B.

"Bronek Kozicki" <brok@rubikon.pl> wrote in message news:413B45F2.60303@rubikon.pl...
My example used a bare-bones windows and linux socket-wrapping library by Jasen Plietz. I wrote to hime to see if I could use it, but never heard from him. Looking in the sandbox and files sections instead, there were so many socket libraries that I was a bit overwhelmed. I didn't want to use Hugo Duncan's work, since his library appears quite large. I'm trying to avoid using the low-level windows and posix socket functions, since that is not my expertise.
Please see 'Planned Changes'-->Item 4 (http://tinyurl.com/6rtkz) and
'Fututer
Directions' (http://tinyurl.com/6r8p2).
I WILL take a look
Thanks. Jonathan

Dear boosters, Due to the fact that IOStreams review is extended until 11th of sept, I was wondering if it would be better for you to review the Output Formatters library starting as 12th of Sept. Otherwise, it'll go as planned, starting two days from now - on the 7th. Suggestions are welcome. Best, John
-- John Torjo Freelancer -- john@torjo.com Contributing editor, C/C++ Users Journal -- "Win32 GUI Generics" -- generics & GUI do mix, after all -- http://www.torjo.com/win32gui/ -- v1.3beta released - check out splitter/simple_viewer, a File Explorer/Viewer all in about 200 lines of code! Professional Logging Solution for FREE -- http://www.torjo.com/code/logging.zip (logging - C++) -- http://www.torjo.com/logview/ (viewing/filtering - Win32) -- http://www.torjo.com/logbreak/ (debugging - Win32)

"John Torjo" <john.lists@torjo.com> wrote in message news:413B124A.9060006@torjo.com...
Personally, I'd prefer that you delay the Output Formatters review. While there has been much thoughtful discussion about my library, I'm hoping that a few more reviewer will have time to take a close look at its implementation. Having two reviews at the same time will make this less likely. The fact that the More IO and IOStreams reviews overlapped was not such a big deal since the libraries themselves overlap and More IO is fairly small. Best Regards, Jonathan Turkanis

On Sun, 05 Sep 2004 15:19:06 +0200, John Torjo wrote
I think it needs to be delayed -- we don't usually run 2 reviews in parallel. The final decision lies with Tom, but I assumed he was going to push back the other reviews when I he approved the extension of IOStreams. Jeff

From: "Jeff Garland" <jeff@crystalclearsoftware.com>
I apologize for not getting my review in by 9/11, but I simply couldn't finish it by then. Even now, I cut things short. Nevertheless, here is my review. There are many comments below, but I'll provide the executive summary first. _________________________________________________________________ Should the library be accepted into Boost? Absent Daryle Walker's strong comments against this library, I'd have said, "Absolutely." Given those comments, I must be more guarded. The MoreIO library did less to simplify creating and managing streams and the cognitive complexity thereof, and it does not provide the filtering that the Iostreams library offers. Still, MoreIO offers some facilities and an approach that leads to questions regarding whether these two are competing or complementary libraries. While more investigation regarding the strengths and synergies of the two libraries seems warranted, it may be best to accept both and sort out the matter later. _______________________________ Design Evaluation The library is well conceived and executed. The design seems sound. There are still questions regarding the overlap and differences between this library and MoreIO. There are fundamental, philosophical differences between the libraries. Use cases should be conceived and compared to see how the libraries can handle each. Only then will we know whether they are complementary or competing and, if the latter is the case, which proves superior according to various criteria including ease of use and performance. It may well be that the stream[buf] parts of MoreIO should simply be added to, even integrated with, this library. _______________________________ Implementation Evaluation I did not have the time to study the implementation. _______________________________ Documentation Evaluation The documentation is astoundingly thorough and quite clear overall. Many have already made suggestions for improvement, but those can only be seen as making good better. I have included lengthy comments on the documentation, but I could not even visit every page! _______________________________ Potential Usefulness Evaluation This library makes creating streams and streambufs accessible to many more programmers than does the Standard IOStreams Library. This means that programmers will be able to create useful components they otherwise might avoid creating. This means the library should prove extremely useful. _______________________________ Miscellaneous I did not have the time to use the library. My review was based soley upon the documentation and some inspection of the code. I spent many hours reading the documentation (much to my wife's chagrin!). I have written manipulators and a few streambufs over the years, but I'm not truly expert with the inner world of streams and streambufs. _________________________________________________________________ Library Name I agree with the sentiment of other reviewers in that the library is misnamed. Since this library provides filtering, complete with chaining, as well as to make it easy to create streambufs and streams, the current name fails to capture all of these features. Since the Standard Library provides streambufs, streams, manipulators, etc., and yet it is called the IOStreams Library, perhaps there's nothing wrong with simply calling this library Boost.IOStreams. (I do think this library should use the same capitalization: "IOStreams" not "Iostreams.") _________________________________________________________________ Design Comments _______________________________ Miscellaneous Wouldn't the InoutResource concept be better named "BidirectionalResource," though that is longer? (Same for InoutFilter.) "Peekable" does not imply being able to put back a character. Wouldn't "Undoable" be a better name for this optional behavior? The normal template parameter abbreviation for a character type is "Char." I don't think abbreviating it to "Ch" is helpful. There's no information provided on the performance of this library. How much overhead does it add compared to custom solutions? For example, how much memory and runtime overhead is there wrt MoreIO's mechanisms? How much overhead is there wrt a custom filtering solution? These are answers users of the library need to judge whether it is sufficient or must they choose another approach. The filter chain ordering for input versus output is awkward. It should always be from the beginning to end; let the library reverse the order. _______________________________ boost::io::converter "converter" isn't well-named. It only widens characters for the resources it converts, so it should be called something else. So that begs the new name. I thought of "widener," but while it's correct English, it's also odd sounding, at least to my ear. "character_widener," would be a little more explicit. Perhaps "expander" or "character_expander" would work. Can converter be used with a wide character resource? Imagine a generic "widen" which ensures that the resulting streambuf or stream uses wide characters. convert's "Int" template parameter is poorly named. "Int" is, of course, a common abbreviation for "integer" in C++. I suggest using "Intern" or even "Internal." The name of "IntTr" should follow suit. _______________________________ basic_newline_filter I realize that this is only an example, but there is no apparent protection from misconfiguring the constructor flags. Grouping the related options into enumerated types and taking separate parameters for each group would make it safer to use. (You can overload the bitwise OR operator to permit combining them conveniently.) That is, since print_CR, print_LF, and print_CRLF are mutually exclusive, they should be part of an enumerated type that does not permit bitwise OR'ing. Since the posix, mac, and windows values are meant to supplant all of the other values, they should be part of their own enumerated type and should be used in a separate constructor. The remaining options can be part of a third enumerated type, with bitwise OR support, that constitutes the second argument to the existing constructor. The ignore_CR and ignore_LF values are poorly named. I suggest ignore_extra_CRs and ignore_extra_LFs. The print_CR, print_CRLF, and print_LF values would be better if named write_CR, write_CRLF, and write_LF, since no printing is occuring, at least in general. _________________________________________________________________ boost::io::reverse Isn't the direction of a filter independent of the filtering? That is, why isn't a filter's interface based upon these semantics: char_type filter(char_type ch); streamsize filter(char const * input, streamsize n, char const * output); Then, the source and destination of the data is under the control of the filter() function and means that whether the filtering is used on input or output is an orthogonal decision. This interface should be every bit as efficient as what you've specified, but makes filters more straightforward and flexible. _________________________________________________________________ Documentation Comments In case it makes a difference, I was reading the documentation at http://home.comcast.net/~jturkanis/iostreams/libs/io/doc/ on 4 SEP 2004 rather than downloaded documentation. _______________________________ General "boiler plate" is one word; I'm guessing it will appear in various places throughout the documentation, so I'm calling it out at the top. When formatting source code may I suggest not indenting twice for braces? Doing so pushes code too far right and reduces readability due to long lines. (This occurs on tutorial.html and read.html, for example.) When referring to concepts, always capitalize them. Thus, "filter" becomes "Filter" in most uses, for example. In many places you mention that something works with a Source or a "standard input stream or a standard stream buffer" or similar. Wouldn't it be better to clearly define once that std::basic_istream models Source, std::basic_ostream models Sink, std::basic_iostream and std::basic_streambuf model both? Then you need never mention "standard xxx stream or stream buffer" again. Otherwise, I'd prefer that "standard input stream" be spelled "std::basic_istream," that "standard stream buffer" be spelled "std::basic_streambuf," etc. _________________________________________________________________ menu.html _______________________________ Contents This is confusing. Apparently, the leading "+" means a different page, whereas entries without the "+" are on the current page, but there is no legend explaining that. Why do some pages, like tutorial.html have a mini-TOC at the top, whereas others, like home.html, which may be the only unique one, get their sections listed in the Contents frame? I'd prefer to see all of the pages handled the same. Put another way, remove "Conventions" and "Acknowledgments" from Contents; they can be found by looking at home.html. _________________________________________________________________ home.html _______________________________ Overview "In addition to providing an abstract framework the library ...in their own right. These include components...bzip2 formats." This is difficult to read due to lack of punctuation and other problems. I suggest rephrasing as: In addition to providing an abstract framework, the library provides a number of filters, sources, and sinks, which also serve as examples of how to use the library. These classes provide access to memory mapped files, file descriptor-based file access, code conversion, text filtering using regular expressions, line-ending conversions, and (de)compression for the zlib, gzip, bzip2 formats. Filtering Streams and Stream Buffers "For filtering data the Iostreams Library provides the templates ^ add comma filtering_streambuf and filtering_stream. Instances of filtering_streambuf or filtering_stream contains chains of ^^^^^^^^ subject/verb agreement: change to "contain" _______________________________ Conventions The second bullet would be easier to follow if you broke each piece into a separate bullet: * Specializations of std::basic_istream will be refered to as standard input streams * Specializations of std::basic_ostream as standard output streams etc. I don't think you should "omit the qualifier _standard_ for brevity." Otherwise, your "conventions" aren't. There's nothing wrong with what you've done, but why not use the phrase, "standard streambuf" instead of "standard stream buffer?" It's shorter and in keeping with what anyone with some familiarity of Standard IOStreams would expect. _______________________________ Footnotes I don't see any text that references the footnotes at the bottom of the page, so their presence is confusing. (Further reading reveals that these footnotes appear at the bottom of every page, regardless of whether they are referenced. May I suggest an endnotes page to which all references point. The notes won't be at the foot of the pages which reference them, but they won't appear on pages that don't. _________________________________________________________________ tutorial.html _______________________________ Defining Streams and Stream Buffers for a New Data Source In discussing the hypothetical read() function, you state that, "If an integer less than n is returned, the end of the stream has been reached." What about sources for which more characters will arrive but are not presently available? " The Iostreams Library allows users to create a standard input stream or stream buffer by defining a classes with a single ^^^^^^^ class "Here source is a convenience base class which provides several member types as well as default implementations of several member functions. (The member functions are not needed here.)" The parenthetical sentence is a little confusing. I suggest rewording like this: Here, source is a convenience base class which provides several member types as well as default implementations of several member functions which aren't used in this example. "We can also define a standard input stream:" I suggest italicizing "stream" and rephrasing like the following, for clarity: We can also define a standard input <I>stream</I> using random_source: " Both streambuf_facade and stream_facade have constructors and overloads of open which forward their arguments to filter or ^^^^^^^^^ to the filter resource constructor. Therefore, the first example could have ^^^^^ previous (That change will avoid making the reader count examples and wonder which code blocks are examples, and which are not. Alternatively, should this sort of reference appear elsewhere, you might consider numbering the examples in a caption so they're easy to spot and call out by number.) _______________________________ Defining Streams and Stream Buffers for a New Data Sink " The case is similar if we wish to define a standard stream buffer to write to a new type of output device." That's worded a little awkwardly. I suggest this: If we wish to define a standard stream buffer to write to a new type of output device, the required functionality is similar to that for creating a new data source. "Here sink is a convenience base class which provides several member types as well as default implementations of several member functions. (The member functions are not needed here.)" The parenthetical sentence is a little confusing. I suggest rewording like this: Here, sink is a convenience base class which provides several member types as well as default implementations of several member functions which aren't used in this example. "We can also define a standard output stream:" I suggest italicizing "stream" and rephrasing like the following, for clarity: We can also define a standard output <I>stream</I> using vector_sink: _______________________________ Filtering Input "...from a provided Source and modifies it before returning it to the user:" ^^^^ caller " Here input_filter is a convenience base class which provides several member types as well as default implementations of several member functions. (The member functions are not needed here.)" The parenthetical sentence is a little confusing. I suggest rewording like this: Here, input_filter is a convenience base class which provides several member types as well as default implementations of several member functions which aren't used in this example. "The function boost::io::get reads a character from an arbitrary Source; it is provided by the Iostreams Library to allow all Sources to be accessed with a single interface." That's worded awkwardly. I suggest rephrasing like this: The function boost::io::get reads a character from an arbitrary Source. This is possible because the Iostreams Library provides the same interface for all Sources. "The last item added to the chain could be any Source." This is the first time you've used the term, "chain," so this is confusing as written. I suggest the following: Notice that push is overloaded to accept both filters and sources (an alphabetic_input_filter and std::cin, in this case). When you push multiple items (one or more filters and possibly a source), you form a "chain." A source is only valid as the last item in a chain. In the following example, reproduced here, the order seems backward: filtering_streambuf<input> in; in.push(alphabetic_input_filter()); in.push(random_source()); If that's the correct order, then you need to explain, even here in the tutorial, why the order is as shown. "We could also add any number of InputFilters to the chain before adding the Source." Since "we" aren't doing this, it would read better if written like this: Any number of InputFilters can be added to the chain before adding the Sink. "They could also use the following convenience typedefs:" I'm not sure what the antecedent to "they" was supposed to be, but I suggest rephrasing that like this: The following typedefs can make using the foregoing types easier: _______________________________ Filtering Output I suggest that your example convert text to lower case as this will eliminate language concerns about being able to map a single lower case character to multiple upper case characters. " Here output_filter is a convenience base class which provides several member types as well as default implementations of several member functions. (The member functions are not needed here.)" The parenthetical sentence is a little confusing. I suggest rewording like this: Here, output_filter is a convenience base class which provides several member types as well as default implementations of several member functions which aren't used in this example. "The function boost::io::put writes a character to an arbitrary Sink; it is provided by the Iostreams Library to allow all Sinks to be accessed with a single interface." That would read better like this: The function boost::io::put writes a character to an arbitrary Sink. All Iostreams Library Sinks use the same interface. "We could also add any number of OutputFilters to the chain before adding the Sink." Since "we" aren't doing this, it would read better if written like this: Any number of OutputFilters can be added to the chain before adding the Sink. "They could also use the following convenience typedefs:" I'm not sure what the antecedent to "they" was supposed to be, but I suggest rephrasing that like this: The following typedefs can make using the foregoing types easier: _______________________________ Efficient Filtering: Buffered Filters "In such cases, each time a character is read from an ordinary InputFilter one incurs the overhead of a call to its member function get." You've changed from "we" to "one" and it sounds odd. Try this: In such cases, reading a character from an ordinary InputFilter means calling its get member function, thus incurring the overhead of a (virtual?) function call. "To avoid this overhead, the library allows users to define Buffered filters which can process several characters at once by implementing a member function read. This function will typically be called only when the buffer associated with the filter by the Iostreams Library becomes full." The reference to "users" instead of "us" sounds odd, and the second sentence is phrased awkwardly. To avoid this overhead, we can create a Buffered filtered which can process several characters at once by implementing a read member function in lieu of the get member function. The Iostreams Library typically calls read only when the buffer associated with the filter is full. That means that the larger the buffer is, the fewer times read must be called, reducing its overhead. (It isn't clear from the tutorial whether read supplants get or augments it. I assumed the former in writing the above.) "OutputFilters can also be Buffered. For example:" You should mention that this is via the write mf: The Iostreams Library also supports Buffered OutputFilters via the member function write. Here's the Buffered version of the toupper_filter: _________________________________________________________________ examples.html _______________________________ Tab-expanding OutputFilter "This example uses an OutputFilter to replace each tab character in a code excerpt with an appropritate number of spaces." ^^^^^^^^^^^^^^ ^^^^^^^^^^^^ some text appropriate (The "code excerpt" to "text" change reads better since there is no "code excerpt" apparent yet.) _______________________________ Line-wrapping OutputFilter "This example uses an OutputFilter to wrap text so that each line is no longer than a specified maximum value. No attention is paid to word boundaries." A little rewording would make this read better: This example...specified maximum value, ignoring word boundaries. _______________________________ Presidential OutputFilter The political humor of this example is unwarranted in Boost documentation. Perhaps you could rename it to "Texas OutputFilter" or "Southern U.S. OutputFilter." There is no description of what the filter actually does in this section. _______________________________ Uncommenting InputFilter "This example uses an InputFilter to remove shell-style..." ^^^^^ Unix shell _______________________________ Regex OutputFilter "This example uses a regex_filter to remove C and C++-style comments from a code excerpt. Although somewhat more ^^^^^^^^^^^^^^^^^^^ omit sophisticated than the Uncommenting InputFilter, above, the ^^^ filter in this example is not adequate for use in production..." ^^^^^^^^^^^^^^^^^^^^^^ this filter _________________________________________________________________ concepts.html Throughout this page, all references to "resource," "filter," etc. should capitalize the words in keeping with their being concepts. _______________________________ Overview "...an InputFilter, which filters input read from a Source, and an OutputFilter, which filters output written to a Sink." I suggest a little rephrasing to make this read better and fill in a couple of gaps: an InputFilter, which filters input read from a Source or another InputFilter, and an OutputFilter, which filters output read from a Source or another OutputFilter, before it is written to a Sink. "InputFilters, OutputFilter and their refinements are called..." ^^^^^^^^^^^^ OutputFilters " In general, a filter or resource may provide access to an input sequence, for reading, an output sequence, for writing, or both." ^ ^ omit both commas "The full sets of requirements for the filter and resource ^^^^^^^ ^^^^^^ ^^^^^^^^ set Filter Resource concepts are summarized in the definitions of the concepts ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Resource and Filter which form the roots of the concept..." ^^^^^^^^^^^^^^^^^^^^^^^^^ omit _______________________________ Resource Concepts For Source, Sink, InoutResource, and SeekableResource, change "Provides" to "Resource providing." "SeekableResource" refers to a read/write "head," but few, if any, Resources for which this library will be used are physical devices with read/write heads. I suggest using "location," "position," or "offset" in lieu of "head" here and in other parts of the documentation. In "SeekableResource," you need a comma between "single" and "repositional." _______________________________ Filter Concepts InputFilter, OutputFilter, InoutFilter, and SeekableFilter refer to a "stream buffer." Shouldn't they refer to Sources and Filters instead? In "SeekableResource," you need a comma between "single" and "repositional." _______________________________ Optional Behavior "Closable: A filter or resource which receives notifications immediately before a stream is closed" "Notifications" is unexpected and you refer to a stream. Try this: Closable: A Filter or Resource which is told to close immediately before a Source or Sink is closed. "Localizable: A filter or resource which receives notifications when the locale of a stream or stream buffer is set using basic_ios::imbue or basic_streambuf::pubimbue." "Notifications" is unexpected and you refer to a stream (buffer). Try this: Localizable: a Filter or Resource which is informed when the locale of a Resource is set using.... In "Direct," you refer to a socket-like interface, but not everyone will understand what you mean. I suggest using the phrase, "function-based interface." _______________________________ Convenience Templates and typedefs ______________ Overview "They ease the task of defining filters and resources...." ^^^^ "simplify" would read better "The template resource and its specializations...new resource types; the template filter and its specializations...new filter types; the template buffered_filter and its specializations...new buffered filter types." Split that into three separate sentences to reduce complexity. "There are also wide-character versions of these templates, resource, wfilter..." ^^^^^^^^ namely wresource _________________________________________________________________ filter.html _______________________________ Description "The class template filter, its subcass wfilter...Library to ^^^^^^^ subclass ease the definitions of new models of the various filter..." ^^^^^^^^^^^^^^^^^^^^^^^ simplify creating "...Library; they are inteded to be used as base..." ^^^^^^^ intended _________________________________________________________________ buffered_filter.html _______________________________ Description "The class template buffered_filter, its subcass ^^^^^^^ subclass buffered_wfilter...Library to ease the definitions of new..." ^^^^^^^^^^^^^^^^^^^^^^^ simplify creating "...Library; they are inteded to be used as base..." ^^^^^^^ intended _________________________________________________________________ modes.html _______________________________ Overview Merge the second paragraph with the first. It's distracting being separated. "Readers new to the Iostreams Library should feel free to ^^^^^^^ ^^^^^^^^^^^^ Users omit concentrate primarly on these two modes." _______________________________ Definitions of the Modes "Whether the reading or writing heads are repositionable, and if so, whether there are separate heads for reading and writing or a single read/write head." Rewriting without the use of "head" and correcting the position of a comma: Whether the reading or writing locations are repositionaable and, if so, whether the positions for reading and writing are separate. "* Whether a filter or resource is Closable." This should be a single line. Table: - change "Involves a single sequence" to "A single sequence" throughout. - change "Involves a two separate" to "Two separate" _______________________________ Mode Hierchy Diagrams ^^^^^^^ Hierarchy The diagram does not translate well to grayscale printing; there's no apparent difference between the blue and green. Using arrows, like with UML class diagrams, would make the lines more meaningful. _______________________________ Mode Tags "As with standard library iterator category tags, the tag corresponding to a mode is convertible to each of the tags corresponding to modes which the first mode refines." ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ of the modes it refines. "This is useful to help reduce...different of filter types." ^^ omit You should provide or link to examples that illustrate using some of the more common modes. _______________________________ The Metafunction mode You need to include an example and the signature of mode so we know how to use it. _________________________________________________________________ policy_based_streams.html _______________________________ Overview "Instances of the policy class can be associated and ^^^ with and disassociated with an instance of streambuf_facade using..." ^^^^ ^^^^^^^^^^^^^^^^ from a streambuf_facade specialization "The class template stream_facade is a policy-based stream template which derives from one of std::basic_istream, std::basic_ostream and std::basic_iostream depending on the mode of the policy class. As with streambuf_facade, instances of the policy class can by associated and disassociated with an instance of stream_facade using its member functions open and close." I suggest a few changes to that paragraph to make it more readable and to correct a few errors: As a convenience for those needing a stream, the library also provides the class template stream_facade, which is a policy-based stream class template. It derives from std::basic_istream, std::basic_ostream, or std::basic_iostream, depending upon the mode of the policy class. As with streambuf_facade, instances of the policy class can be associated with and dissociated from an instance of stream_facade using.... _______________________________ Reference ______________ Class template streambuf_facade ______ Description "Policy-based stream buffer template with an interface...." ^^^^^^^^ class template ______ Synopsis Using "T" as the name of a policy class template parameter is confusing. "T" is, of course, commonly used for a data type stored in a container and for similar purposes. The closest thing to "T" in streambuf_facade is the "Ch" nested type in the policy class. Change "T" to a more meaningful name. ______ Template Parameters "T - A model of the filter or resource concepts. Specializations of streambuf_facade with filter types are used internally by the Iostreams Library to construct chains of filters and resources. Users of the library never need to specialize streambuf_facade with a filter type." Since only the library uses filter types for "T", why not name "T" "Resource" and let the library worry about using filters for the "Resource" parameter? Also, given that only the library does this, omit the last two sentences or move them to an Advanced Topics section. "Tr - A C++ standard library charatcer traits type ([ISO], ^^^^^^^^^ character 21.1.1) with...the character type Ch of T." ^^^^^^^ T::Ch "Alloc - A C++ standard library allocator...where Ch is the ^^^ character type of T." ^^^^^^^^^^^^^^^^^^^ T::Ch, the character type in T. ______ streambuf_facade::streambuf_facade You have two sections with the same heading; both should fall under the heading "Constructors." Otherwise, "streambuf_facade::streambuf_facade" should be written as "streambuf_facade<T,Tr,Alloc,Mode>::streambuf_facade." ______ The first streambuf_facade::streambuf_facade "Before the instance can be used for i/o, one of its..." ^^^^^^^^ streambuf ______ The second streambuf_facade::streambuf_facade "Constructs a streambuf_facade which is ready to perform i/o, where the parameters have the following interpretations:" ^^^^^^^^^^^^^^^ meaning You don't call out function parameters like this many other places. There's nothing wrong with it, but you should choose to explain them in a paragraph or a list and do that throughout the documentation. "buffer_size - The size of any buffers that need to be allocated" In the description of open(), you mention that a size of zero means that the streambuf is unbuffered. Doesn't that apply here, too? "pback_size - The size of the putback buffer, relevant only..." ^^^^^^^ buffer, if any. This is "The T constructors involved must take all arguments by..." ^^^^^^^^ so invoked ______ streambuf_facade::open ^^ <T,Tr,Alloc,Mode>:: "Assocaites the given instance of T with this instance of ^^^^^^^^^^ Associates streambuf_facade, if there is no such instance currently ^^^^ T associated; otherwise, throws std::ios_base::failure. The second ^^^^^^^^^^ associated with this parameter determines the size of any buffers that need to be allocated; a value of zero indicates that i/o should be unbuffered. The third parameter determines the size of the putback buffer; it is relevant only if Mode is a refinement of ^^^^^^^^^^ buffer, if any. It "Each of these members associates with this instance of streambuf_facade a newly constructed instance of the policy type ^^^^^^^^^^^^^^^^^ new T constructed from the given lists of arguments, if there is no such instance currently associated; otherwise, they throw ^^^^^^^^^^^^^^^^^^^^^ associated with this. Otherwise std::ios_base::failure. The T constructors involved must..." ^^^^^^^^ so invoked ______________ Class template stream_facade ______ Examples Where are the streambuf examples? You said they were the "fundamental components" of the library. ___ Defining a simple ofstream ^^^^^^ Simple _________________________________________________________________ filtering_streams.html _______________________________ Synopsis There are two, identical declarations of boost::io::filtering_streambuf. _________________________________________________________________ adapters.html _______________________________ Overview The example references should be by name ("Reading From an STL Sequence" rather than "Example I.") to make the links more meaningful. "2. Filtering stream and stream buffer constructors, as well as their member functions push, are overloaded...." ^^^^^^^^^^^^^^^^^^^^^ push member functions _______________________________ Examples Do any of these techniques apply to filtering_streambufs? ______________ I. Reading from an STL Sequence ^^^^^^^^^^^^^^^ Reading From ______________ II. Overwriting an STL Sequence ^^^^^^^^^^^^^^^ Overwriting ______________ III. Appending to an STL Sequence ^^^^^^^^^^^^^^^ Appending To "A stream which appends characters to a sequence, say std::vector, can be defined as follows:" ^^^^^^^^^^^ a std::vector _________________________________________________________________ code_conversion.html _______________________________ Overview "converter: A resource adapter which which takes a resource with ^^^^^^^^^^^ which a narrow character type and produces a resource with wide ^^^^^^^^^ the corresponding wide character type by introducing a layer of code conversion performed using a std::codecvt." ^^^^^^^^^^^^^^^ using "* converting_streambuf: A wide-character stream buffer template having an interface essentially identical to Don't break the paragraph here. filtering_streambuf, with the following additional property: after zero or more wide-character filters have been added to the chain, a sequence of zero or more narrow-character filters and a narrow character resource may be added. A converter is inserted automatically between the final wide-character filter and the first narrow-character filter or resource." Rephrasing that will simplify it and make it clear: ...essentially identical to filtering_streambuf, which automatically inserts a converter between the last wide character filter, if any, and the first narrow character filter or resource. (Any wide character filters must precede the narrow character filters and resource in the chain.) A similar change to the converting_stream paragraph would be appropriate. _________________________________________________________________ convenience.html _______________________________ Overview You mention a convenience function named "copy" but the only function shown is operator <<. _________________________________________________________________ converter.html _______________________________ Description "The class templates converter is...and produces a resource with ^^^^^^^^^ template wide character type" ^^^^ a wide "If a narrow character resource performs input and output using two distinct sequences (see Modes)," ^^^^^^^^^^^ omit; you link to inout directly later in the same sentence. "Otherwise, attempting to spcialize converter results...." ^^^^^^^^^ specialize _______________________________ Reference ______________ Template parameters ^^^^^^^^^^ Parameters ______________ converter::converter Calling a constructor a member, however accurate, is odd. Why not just call it a constructor? "The third parameter determines the size of the buffers or ^^^^^^^ buffer buffers used for code conversion." "The effect of invoking imbue while code conversion...." ^^^^^ Change to use the code typeface ______________ converter::open "Assocaites the given instance of Resource with this instance of converter, if there is no such instance currently associated; otherwise, throws std::ios_base::failure." "Associates" is misspelled, but rephrasing it would work well: Makes a copy of the Resource to be used by the converter. If the converter already has a Resource, it throws std::ios_base::failure. I don't think "associates" leaves out an important detail, suggesting that the Resource is tracked by pointer and that the Resource's lifetime must extend to the next call to close(). "The third parameter determines the size of the buffer or buffers used for code conversion." ^^^ for the ______________ converter::close "Disassociates from this instance of converter any instance of the resource type Resource currently associated with it, calling cleanup functions as appropriate and destroying the associated instance of Resource." This is awkwardly phrased. I suggest: Destroys the converter's current Resource, if any, calling the cleanup functions of any filters and the Resource, as appropriate. I don't think "dissociate" (not "disassociates," BTW) is necessary. _________________________________________________________________ exceptions.html _______________________________ The Boost Iostreams Library ______________ Policy The text is formatted differently than on other pages for this heading level. "...implements the protected virtual memeber functions..." ^^^^^^^ member ______________ Rationale The text is formatted differently than on other pages for this heading level. _______________________________ Exception Safety "1. Resources are freed by calling destructors or close, if appropriate." Don't break the line between "or" and "close." "3.b. ...If the current resource is removed from the chain and a new one added..." ^^^^^^^^^ one is added "Conditions 2. and 3. rely on the specification of the filter and resource concepts. Note that there is no guarantee that a stream's or stream buffer's character sequences are in a consistent state, i.e., that data has not been corrupted, and no way in general to determine the state of these sequences after an exception." After reading this paragraph, I'm left to wonder whether removing and replacing the resource, as described in 3.b, means that the state of the character buffers is well-defined or still unknown as described here. IOW, this paragraph seems to contradict 3.b and should be rephrased to clarify how the state of the buffers can be known and corrected. _______________________________ Acknowledgments "Thanks to Angelika Langer and John Torjo for discussion of exceptions." What does "for discussion of exceptions" mean? Did you have conversations with them on the subject? Then you need to write, "for our discussions on exceptions and exception handling." _______________________________ Footnotes "1...but that "if the exception is rasied due to an error..." ^^^^^^ raised _________________________________________________________________ regex_filter.html _______________________________ Description "The class template basic_regex_filter filters data using regular expressions the Boost Regular Expression Library." ^^^ via the "As unfiltered data is scanned for matches, portions of data ^^^^^^^^^^^ the portions of the which fall between regular expression matches are forwarded unchanged." _______________________________ Reference ______________ basic_regex_filter::formatter "The type of object which a basic_regex_filter uses to...." ^^^ formatter is the "Since Boost.Function objects are implictly constructible from function objects with the correct signature, users of regular expression filters may define their own function objects with the correct signature and pass them to the basic_regex_filter constructor..." This is incomplete. One can write free functions, lambda functions, etc. that satisfy Boost.Function to create formatters. ______________ basic_regex_filter::basic_regex_filter Here's another example of the (apparently) less common bulleted list of parameter descriptions. "replace - A function which will be passed each match_results object in succession and whose return values will be be used as replacement text" Rewording this will make it clearer: A Boost.Function object to which each regular expression match is passed. The return value is used as the filter's result in lieu of the matched text. _______________________________ Example "See Examples." ^^^^^^^^ Regex OutputFilter _________________________________________________________________ one_step_filter.html _______________________________ Description "Its derived classes model DualUseFilter so that it may be used ^^ they? for either input or output." _________________________________________________________________ newline_filter.html _______________________________ Description "The class templates basic_newline_filter is a DualUseFilter ^^^^^^^^^ template which converts between the text file formats used by...." ^^^^^^^ among "Its sole constructor takes an integral flag parameter used to specify the source and target formats." I don't see the value of this here. In the description of the constructor further down the page, this argument is discussed fully. "Note: It is not known if specializations of basic_newline_filter other than basic_newline_filter<char> are useful. If not, the template parameter will be eliminated." Make this a footnote referenced from "Ch" in the Template Parameters section. _______________________________ basic_newline_filter::basic_newline_filter You misspelled "interpreted" in the table several times. _________________________________________________________________ compression.html _______________________________ Acknowlegments "The compression/decompression filters were influences by the..." ^^^^^^^^^^ influenced "Thanks also to Jean-loup Gailly and Mark Adler and to Julian ^^^^^^^^^^^^^^^^^^^^^^ , Mark Adler and Seward for making their excellent librarys available..." ^^^^^^^^ libraries _______________________________ Installation "Alternatively, the binaries can be build from the source code." ^^^^^ built _________________________________________________________________ file_based_resources.html _______________________________ Overview "...access to memory-mapped files on Windows an POSIX." ^^ and _______________________________ Acknowledgments "The memory-mapped file resources are based on work of Craig..." ^^ on the _________________________________________________________________ file.html _______________________________ Reference ______________ Class template basic_file_source ^^^^^^^^ Template ______ basic_file_source::basic_file_source "Constructs...std::basic_filebuf buf opened as follows:" ^^^ but ______________ Class template basic_file_sink ^^^^^^^^ Template ______ basic_file_sink::basic_file_sink "Constructs...std::basic_filebuf buf opened as follows:" ^^^ but ______________ Class template basic_file_resource ^^^^^^^^ Template ______ basic_file_resource_::basic_file_resource ^^^^^^^^^ resource "Constructs...std::basic_filebuf buf opened as follows:" ^^^ but _________________________________________________________________ text_processing.html _______________________________ Overview "The...provides two types of filters for test processing:" ^^^^ text _________________________________________________________________ rationale.html _______________________________ Design Decisions ______________ Generic Design "2. The library can be easily extended to handle..." ^^^^^^^^^^^^^^^^^^^^^^ can be extended easily (avoids split infinitive) "reduce code bloat by internally wrapping filters and resources in polymorphic classes with virtual function read, write, ^^^^^^^^ functions etc. without a noticable performance impact." ^^^^^^^^^ noticeable ______________ Chain Interface "If there is a need for it, it can easily be restored." ^^^^^^^^^^^^^^^^^^^^^^ can be restored easily (avoids split infinitive) ______________ Return Type of write "...cannot be written, was adopter for these reasons:" ^^^^^^^ adopted _______________________________ Planned Changes "2. The concept Buffered will be renamed MultiCharacter, since it is more descriptive and Buffered would be better used elsewhere." I don't understand this point. "MultiCharacter" does not convey anything useful. Even an unbuffered filter processes multiple characters. "Buffered" seems to convey the right idea. More detail is needed about where you would prefer to use "Buffered" and more consideration of what to call what is now called "Buffered" is needed. "3. ...The concept of a smart resources, which already..." ^^^^^^^^^^^^^^^^^ a smart resource -OR- smart resources "4. ...This is necessary to accommodate i/o models just as..." ^^^^ such? "1. Adopt the convention that reads and writes always block until at least one character is written." ^^^^^^^ transferred _________________________________________________________________ portability.html _______________________________ Overview "...together with a coresponding Jamfile." ^^^^^^^^^^^^ corresponding _________________________________________________________________ installation.html _______________________________ Overview "4. ...and depend on the non-boost library" ^^^^^ Boost " 5. ...and depend on the non-boost library" ^^^^^ Boost _________________________________________________________________ adapt.html _______________________________ Overview "The overloaded function template adapt takes an OutputIterator, a pair of ForwardIterators or a standard i/o stream or stream buffer and returns a resource." The series is confusing as written; I suggest: The overloaded function template adapt takes an OutputIterator, a pair of ForwardIterators, a std::basic_iostream, or a std::basic_streambuf, and returns a resource. _______________________________ Reference ______________ adapt - Iterator Ranges ______ Template Parameters "Source - A model of ForwardIterator." ^^^^^^ FwdIt ______ Return Value "Returns a Resource...delimited by first and last." ^^^^^^^^^^^^^^^^^^^^^^^^^^^ It is probably well understood by all readers, but it might be better to express this more formally. _________________________________________________________________ back_inserter.html Broken link to <http://home.comcast.net/~jturkanis/iostreams/libs/io/doc/functions/classes/back_inserter.html#synopsis> _________________________________________________________________ close.html _______________________________ Description "This gives filters and resources an opportunity to free resources or reset their states in preparation for a fresh..." ^^^^^^ state _______________________________ When is close invoked? It is premature to discuss the invocations of close() without showing its synopsis. You should put the synopsis just after the Description section. ______________ streambuf_facade and stream_facade This section is a bit confusing. First you say that r.close() calls boost::io::close(r, std::ios_base::in) followed by boost::io::close(r, std::ios_base::out). Then, you say that if "r" controls a single sequence, those calls wind up, in effect at least, calling r.close(). ______________ filtering_streambuf and filtering_stream You used the term "complete" here. I recall it being defined elsewhere, but it would be useful if you had a single point of definition to which you could link whenever you mention it. The sequence of close() calls never mentions buf1. Is there a mistake? If not, some clarification is in order. "...each element receives two closure notifications..." ^^^^^^^^^^^^^^^^^^^^^ calls to close _______________________________ Reference ______________ Synopsis You've used "T" to mean a Resource in one case and a Filter in the other. This is confusing. You should use consistent parameter names among all of the library templates. _________________________________________________________________ copy.html _______________________________ Description "The function template copy reads data from a given model of Source - or from a standard input stream or stream buffer - and writes it to a given model of Sink - or from a standard output stream or stream buffer - until the end of stream is reached." This sentence is confusing due to the number of hypenated phrases. If I've got it right, it should be written like this: The function template copy reads data from a model of Source, which might be a std::basic_istream or std::basic_streambuf, and writes the data to a model of Sink, which might be a std::ostream or std::streambuf, until the end of the input stream. "The size of the temporary buffer used may be supplied..." ^^^^^^^^^^^^^^^^ temporary buffer between the input and output streams _________________________________________________________________ get.html _______________________________ Reference ______________ Description On many pages, "Description" is the topmost section. Here, it's a subsection of "Reference." Why? This section duplicates the information provided in the Overview, so one of them is superfluous. (This applies to many pages.) "Attemps to extract a character from a given instance..." ^^^^^^^ Attempts ______________ Template Parameters "Source - A...or a standard input stream or stream buffer type." ^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^ Why not just write "std::basic_istream" and "std::basic_streambuf?" (This applies to many pages.) ______________ Return Type If the return type is known to be "typename std::char_traits<typename char_type<Source>::type>::int_type" why not just show that as the return type in the synopsis? _________________________________________________________________ imbue.html _______________________________ Description "Alothugh imbue is designed to be overloaded..." ^^^^^^^^ Although _________________________________________________________________ read.html _______________________________ Overview "The first overload.... The second overload...." No overloads have yet been shown, so this is premature. Move these sentences to Reference|Semantics. _______________________________ Example An example is premature when you haven't shown the synopsis or described the parameters. IOW, this section should follow the Reference section. _______________________________ Reference ______________ Description On many pages, "Description" is the topmost section. Here, it's a subsection of "Reference." Why? This section duplicates the information provided in the Overview, so one of them is superfluous. ______________ Synopsis The second overload extends past the screen, depending upon the text size. ______________ Template Parameters "T - For the first overload, a model of Source or a standard input stream or stream buffer type. For the second overload, a model of InputFilter." The fact that you must describe the parameter this way is a clear indication that you need to use better parameter names. How about: template<typename Resource> std::streamsize read(Resource& t, ...); template<typename InputFilter, typename Source> std::streamsize read(InputFilter& t, Source& src, ...); Since the definition of "Source" refers to T's character type, you'll have to make reference to Resource's or InputFilter's character type. With these distinctions in place, the subsequent Semantics sections won't need to replicate the declarations. _________________________________________________________________ reverse.html _______________________________ Description "The function template reverse takes a filter and returns a new filter which performs the same filtering operation as the original filter but which is an OutputFilter if the orginal filter was an InputFilter and and InputFilter if the orginal filter was an OutputFilter. Roughly speaking, it reverses the direction of a given filter." The ideas here are conveyed in the wrong order and why, "roughly speaking?" The function template reverse reverses the filtering direction of a given filter. It turns an InputFilter into an OutputFilter and an OutputFilter into an InputFilter. You mention memory overhead that can be a side effect of using reverse(). Are there performance issues as well? Also, shouldn't that paragraph be set off with "Note:" or "Warning:"? You mention that using reverse may not be suitable for some data streams, but you need to give more guidance regarding why one would want to use reverse(). _________________________________________________________________ seek.html _______________________________ Overview "The function template seek is used by the Iostreams Library to perform random access withing a sequence controlled by a..." ^^^^^^^ within "seek is designed...specialized for resource types external..." ^^^^^^^^ Resource _______________________________ Reference ______________ Template Parameters "...A model of one of the resource concepts which..." ^^^^^^^^ Resource ______________ Function Parameters "way - ..." You refer to std::ios_base::beg twice. The latter should be std::ios_base::end. Also, "way" would be better named "dir" or "direction," don't you think? _________________________________________________________________ write.html The same comments apply to this page as to read.html, above. _________________________________________________________________ io_traits.html _______________________________ Overview "The header <boost/io/categories.hpp> contains category tags for classifying models of the various filter and resource concepts." ^^^^^^ ^^^^^^^^ Filter Resource "The header <boost/io/io_traits.hpp> contains the definitions of the metafunctions char_type and category, used to associate two fundamental types with each model of one the filter or resource concepts:" I'm not quite sure how the end of that sentence should be phrased, but it's confusing as is. Perhaps this conveys the right information clearly: ...used to determine the type of characters processed by, and the category, or capability description, of Filters and Resources. ______________ Character Type This section seems out of place. It should be titled, "char_type," but you haven't introduced that template yet. I suggest that this descriptive text be moved to the Class Template char_type|Description section. ______________ Category This section, too, is out of place. I suggest moving the content to Class Template category|Description. _______________________________ Class Template char_type ______________ Description "Metafunction associating a character type to each filter or resource type." char_type is a metafunction associating a character type with a Filter or Resource type. The result is the character type the Filter or Resource reads and/or writes. (I merged your original with the earlier Character Type section text and rephrased the result.) ______________ Template parameters ^^^^^^^^^^ Parameters "T - A model of one of the filter or resource concepts, or a standard stream or stream buffer" As in other places, capitalize "filter" and "resource" when referring to them as concepts. _______________________________ io_traits::type Where did this come from? I don't understand how io_traits::type is related to char_type (without looking at the code, of course). _______________________________ Class Template category ______________ Description "Metafunction associating a category tag to each filter or resource type." category is a metafunction that associates a category tag with each Filter or Resource type. The category indicates whether the type is a Filter or Resource, what the type's mode is, and what optional behavior the type provides. The category is similar to the iterator_category member of std::iterator_traits. Types which represent categories are called category tags. (I merged your original with the earlier Category section text, and rephrased the result.) _______________________________ Category Tags "...simply define a struct extending the existing tags." ^^^^^^^^^ (multiply) derived from The table doesn't which tags are mutually exclusive. Is that exclusivity enforced? For example, a type cannot be both a Filter and a Resource, so only one of filter_tag and resource_tag may be used to create a legitimate category. Is that enforced? Is a category tag validated in any way? That is, what if a developer specifies neither filter_tag nor resource_tag. Is the category tag still valid, will it produce a meaningful error, or will it wreak havoc? _________________________________________________________________ mode.html _______________________________ Description "Metafunction returning a mode tag corresponding the the mode of a model of one of the filter or resource concepts or of a standard stream or stream buffer." mode is a metafunction that determines the mode type that indicates the mode of the Filter or Resource. The mode can indicate whether the type is usable for input, output, or both, and whether and how the type supports seeking. _________________________________________________________________ filtering_stream.html _______________________________ Description "Standard stream template used for filtering data." filtering_stream is a standard stream template used for filtering data. _______________________________ Reference In the function sections, you don't separate descriptions of template parameters from those of function parameters. While it does make things consume more space, I think "Template Parameters" and "Function Parameters" headings would be helpful. ______________ Template parameters ^^^^^^^^^^ Parameters "Tr - The traits type" You should mention that std::char_traits<Ch> is the default for this parameter (it's in the synopsis, of course, but it bears repeating). You should also document which features of the traits class you need so folks that want to write their own can. (You might capture this information in one place to reference from multiple places.) ______________ filtering_stream::filtering_stream This heading is repeated several times because there are several constructors. It would be better to preface them all with one heading, "Constructors." ______________ filtering_stream::filtering_stream (#2) "t - An instance of T" This description is not helpful. I can see what it is from the synopsis. Perhaps: The initial Filter or Resource in the filtering_stream. "The parameters have the following interpretations:" ^^^^^^^^^^^^^^^ meaning ______________ filtering_stream::filtering_stream (#3) There's no parameter list introductory statement like, "The parameters have the following meaning." The parameter list doesn't describe "t" as did the second. ______________ filtering_stream::filtering_stream (#4) The description leaves me wanting more. IIUC, this creates a complete filtering_stream with no Filters that uses an iterator based Resource. For input, that would mean that only the characters in the range [first, last) are read. For output, that would mean that characters are written through *first++ until first == last. If I've gotten that right, you need to say something like that so no one else must deduce the same information. Also, this filtering_stream would be complete, right? There's no parameter list introductory statement like, "The parameters have the following meaning." In fact, you've been inconsistent in this on many pages. However, adding a "Function Parameters" heading should be sufficient in and of itself. ______________ filtering_stream::push (#1) "T - A CopyConstructible model..." You should reference a footnote that contains the information that follows regarding using boost::ref, etc. (That is, move that information to a footnote and then reference it from here.) You'll need to do this elsewhere, too. "t - An instance of T" Can you describe this better? ______________ filtering_stream::push (#1) Parameter "t" is not in the parameters description list. ______________ filtering_stream::push (#2) The signature of this function is identical to the previous push overload. Thus, you should merge the two in the synopsis and when documenting them. ______________ filtering_stream::push (#3) "Iter - An iterator type...Ch and whose traveral..." ^^^^^^^^ traversal ______________ filtering_stream::pop "If the chain is initially complete, causes..." ^^^^^^^^^^^^^^^^^^^^^ was complete prior to calling pop ______________ filtering_stream::emtpy "Returns true if the underlying chain is empty." ^^^^^^^^ contains no Filters or Resources. ______________ filtering_stream::size "Returns the number of stream buffers in the underlying chain." ^^^^^^^^^^^^^^ Should that be "Filters and the Resource, if any?" ______________ filtering_stream::reset "Clears the underlying chain. If the chain is initially complete, causes" If I understand correctly, that should be worded more like this: If the chain was complete before calling reset, it closes and releases all Filters in the underlying chain. In either case, it closes and releases the Resource, if any. _________________________________________________________________ filtering_streambuf.html Many of the filtering_stream.html comments apply. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;

"Rob Stewart" <stewart@sig.com> wrote in message news:200409131343.i8DDhOa15411@lawrencewelk.systems.susq.com...
Thanks for your very detailed comments! I really appreciate the enormous amount of work you have obviously done examining the library interface and documentation. Responding to your comments is also an enormous amount of work ;-) Here is my reply to the points you make in about the first half of your post. I'll try to reply to the rest tomorrow.
There is no overlap between the manipulators from More IO and the present library. I voted to accept all the manipulators (partly as a result of your comments) so I don't see any problems there. streambuf_wrapping doesn't really overlap with the present library either. I voted to reject because I thought it didn't simplify writing streams enough to justify including it. If it is accepted, however, I don't see any real conflict. So the real question is: if one or more of null_buf, pointerbuf array_buf and value_buf (I may be spelling these wrong) is accepted, would it be advantageous to reimplement it using the present library. There are two issues: 1. Performance Reimplementing pointerbuf and array_buf using the present library should produce exactly the same runtime performce, since resources which present their character sequences as in-memory arrays are specially optimized (see direct_streambuf.fppp at http://tinyurl.com/3qrsx). Reimplementing null_buf and value_buf should improve runtime performance, since the implementations from More IO are not buffered. The reimplemented versions of null_buf and value_buf will have somewhat larger generated code size. Perhaps this would also be true for pointerbuf and array_buf. Using the reimplemented versions will also cause more source code to be included during compilation. This is because Daryle's implementations are self contained while and mine depend on parts of type traits and mpl as well as a certain amount of internal infrastructure. Daryle clearly feels this is an important issue; I think the amount of code included is in keeping with other boost libraries. 2. Interface. Streams and stream buffers generated using the present library have a simple open/is_open/close interface similar to std::fstream. I think including the streams and stream buffers from More IO as is, rather than giving them a compatible interface, will be confusing to users.
Could you elaborate?
See the above discussion on performance.
I prefer integration.
Thanks!
Many have already made suggestions for improvement, but those can only be seen as making good better.
I'm glad to hear you say that. There were so many comments on the inadequacey of the docs that I was starting to think they were simply awful.
I agree, but couldn't think of anything better.
Steve Teal's "C++ IOStreams Handbook" and Langer and Kreft's "Standard C++ IOStreams and Locales" use the spelling 'IOStreams'. The C++ standard, however.uses 'Iostreams'. (See 27.1 [lib.iostreams.requirements]). I think 'IOStreams' looks a bit funny, but I'm happy to use it if that's what people like.
Bidirectional was the name I first gave it (http://tinyurl.com/4gskg). I changed it because the filters and resources in question manage two separate sequences, whereas bidirectional iterators represent a single sequence. I chose 'inout' because such components act like an input component glued to an output component. I don't like the name much, and would happily revert to bidirectional, I people don't feel that it's misleading.
"Peekable" does not imply being able to put back a character.
I know. The natural choice would have been Putbackable, which sounds terrible. I choice Peekable because if you can put back a character than you can peek ahead in the stream by reading a character and then putting it back.
Wouldn't "Undoable" be a better name for this optional behavior?
It sounds a bit to general. For instance, an Undoable Sink might allow characters already written to be cancelled.
The normal template parameter abbreviation for a character type is "Char." I don't think abbreviating it to "Ch" is helpful.
The standard uses charT; so do Langer and Kreft. The Dinkumware docs use Elem. I don't recall seeing Char, but I'm not against it.
Performance comparable to hand-written components is listed as one of the main design goals. (Rationale-->Design Goals-->Item 3, at http://tinyurl.com/3z9ou). The footnote to that item (http://tinyurl.com/4th3v) mentions some performace comparisons I performed. To summarize, I found that for output, stream_facade<file_descriptor_sink> was slightly faster than the Dinkumware std::ofstream (but it does no synchonization.) Since the overhead of filesystem access might be expected to dominate in that comparison, I also tested a custom ostringstream against the Dinkumware component, and found identical performance. It would have been natural to test input and random access as well. I didn't do this because I wasn't convinced that the tests I was running (from <libs/io/test/detail/verification.hpp>) had any relation to typical use cases. (BTW, I expect random-access performance won't be so good right now, since I haven't optimized streambuf_facade::seekoff for short, frequent seeks. This can be done if necessary.) I didn't test filtering performance because at the time I didn't have anything to compare it with except for James Kanze's implementation, which uses pre-standard streams. Now that I have JC van Winkel's filtering implementation and Robert Ramey's Dataflow Iterators, I can run some head-to-head tests. I have spent a lot of effort on optimization, and expect my implementation to compare favorably.
For example, how much memory and runtime overhead is there wrt MoreIO's mechanisms?
There should be slightly more memory overhead with the reimplementation of null_buf and value_buf using my library, since it provides buffering by default.
See above.
I'm surprised nobody mentioned this earlier. Having the resource at the end dramatically simplifies the interface. It allows an instance filtering_stream of filtering streambuf to be considered 'open' as soon as a resource is pushed, and 'closed' as soon as it is popped. This allows a simple stack-like interface. The alternative would be (i) to add open/close functions (which are part of the interface of streambuf_facade, but would currently be redundant for filtering streams, OR (ii) to have a first-time switch which automatically 'opens' the filtering stream as soon as the first i/o is performed. In addition, allowing filters to be pushed after a resource would give many new users the impression that they can add filters *after* i/o is in progress. As has been discussed during the review, this is not currently supported; support can be added in limited circumstances, but not generally. Consider: filtering_ostream out; out.push(file_sink("log")); out.push(base_64_encoder()); out << "hello world!\n"; // stream is implicity 'open' out.push(zlib_compressor()); // error!
How about code_converter?
It only widens characters for the resources it converts, so it should be called something else.
converter is basically an easy to use wrapper for codecvt, which is the standard library's component for code conversion. So 'code_coverter' (or converter, for short) seems appropriate. George Garner suggested that reverse code conversion be supported too, and I am inclined to agree, though it is much less useful.
This is a bad choice, since 'widening' is performed by a ctype, not a codecvt.
Perhaps "expander" or "character_expander" would work.
Same as widener -- to me it implies a character-by-character transformation.
In theory, yes. If you supply a locale containing a codecvt<wchar_t, wchar_t>, then you could use converter<wfile_source> to "widen" a wide-character stream. However, you can't depend on an implementation providing specializations codecvt<wchar_t, wchar_t> or even codecvt<char, wchar_t>. I'm inclined to parameterize converter by a codevt type, as suggested by George Garner: template< typename Resource, typename Codecvt = fetch_from_global_locale > If Codecvt::extern_type does not match the character type of Resource, but Codecvt::inter_type does, the codecvt will be automatically 'reversed'.
True. I think I was trying to fit within the 80 character line limit, without too many line breaks.
I was trying to keep things simple. Specifying more than one of print_CR, print_LF and print_CRLF yeilds a runtime error. Even if print_xxx had it's own enumeration type, it would still be possible to specify illegal values.
The ignore_CR and ignore_LF values are poorly named. I suggest ignore_extra_CRs and ignore_extra_LFs.
Sounds reasonable.
How about output_xxx instead of print_xxx?
Yes, in theory, put in practice some filters are much easier to express as an output filter than an input filter, or vice versa.
This is what symmetric filter does (http://tinyurl.com/3lz82). I'm planning to optimize the case where a symmetric filter is added to a filter chain, so that symmetric filters should be ruthlessly efficient. The problem with making symmetric filters the default (or exclsuive) filter concept is that it's unexpectedly difficult to implement them correctly.
_________________________________________________________________ Documentation Comments
Since I agree with most of your suggestions, I'll only comment on those where I disagree or where further explanation is needed.
That's fine.
Thanks for pointing that out. (I'd guess it went through the typical transformation: boiler plate --> boiler-plate --> boilerplate.)
Do you mean: struct tolower_filter : public buffered_input_filter { template<typename Source> streamsize read(Source& src, char* s, streamsize n) { streamsize result = boost::io::read(src, s, n); for (streamsize z = 0; z < result; ++z) s[z] = tolower(s[z]); return result; } }; Should be struct tolower_filter : public buffered_input_filter { template<typename Source> streamsize read(Source& src, char* s, streamsize n) { streamsize result = boost::io::read(src, s, n); for (streamsize z = 0; z < result; ++z) s[z] = tolower(s[z]); return result; } }; ? Sounds like an improvement.
When referring to concepts, always capitalize them. Thus, "filter" becomes "Filter" in most uses, for example.
I realize I don't follow a consistent convention, but I've tried to distinguish beetwen concepts, models of concepts, and instances of models of concepts. Obviously I haven't done a very good job, but I'm not sure using the capitalized version everywhere is the right solution. (Maybe it is)
This is just because I didn't update the docs well enough after the most recent rewrite. FWIW, 1. Originally, standard streams and stream buffers were not models of Resource. 2. Then I went out of my way -- by making the concepts 'external' in Thorsten Ottosen's sense -- to make streams and stream buffers full-fledged models of Resource. 3. Then, for exception safety reasons, I decided that Resources must be CopyConstructible, so that streams and stream buffers would simply be 'treated as' Resources. 4. Finally, I dropped the CopyConstructible requirement but neglected to update the docs everywhere.
'+' means that the entry is expandible by clicking. '-' means the entry is collapsable. See http://tinyurl.com/65234.
When I added the tree control, I forgot to get rid of all the local TOC's.
'Stream buffer' is used by the standard and by Langer and Kreft, presumably because streambuf is the specialization of basic_streambuf for the type char.
I forgot to remove them when I moved Tutorial to its own page.
(Further reading reveals that these footnotes appear at the bottom of every page,
I hope not!
regardless of whether they are referenced.
_________________________________________________________________ tutorial.html
I see a grammatical problem here -- "forward their arguments to filter or resource constructor" -- but I don't understand what changes you are suggesting.
I see your point, but it might be better just to use your first sentence. The second sentence provides more information than needed here.
"The last item added to the chain could be any Source."
I mentioned the rationale, but obviously I need to point out the ordering issues in the tutorial.
"We could also add any number of InputFilters to the chain before adding the Source."
"They could also use the following convenience typedefs:"
I think the first sentence was changed without updating the second. There are several choices for this type of passage: 1. Use the passive voice everywhere. 2. Use 'we' -- this sounds natural to me because it's used in mathematical papers. 3. Use 'you' 4. Use 'the user' Which should one prefer? Settling on one of the above and using it consistently should resolve many of your (snipped) suggestions/objections below.
_______________________________ Efficient Filtering: Buffered Filters
You've changed from "we" to "one" and it sounds odd. Try this:
Agreed.
It's not virtual.
(It isn't clear from the tutorial whether read supplants get or augments it. I assumed the former in writing the above.)
You assumed correctly.
_________________________________________________________________ examples.html
I'm not sure I agree. But noted.
Perhaps you could rename it to "Texas OutputFilter" or "Southern U.S. OutputFilter."
I don't think this would help. Texans don't typically say 'subliminable,' for instance.
There is no description of what the filter actually does in this section.
I'm thinking each example should have its own page, with a detailed description of what the filter does, a guided tour of the source code, and an explanation of the example output.
What's the problem here?
But that would be false. In order to chain filters, the library has to wrap filters with adapters so that they become resources.
How about 'stream position indicator'?
Right. Until the latest revision, filters had to communicate with stream buffers.
How about: Closable: A Filter or Resource with a member function close which is called when the end of a character sequence is reached.
I see your point, but Direct Resources also have a function based interface.
_________________________________________________________________ modes.html
How would you phrase the whole sentence?
_______________________________ Definitions of the Modes
Seems to be a mozilla bug. Even doesn't help.
I don't follow.
This section contains a link to the reference docs for mode: http://tinyurl.com/5oqsy. Wouldn't it suffice to provide an example there?
_______________________________ Reference
"Policy-based class template deriving from a specialization of basic_streambuf with an interface...." ?
I adopted this convention from "C++ Templates", p. 10. FilterOrResource would be more decriptive, but it's too long. What would you suggest?
Okay, except that I've been toying with exposing some of the undocumented stuff. Thanks again -- see you tomorrow ;-) Jonathan

Johnathan,
Good. To amplify on why this is needed, let me note that there are non-traditional applications that should be considered for an IOStreams library. String search algorithms (e.g. Aho-Corasik) are commonly used to match genetic patterns and molecular sequences. Perhaps someone will want to use your streams to convert between DNA and RNA "code pages." Parameterizing the converter makes that possible. Regards, George.

From: "Jonathan Turkanis" <technews@kangaroologic.com>
Agreed.
I see a conflict because of the dramatically different approaches to "wrapping" a streambuf. If both remain, then each needs more information and rationale so users understand which to choose for a given use case.
Great.
Reimplementing null_buf and value_buf should improve runtime performance, since the implementations from More IO are not buffered.
The question, then, is whether MoreIO should add buffering to remove that difference.
With the addition of buffering, this nebulous difference may be moot. Real numbers will make it possible to judge the value of both approaches, of course.
If yours were orders of magnitude longer to compile, this might be an issue, but I doubt that's the case.
This is one of the reasons why I'm concerned about keeping both. I like the simplicity of your approach, especially since the code size and performance is nearly the same. Nevertheless, without hard numbers on these differences, it's hard to judge objectively. There's also the issue of whether MoreIO would be of more benefit to those doing embedded C++, for example.
Daryle thinks the Standard IOStreams I/F is acceptable as is and worked within it. You hide the complexities of that I/F behind a minimal, but sufficient -- at least until async is dealt with fully -- I/F. His approach is documented in the standard and many C++ books.
I'd seen it spelled "IOStreams" so many places that I was sure it was that way in the standard. Go figure.
Generally, "bidirectional" doesn't convey any information regarding a common or separate buffer, so I think it works fine.
Many applications have only one level of undo and don't allow everything to be undone. Consequently, I don't think this is much of a problem. How about "revertable?"
You're not using the "T" suffix, so "Char" is the same as "charT" in my estimation.
The information, as I recall, wasn't rigorous, or at least not reported that way. Also, I think it was limited to a few compilers and one or two streams.
Yes, I recall that now. It does suggest that your library does about as well as can be done. This should be discussed and highlighted better, I think. Also, a wider range of tests on many compilers would lend greater credibility to the conclusions.
That sounds good.
The only fair comparison is, I think with both buffered. Conversely, what about a version from your library that isn't buffered?
I understand.
(i) to add open/close functions (which are part of the interface of streambuf_facade, but would currently be redundant for filtering streams, OR
I don't think this would be a problem, but I don't see -- and I'm not going to go look right now, sorry -- why it is redundant for filtering streams. Also, couldn't the function be called "complete" or "add_resource?"
(ii) to have a first-time switch which automatically 'opens' the filtering stream as soon as the first i/o is performed.
This will add memory and runtime overhead it would be best to avoid.
This won't be a problem with complete() or add_resource().
That seems pretty reasonable, but I'm not very lucid right now. ;-)
Reasonable.
That's pretty reasonable, I think.
If no operators are defined for the enumerated type, then one must go out of one's way to provide illegal values.
If you prefer "output" to "write" that's fine.
I don't recall reading information on this rationale. It should be part of the docs, maybe a FAQ. However, I'd like more information on why filters are difficult to write bidirectionally. The I/F I mention above seems quite simple. (Note, I"m not suggesting that filters support simultaneous bidirectionality by default. For that, I can understand a special category or mode tag.)
Yes.
OK. I never hovered over them, so I never noticed that they were active.
OK. I guess I usually just abbreviate it.
No, you're right. They were at the bottom of a number of pages, but I never came back to this comment to note that. (I generalized too soon.)
"...forward their arguments to the filter or resource constructor. Therefore, the previous example could...."
I use we in comments (design notes, and such), and you in tutorial type text. After all, I'm teach "you," the reader in such text. I think you should write the docs as if you are writing to a single reader. That makes it easiest to get things consistent.
Hence my making it parenthetical.
It sounds as though the filter is only good for a hard coded string or something. Omitting "from a code excerpt" makes it sound more general.
Suffice it to say that I wrote it that way because I thought that was accurate. Therefore, it wasn't sufficiently clear to prevent my jumping to the wrong conclusion.
Why "indicator?"
Good.
Well, you need something more specific and plain than "socket-like interface."
"Users new to the IOStreams Library should concentrate primarily on these two modes."
Change to, "This is useful to help reduce...different filter types."
I did see mode.html later, but I don't think I noticed the link when I was looking at this page. An example on that other page is the right place, but maybe you can call attention to the link?
Good.
You need a supercategory of Filter and Resource. "Component?" -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;

"Rob Stewart" <stewart@sig.com> wrote in message:
From: "Jonathan Turkanis" <technews@kangaroologic.com>:
"Rob Stewart" <stewart@sig.com> wrote in message:
My library doesn't allow users to wrap a streambuf. stream_facade wraps a streambuf internally, but that's an implementation detail.
This should be the explanation: "If you have an existing streambuf implementation and you can't or don't want to reimplement it as a Resource, use streambuf_wrapping.hpp; otherwise, you should probably reimplement is as a Resource and and use stream_facade." produce
There are some memory-use optimizations which I didn't apply, to avoid further obfuscating the code. For instance, output-only streambuf_facades have a pback_size_ member, and streambuf_facade's representing Resources have a next_ pointer which is always null. This can be eliminated with metaprogramming.
I provide a framework to make it easier to define new stream buffers. Daryle provides implementations of a few specific stream buffers. Since I don't claim that all stream buffers should be constructed using my framework, and since users of Daryle's stream buffers will generally never need to know how they are implemented, I don't see any conflict. people
Good -- if there are no objections, I'll switch.
This still sounds too general. Maybe 'PutbackResource'?
Neither the standard nor Langer and Kreft use the convention that template parameters should be lowercase identifiers with 'T' appended. So I don't see why "Char" is the same as "charT". But Char is slightly more explicit, so I think I may switch. main performace
It was fairly rigorous, but not reported that way. I didn't finish the project and publich the results because - It was taking a long time :-) - I couldn't claim that my tests represented typical use cases - Initial results satisfied me that at most some fine-tuning would be required; no major overhaul was needed. the
Once I get some tests which I believe represent typical use cases, I can publish results for a wide range of platforms.
Depending on whether a streambuf_facade represents a single sequence or two separate sequences, streambuf_facade has one or two members representing buffers. Since buffering has to be turned off at runtime, there's no way to eliminate these members for unbuffered streambuf_facades. (This will change if I add a buffering policy as a template parameter.) If the buffers are never allocated, the overhead of each is unused member variables of type char* and a std::streamsize.
It's redundant because adding a resource is currently the equivalent of 'open', and popping a resource is the equivalent of 'close'.
If you mean that the above should be rewritten filtering_ostream out; out.push(file_sink("log")); out.complete(base_64_encoder()); out << "hello world!\n"; out.push(zlib_compressor()); // error! you may be right that users would be less likely to make this mistake. I don't see how add_resource would help at all. I believe the current stack-like interface is elegant and intuitive. Reversing the order will also be confusing if I adopt JC van Winkel's pipe notation, which I plan to do. If I adopt both changes, the following would be equivalent: filtering_ostream out; out.push(file_sink("log")); out.push(base_64_encoder()); out.complete(newline_filter(newline::windows)); --- filtering_ostream out( newline_filter(newline::windows) | base_64_encoder() | file_sink("log") );
_______________________________ boost::io::converter
It shoudl be intern_type. In other words, the codecvt would be used *as is* if its external character type is the same as the character type of Resource. It would then be a normal 'widening' codevt. If, OTOH, its internal character type matches the character type of Resource, the codevt would be 'reversed' so that it would be a 'narrowing' codecvt. (As I mentioned elsewhere, the terms 'widening' and 'narrowing' are somewhat misleading.) E.g., converter< file_descriptor_source, utf8_codecvt_facet > would allow one to read a sequence of wide characters from a file containing multibyte characters. OTOH converter< array_source<wchar_t>, utf8_codecvt_facet > would allow one to read a sequence of multibyte characters from an array containing wide characters. The first use would be by far the most common. If the proposed formulation is considered too error prone, a boolean template parameter could be added: template< typename Resource, typename Codecvt = fetch_from_global_locale, bool Reverse = false > class converter;
Under your proposal, would a typical construction of a newline_filter look like this: newline_filter(write_CR, accept_LF | accept_CR | accept_CRLF ) instead of newline_filter(write_CR | accept_LF | accept_CR | accept_CRLF ) ?
_________________________________________________________________ boost::io::reverse
The problem with making symmetric filters the default (or exclsuive) filter concept is that it's unexpectedly difficult to implement them correctly.
I don't recall reading information on this rationale. It should be part of the docs, maybe a FAQ.
Okay.
Neither of your suggested interfaces is sufficient. The first one allows only character-for-character substitutions. The second, depending on the interpretation of the return value, needs to be augmented to indicate how many characters of the input sequence or the output sequence were consumed. It's somtimes necessary, e.g., to achieve a good compression ratio, to allow symmetric filters to output fewer characters than possible. In that case, one needs a boolean parameter to instruct the filter to flush it's buffers. As an example of the difficulty of writing symmetric filters, look at http://tinyurl.com/6bu23. It took my two hours to get the toupper_symmetric_filter to work properly. (Note that I added the internal buffer just to simulate a realistic use case.) If you're not convinced, try implementing the example filters as Symmetric Filters. Not impossible by any means, but substantially less intuitive. Also, even when the basic idea of the implementation is clear, it's easy to make mistakes with pointer arithmetic.
_________________________________________________________________ Documentation Comments
Got it.
What about in the ordinary case (not comments, not tutorials)?
Okay.
I guess when I'm describing how chains work, I'll mention that filters need to be adapted before being fed to the upstream filter. I think the concept documentation and the tutorial are pretty clear that the downstream component must be a resource.
Substituting 'stream position' for 'reading head', etc., yields some funy stuff like: "Modes can be categorized in several ways ... 3. Whether the reading or writing stream positions are repositionable, and if so, whether there are separate stream positions for reading and writing or a single read/write stream positions." "Seekable: a single sequence of characters, for input and output, with a combined repositionable read/write stream position." How would you phrase this stuff?
How about: "A resource is Direct if it provides access to its controlled sequences as in-memory arrays rather incrementally using functions such as read or write."
Okay.
I see -- the extraneous 'of'
See "The Class Template Mode" for example usage. ?
The trouble is I want the concept names to be unique not just within the library but in a wider context, including the standard library and the rest of Boost. So the concept name should have IO in it somewhere. Best Regards, Jonathan

From: "Jonathan Turkanis" <technews@kangaroologic.com>
I'm not sure Daryle would agree with that. Anyway, my point is that if Boost accepts both libraries, then the two of you need to determine the synergies and differences between your libraries and ensure users understand the value of each approach.
That, of course, doesn't follow the "able" convention you've established. Otherwise, it does get right to the point clearly.
Perfectly reasonable.
Gotcha.
Yes.
see how add_resource would help at all.
Because "add_resource" was offered as a synonym for "complete."
The first example is using the proposed, new syntax, so I'd prefer to see it written like this: filtering_ostream out; out.push(base_64_encoder()); out.push(file_sink("log")); out.complete(newline_filter(newline::windows)); Then, the second, which is confusing as written, should be: filtering_ostream out( base_64_encoder() | file_sink("log") | newline_filter(newline::windows)); Then, the two are quite similar.
Yes.
I presume, then, that this would work: boost::optional<char_type> filter(char_type ch);
interpretation of the return value, needs to be augmented to indicate how many characters of the input sequence or the output sequence were consumed. It's
Easily solved.
needs a boolean parameter to instruct the filter to flush it's buffers.
I don't quite understand your point, but that's immaterial. It sounds like something like this would work: std::pair<streamsize, streamsize> filter(char const * input, streamsize n, char const * output); Provided those interfaces are close, wouldn't this make writing symmetric filters easier?
It looks like the difficulty stemmed from having to keep track of whether there are more characters available on the input and whether writing to the output was still possible. Isn't this an indictment of the current approach? What I'm suggesting is that your library take care of reading and writing data, and let the filters only be concerned with providing the conversion of the input to some output. (I don't know the internals of your library, and I'm probably missing something key, but it seems as if some simplification should be possible.)
What is an "ordinary" case? Personal correspondence? Scientific report? Essay on the current geopolitical state of the world?
I suggested "location" as a way around "repositionable positions." Nevertheless: Modes can be categorized in several ways...reading or writing stream positions are Seekable and, if so, whether there.... and: Seekable: a single sequence of characters for input and output, with a common read/write stream position that can be moved to different parts of the sequence.
Bingo.
Yep.
Neither "Filter" nor "Resource" contain "IO." I think you're alluding to the generality of "Component" but isn't that a problem for "Resource," too? You can add an "IO" prefix, if you like, but offhand, I can't think of anything better. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;

"Rob Stewart" <stewart@sig.com> wrote in message:
I see your point now -- someone wants (no apostrophe ;-) to write a streambuf/stream pair. Daryle: Write the streambuf from scratch, then use streambuf_wrapping.hpp Jonathan: Write a Resource, then use streambuf_facade and stream_facade I agree with Jonathan :-) I've already given most of the reasons in my reply to Dietmar Kuehl, so if you don't mind, I'll quote myself (sorry for the length): "Jonathan Turkanis" <technews@kangaroologic.com> wrote:
I should add: even if someone reads one of the several available books on the standard iostreams library and decides to write a stream buffer from scratch, there's a good chance the implementation will suffer one of the following problems: (i) buffering will be omitted, since it's hard to do correctly. (ii) buffering will be provided, but mistakes in pointer arithmetic will cause subtle errors (iii) sub-optimal algorithms will be used. Note that two of Daryle's stream buffers suffer from defect (i).
"Peekable" does not imply being able to put back a character.
I know -- that's because 'Putbackable' is ugly, even when joined to 'Resource.' Here are some other ideas (based on your suggestions and a thesaurus): RevertableSouce, RestorableSource, UndoableSource, ReinsertableSource. progress.
But here, the component being added with add_resource (the base_64_encoder) is not a resource at all!
This seems totally screwy to me. ;-) There are two resonable conventions: I. Push the resource first, then push the filters, in order, starting with the one furthest from the user. II. Push the filters first, in order (the reverse of I), starting with the one closest to the user, then push the resouce. II is the convention I adopted, for reasons already explained. In the above example, there are two possibilities: I. file_sink <-- base64_encoder <-- newline_filter II. newline_filter --> base64_encoder --> file_sink (The arrows indicate the flow of data.) I can't see any justification for putting the resource in the middle, as you have done.
_______________________________ basic_newline_filter
That sounds like a good idea. Then there would be two constructors, used as follows: newline_filter(write_LF | accept_LF | accept_CR | accept_CRLF); newline_filter(posix); I guess this is what you already said.
_________________________________________________________________ boost::io::reverse
Right. Or a basic_character<char_type>, to deal with the 'no input currently available -- try back later' case.
This is the case with zlib. The longer it can store up input, the better the compression ratio it achieves. See http://www.gzip.org/zlib/manual.html#deflate Of course, if I merge the InputFilter and OutputFilter concepts, I can have both Filter and FlushableFilter concepts without increasing the number of concepts. (Except I want to use Flushable for something else.)
There are still two problems: 1. the output buffer is const, which seems wrong. 2. the filter has no way of knowing the size of one of the two provided buffers, depending on the interpretation of the streamsize parameter. So putting aside the issue of flushing, your suggested interface should be std::pair<streamsize, streamsize> filter( char const* input, streamsize input_size, char* output, streamsize output_size ); I consider this interface pretty much equivalent to mine. In fact, I considered having SymmetricFilters return std::pair<streamsize, streamsize> -- I can't remember why I chose the present interface. At any rate, I consider them equivalent and don't see how your version makes things easier. There's another problem with throwing out the current InputFilter and OutputFilter concepts. A filter which performs both input and output with two separate character sequences -- currently called InoutFilter but soon to be renamed BidirectionalFilter -- needs some way to know whether it's being asked to perform input or output. So the full interface becomes: boost::optional<char_type> filter(char_type ch, ios::openmode); for one-character-at-a-time filtering, and std::pair<streamsize, streamsize> filter( char const* input, streamsize n, char* output, streamsize n, ios::openmode ); for multi-character filtering. I'm much more comfortable sticking with the simple InputFilter and OutputFilter concepts, and providing SymmetricFilters for advanced applications. I should probably put a note in the tutorial that in theory a filtering algorithm is independent of whether input or output is being performed, and provide links to the documentation for reverse and for SymmetricFilter.
What is an "ordinary" case? Personal correspondence? Scientific report? Essay on the current geopolitical state of the world?
Reference documentation.
You need a supercategory of Filter and Resource. "Component?"
The trouble is I want the concept names to be unique not just within the
I like it. library
I'm leaning toward Device, which carries i/o connotations, instead of Resource. Your thoughts? Best Regards, Jonathan

From: "Jonathan Turkanis" <technews@kangaroologic.com>
Imagine that.
I've already given most of the reasons in my reply to Dietmar Kuehl, so if you don't mind, I'll quote myself (sorry for the length):
[snip rationale for IOStreams Library vs. MoreIO] You'll need Daryle's side of this when providing the rationale for choosing between the libraries. BTW, the reasons you cite are significant and certainly cause me to favor your approach over Daryle's. Unfortunately, I don't recall feedback from him to counter those specific claims aside from preference. (If preference remains the reason to keep both libraries, then your rationale and any from Daryle should make the case for each library and leave to the user the choice based upon preference.)
These are excellent reasons to hide the details of buffering in a framework and should be part of your rationale.
Note that two of Daryle's stream buffers suffer from defect (i).
I doubt that he omitted buffering because it is difficult.
*Between* these, I prefer UndoableSource. (I wrote "among" first, but couldn't resist.)
But file_sink is.
Well, duh! Let me try that again: filtering_ostream out; out.push(base_64_encoder()); out.push(newline_filter(newline::windows)); out.complete(file_sink("log")); filtering_ostream out( base_64_encoder() | newline_filter(newline::windows) | file_sink("log"));
There are other reasonable conventions. I like this one: III. Push the filters first, in order of data flow, followed by the resource. That is, if you're filtering input, then the filter connected to the Source's output is the first in the list. If you're filtering output, then the filter connected to the Sink's input is the last in the list. Put another way, the first filter to see data appears first, the last one to see data appears last. The only "odd" thing is that the resource always goes last. (Odd because for an input stream, you'd ideally want the resource to be first.)
III. (same as II in this case)
I can't see any justification for putting the resource in the middle, as you have done.
Nor can I. I was clearly distracted when I wrote that.
Bingo.
You wrote, "output fewer characters than possible." That's what I'd like to see! ;-)
Quite right. I just type "const" out of habit and then remove it when appropriate. In this case, I failed to remove it when appropriate.
2. the filter has no way of knowing the size of one of the two provided buffers, depending on the interpretation of the streamsize parameter.
I was just making a tacit assumption that the input and output buffers were the same size.
I don't think I even looked at your SymmetricFilter stuff, but yes, they do appear to be equivalent.
I think you'd just have two instances of the same filter when you want bidirectional filtering. The framework would take care of inserting each instance into the correct data stream.
I was considering that equivalent to a tutorial. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;

"Rob Stewart" <stewart@sig.com> wrote in message:
From: "Jonathan Turkanis" <technews@kangaroologic.com>:
"Rob Stewart" <stewart@sig.com> wrote in message:
You'll need Daryle's side of this when providing the rationale for choosing between the libraries.
In my last conversation with him I asked him for his view on various issues, but didn't hear back. My impression was that amount of code included was his most serious objection to my library.
Good idea.
Note that two of Daryle's stream buffers suffer from defect (i).
I doubt that he omitted buffering because it is difficult.
I realized I left this impression, so I corrected it in a separate message.
*Between* these, I prefer UndoableSource. (I wrote "among" first, but couldn't resist.)
To clarify, are you saying filtering_ostream out; out.push(file_sink("log")); out.add_resource(base_64_encoder()); out << "hello world!\n"; makes sense? <snip stuff that seemed totally screwy> the
I thought of this early on, and liked it until I realized that data can flow in both directions. (See Fig 5 at http://tinyurl.com/5p2f5.) Therefore I still think only I and II make sense, and that II offers significant advantages.
Okay, I'll do it.
Aha -- This is like the like the Schumann sonata marked "as fast as possible" and then later "faster."
2. the filter has no way of knowing the size of one of the two provided buffers, depending on the interpretation of the streamsize parameter.
I was just making a tacit assumption that the input and output buffers were the same size.
This won't typically be true. In fact, one of the buffers may just be the currently valid region of an adjacent stream buffer.
Yours is better, since it allows one of the streamsize values to be -1 to indicate EOF. There's no way to do that when you have to indicate the number of characters consumed by manipulating a pointer passed by reference.
The problem is that you may not want two copies of the same filter. You might want encryption for output and decryption for input. But reducing the number of concepts is an attractive idea, so try to sketch out what the hierarchy would look like with just the two filter interfaces under discussion. For instance, I have to see if random access can be handled okay. Best Regards, Jonathan

"Jonathan Turkanis" <technews@kangaroologic.com> wrote:
But reducing the number of concepts is an attractive idea, so try to sketch out
That should be "I'll try to sketch out ... " ;-)
what the hierarchy would look like with just the two filter interfaces under discussion. For instance, I have to see if random access can be handled okay.
Jonathan

From: "Jonathan Turkanis" <technews@kangaroologic.com>
No, I meant: filtering_ostream out; out.push(base_64_encoder()); out.add_resource(file_sink("log")); out << "hello world!\n";
What does "starting with the one closest to the user" mean?
(That link is broken, but I eventually found the figure.) Offhand, I don't know how one assembles a bidirectional filter and many pages at your web site won't load at present. But, if I can infer the right information, you have a single bidirectional stream(buf) plus a sequence of filters, right? The question is, do you assemble the filters in input order or output order, right? You say that II makes sense in that case, but you have to choose an order for pushing the filters, there isn't just one to choose from. The only thing you can do is decide whether the order is biased toward input or output. That is, you always assemble bidirectional filter sequences as if it was an input filter, for example. Have I missed something?
I was thinking that if you were trying to use one instance to be bidirectional, using two instances managed by the framework would work better and make the filter implementation simpler. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;

"Rob Stewart" <stewart@sig.com> wrote in message: Continuing where I left off yesterday.... Again, I'll only comment on the parts where I disagree, don't understand or believe more explanation is necessary.
streambuf_facade::streambuf_facade
You have two sections with the same heading;
I'm emulating the Dinkumware documentation style. For overloaded functions, there is a heading like basic_istream::operator>> followed by a recapitulation of the relevant part of the synopsis, followed by a decription of each member. I've found the documentation hard to read when I group too many overloads together, since I end up writing stuff like "this seventh member constructs a ..." So I split them up, but give them headers with the same name. Perhaps heading like streambuf_facade::streambuf_facade (I) streambuf_facade::streambuf_facade (II) streambuf_facade::streambuf_facade (III) or streambuf_facade::streambuf_facade (Default) streambuf_facade::streambuf_facade (Forwarding) streambuf_facade::streambuf_facade (From Policy) would be better in cases like this.
I don't see how adding the template arguments helps.
I choose a table explaining the parameters only when I thought it would be the more helpful format. There are cases where it seems unnecessary, e.g., mapped_file_sink(const std::string& path); Constructs a mapped_file_sink to access the file with the given pathname. path - the pathname of the file to be accessed.
Again, adding the template arguments just makes the headers harder to read, IMO.
I don't understand the ^^^^ under 'there is'. I guess I'm viewing your corrections with an incompatible font.
Good point. Based on the comments of reviewers, I plan to 1. explain early in the docs the roles of stream buffers and streams in the standard library i/o framework 2. explain that the streams in the current library are provided only as a convenience 3. Show how to use an arbitrary streambuf with a plain istream or ostream 4. Explain that most of the examples will use streams, since these are more familiar and will probably represent the most comon use of the library.
Thanks, the second should be filtering_stream.
They all apply to the streambuf variants of the stream templates used.
_________________________________________________________________ code_conversion.html
Seems to be a mozilla bug.
I agree that my wording is awkward, but I'm not sure I like your suggestion. How about essentially identical to filtering_streambuf except that both wide- and narrow-character filters and resources can be added to it. The only restriction is that the wide- character components must come first. When the first narrow-character components is added, the libray automatically inserts a code_converter.
I agree this looks funny, and will have to be changed. Refering to the supposedly familiar overload of basic_ostream::operator<< was supposed to be a quick way of explaining what copy does. But it clearly didn't work.
How about: A resource adapter which which takes a narrow-character resource type and produces a wide-character resource type by introducing a layer of code conversion using a std::codecvt. ?
Good point. I thought I was following the Dinkumware convention, but having just checked, I see that I wasn't.
I agree that associate/dissociate is awkward. How about attach/detach?
I'm planning an entire section on lifetime management issues.
My formulation sound more natural to me.
Detaches the converter's current Resource, if any, and destroys it, after calling any appropriate cleanup functions ?
I don't think "dissociate" (not "disassociates," BTW) is necessary.
Thanks. Google tells me that 'dissociate' is about twice as common as 'disassociate'. Maybe 'disassociate' should mean 'insult an attorney who hasn't made partner.'
I used H5 here because I thought the higher levels looked odd. But I think it's a moot point because I should move the rationale to the central Rationale section.
Mozilla -- up to her old tricks.
Good point. After an exception, and before the chain is modified, there is no guarantee.
Yes, this sounds awkard. How about: Angelika Langer and John Torjo provided helpful comments on the role of exceptions in the standard iostreams library. ?
_________________________________________________________________ regex_filter.html
Do you feel the same way about elision of the subject in function documentation which begins with "Returns"?
How about: users of regular expression filters may pass any function or function object to the basic_regex_filter constructor as long as its signature is compatible. ?
______________ basic_regex_filter::basic_regex_filter
How about: A Boost.Function object to which each regular expression match is passed. The return value will be used in the filtered character sequence in lieu of the matched text. ?
I'm not sure I see the problem. Maybe: See Examples (Regex OutputFilter).
Why? I hope it's not "the superstition that 'between' can be used only of the relationship between two things, and that if there are more 'among' is the right preposition." (Fowler) :-)
I wanted it to be prominent during the review. I'm planning to eliminate the note and maybe the template parameter. Do you think it's necessary to use a template here?
at least I didn't say libaries. ;-)
Hmmm. Omitting the 'the' seemed more natural to me. To check common usage, I tried the following Google query: [ "on work of" indebted ] and got this: http://tinyurl.com/4q82c. So I guess you're right. ;-)
_________________________________________________________________ file.html
I think 'buf' is residue from basic_filebuf and should be stricken.
_________________________________________________________________ rationale.html
Where's the split infinitive? (Don't make me quote Fowler again ;-)
Ditto.
Currently, 1. All filters, even those which do not model Buffered, use a buffer by default, for efficiency. 2. Bufered filters can be used without buffering, by specifying a buffer size of 0. The real difference between Buffered filters and non-Buffered filters is that the member functions of Buffered filters accept a *buffer* of characters: read(Source&, char*, streamsize) write(Sink&, const char*, streamsize) This is why I think MultiCharacter is more descriptive. I'd like to use Buffered for those resources which have their own internal buffers so that the library won't automatically add an unnecessary layer of buffering.
Yep.
_________________________________________________________________ adapt.html
Suggestions?
<http://home.comcast.net/~jturkanis/iostreams/libs/io/doc/functions/classes/b... _inserter.html#synopsis> I can't find it.
Are you sure? "The
This is wrong. Call the streambuf_facade instance 'sb'. Then calling sb.close() causes boost::io::close(r, std::ios_base::in) and boost::io::close(r, std::ios_base::out) to be called. These, in turn call r.close().
No, that's correct. buf1 only needs to be flushed using sync().
"...each element receives two closure notifications..." ^^^^^^^^^^^^^^^^^^^^^
I agree that this should be rephrased to eliminate 'notifications', but I don't think it's correct to say that an element receives calls.
As I explained yesterday, I restored the status of standard streams and stream buffers as full-fledged models of Resource, so the hypenated phrases can be removed.
How about: The size of the temporary buffer used for data transfer may be supplied. ?
Yes, I need to get this straight.
Because it's ugly :-) How about [see below]. BTW, I'm considering changing the return type to allow it to represent three types of cases: 1. a character was extracted 2. EOF was reached 3. No characters are surrently available -- try back later. I think this is one of the most important interface issues -- what is your opinion?
I see your point, but I'd like this information up front. How about 'one overload ... the other overload'?
I think renaming the parameters is a good idea, but that recapping the synopsis is also helpful.
in what way?
and why, "roughly speaking?"
Because I haven't formally defined the 'direction' of a filter.
As long as there's plenty of memory around, performance should be better than if reverse weren't implemented in terms of one_step_filter.
Also, shouldn't that paragraph be set off with "Note:" or "Warning:"?
I like "Warning".
Okay. The reason is that you may have an implementation of a filtering algorithm only as an output filter, but want to use the same algorithm for an input filter. As I mentioned before, some algorithms are much easier to implement one way than the other.
_________________________________________________________________ seek.html
The standard uses 'off', 'way', 'which'.
Here's a good place where lowercase seems clearly appropriate. Is Filter a Filter concept?
How about: The header <boost/io/io_traits.hpp> contains the definitions of the metafunctions char_type and category, which associate with each model of Filter or Resource its character type and category. ?
Thanks. This page was a mess, but I wan't sure what to do with it.
This must be a holdover from the days when io_traits was a 'blob' representing two metafunctions. Please point out where you see this, so I can eliminate it.
"simply define a struct deriving from each of the existing tags using multiple inheritance" ?
Categories are validated to a limited extent.
You'll definitely get compiler errors, perhaps along the lines of 'filter_wrapper_impl<any_tag> has no member function read'. I guess I should have a valid_category metafunction and use BOOST_MPL_ASSERT(valid_category<category>) frequently.
Good -- but how about mode is a metafunction returning a tag structure indicating the mode of a Filter or Resource. The mode indicates whether the Filter or Resource is usable for input, output, or both, and the type of seeking it supports, if any.
_________________________________________________________________ filtering_stream.html
_______________________________ Reference
Okay
Surprisingly, perhaps, the traits type is really not needed. I'm assuming that anyone who writes a new character type will also specialize std::char_traits (I should say this). The traits template parameter is provided just so people can construct streams and stream buffers with an alternate traits type, in case the alternate traits type is required by some other API. It won't affect the behavior of the stream or stream buffer. For example, suppose there is an existing library that has a function of the form void f(std::basic_istream<japan_char, japan_traits>&); If I didn't provide the traits parameter, instances of filtering_stream could never be passed to this function.
Yes.
Given the detailed description of T, what more is needed?
The first takes a const reference, the second a non-const reference.
The chain can't contain a Resource unless it's complete. So it should be Clears the underlying chain. If the chain was complete before calling reset, causes each filter and resource in the chain to be closed using the function close. Thanks again. Best Regards, Jonathan

From: "Jonathan Turkanis" <technews@kangaroologic.com>
Those would be distinct, but not informative.
Those would be better. However, rather than rely on headings at all, why not just let the synopsis excerpt divide things?
It is more correct. I was trying to say that if you didn't use the "Constructors" heading, but wanted to keep the streambuf_facade headings, then they really should include the template parameters.
As a reference guide, I'd expect consistency on this point so that I could easily spot desired information.
As you might guess from there only being four carets, it wasn't supposed to be under "there is." Using a fixed width font, the only way you can possibly understand such notation, you'll see that I was suggesting s/such/T/.
So long as you meant "most" and not "all" that should do pretty well.
You apparently have more than one filtering_streams.html and I didn't note the relative URL, so I can't find the section to which I was referring. However, I'm pretty certain I was trying to say that you don't mention filtering_streambufs, only filtering_streams in that context.
s/component/filter/ since this only applies to filters, right? Otherwise, it's fine.
Very nice.
I avoided "associate" altogether in my rewording. Did I not capture the right idea? Anyway, "attach" and "detach" are still not quite right because you make a copy, right?
That will be good.
Why do you need both "detach" and "destroy?" Doesn't "destroy" say it all?
LOL
I presume you'll mention that in your revised docs.
Excellent.
Most text I had read in your documentation, I suspect, was grammatically correct. This stood out, I assume, or I wouldn't have flagged it. However, I don't know whether you were consistent in omitting the article. If so, ignore my comment. If not, choose one convention.
That works.
That muddies the issue for me.
s/Examples/Regex OutputFilter/
Superstition? No, it's just been drilled into my head. You're right of course, that there's nothing wrong with "between" in that sentence.
I don't personally need it, but I can well imagine that folks would need a wide character version. Perhaps in that case, the line endings aren't really wide character?
8^)
It should be removed. I was probably tired and not thinking as clearly as I should have to try correcting your English!
No split infinitive; probably tired again. I do think the change sounds better.
No split infinitive; probably tired again. Either version is fine here.
I understand what you're trying to say now. I'm not fond of MultiCharacter, but offhand, I haven't a better name.
[first, last)
There are automated tools in Boost and there will be many readers. It will surface again, if it is still present.
Each filter and resource has but one state. (This may be a gray area, for all I know.)
OK, but do give thought to clarifying the text.
I assume "element" refers to a filter or resource, in which case calling a function is quite reasonable. What have I missed?
I still think it is useful to mention the entities sharing the buffer.
That may be, but you're hiding the truth.
Given your discussions on async I/O, this is probably necessary, but I've not spent any time considering the ramifications.
I don't understand why that information needs to be in the overview. If one is for internal use, you don't need to highlight it. Without those two bullets, the overview remains complete IMO.
The function template reverse takes a filter and reverses its direction of data flow. It forms an OutputFilter if the original filter is an InputFilter, and vice versa.
That's the sort of information needed in the documentation. Of course, I'm still unconvinced of the efficacy of the current filtering approach, as mentioned previously.
Does that make them good automatically?
Hmmm. I suppose you're right. Wouldn it be better phrased as, "...various Filters and Resources?"
That's fine; be sure to make "character type" and "category" be links.
http://home.comcast.net/~jturkanis/iostreams/libs/io/doc/io_traits.html, about halfway down.
Is it always multiple inheritance? That's what I was trying to convey by parenthesizing "multiply." Otherwise, it's fine.
Anything to help the user get things right.
Fine.
<confused> Is the traits type needed or not? If so, what facilities must it provide? </confused>
From this I infer that the traits type must meet the Standard's requirements for char_traits (21.1.1).
How about this: A Resource, if the filtering_stream is to be complete, or a Filters if the filtering_stream is not to be complete yet.
While I should have seen that, it is easy to miss and documenting them separately is a waste of space and the reader's time. Put the two synopsis excerpts together and document them as one. If you wish, you can mention that difference between the two, but with them adjacent, the difference should be more apparent.
Oops. I did switch those around, didn't I? Your rewrite fails to indicate what happens if the chain is not complete. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;
participants (7)
-
Bronek Kozicki
-
George M. Garner Jr.
-
Jeff Garland
-
John Torjo
-
Jonathan Turkanis
-
Rob Stewart
-
Tony Juricic