[boost] my 2 cents x 10 - Documentation, moreIO, iostreams, move_ptr, fsm, & boost

my 2 cents x 10 - Documentation, moreIO, iostreams, move_ptr, fsm, boost -------------------- Review Periods I would prefer to see a review period of 10-12 days [i like 12 days from a sunday thru the following weeks friday perspective.] The present period is just to short. Most of us are overworked, underpaid, and dont always care to look at works in progress of others until things are a bit more "finalized" and "stable" so to speak. Hence we may not peek early enough at a library to provide good guiding input but we can toss our 2 cents in during the final review melee lol :) ---------------------- Documentation !!! Please utilize & include more diagrams & examples in your library documentation!!!! Compare & contrast with existing implementations that are lacking, State why they are lacking, what they are lacking, with what the proposed library implementation presents to fill the missing pieces, how it is better in some respects to anything it "replaces", and why it replaces some typical library code currently in widespread use or fills a gaping hole. --------------------- moreIO & IOstreams Daryls rationales and motivations are completely lacking in depth and sharpness. The summarys are so general and non-delineated that they are no help to me in discerning the clarity of what he proposes as benefits and properties of his library. an example :: this snippet is from the wrapping streams rationale "The Standard stream base class templates are designed to use stream buffers with an independent lifetime, and include functionality for switching what stream buffer is used." Yes, but are you just reiterating something everyone already knew, do you feel that is a bad thing or a good thing? There are no "supporting" remarks no remarks of impact laden with design criteria or shortcomings. Not coming from a circle where I/O is the most important aspect of my development I am not an "expert" on streams. So reading the documentation which purports to support and sing the praises of this libary was completely useless to me. I had better luck just looking at the code. When i think rationale and motivation i think objectivity must be forefront. Even a simple outline or summary of design criteria clearly delineated and put forth makes it easier to read an overall document of this nature. Project managers and others working on a project need to see clarity of thought, clear cut guidelines, inputs, outputs, and the design that goes in the middle. I am sure as most have laid out the design of their libraries they have ad-hoc flowcharts, points pro & against certain features. You may wish to bring some of those out and flesh them into the documentation so that others can understand the motivations behind your design. It would surely make life easier for anyone reviewing the code to speak to the issues addressed and/or generated by the code. It would elevate the efficiency of the process. Now that isnt to say some fluff isnt nice *grin* null_stream yes. something missing that is necessary. would prefer though to see it constructed using the concept of sinks & sources from the turkanis IO library up for review. streambuf_wrapping I would vote no on this until it is cleaned up and a better rationale presented to me. array_stream maybe. you would have to convince me this buys something appreciable in an efficiency concern over the way the standard streams presently operate. perhaps give me 2 examples a non-I/O expert such as myself could "grasp" concretely. pointer_stream no. the concept would be better implemented again using turkanis IO lib up for review. value_stream no. this specialized stream doesnt seem all that beneficial to stuff it in to a library. if the appl programmer wishes to he can easily fabricate his own. [u have a value_stream for just a char, what about for just a user_type, for just a "string" etc why not a "pair_stream" that would return a pair of values etc ad nauseum] now that isnt to say such a stream cannot be useful on rare occasion but again it would be fairly easy to implement using an appropiate set of "sources/sinks" that could be implemented in turkanis IO lib. -------- Now having said that it may seem like i'm being a pita and trying to tear your hard work into pieces, but i think you have some good "ideas" but perhaps the implementation should come off differently. I think if you perhaps had looked into the other IO lib that jason was developing at the time you 2 coudl have had a meeting of the minds and come up with some great ideas and both done some things differently. This library is a software community and with scales ever growing great ideas shouldnt necessarily be pondered on by lone rocket scientists squirreled away with their code and not picking their heads up to look around and talk (though we all do accept input i'm sure, and even act upon some of it, it seems more often than not we do not communicate as much as we should in the "early stages" and leave it all for the end-game) Perhaps you 2 should get together and collaborate. [now there's a concept] --------- Now my rationale : We all agree a layered/tiered approach to complex software is the best way to go. It allows more degrees of freedom, greater re-user, greater specialization as well, and greater replacement of parts. It does push or mutate the complexity more from a particular piece of complex code into the complexity of "simpler components" with more complex entanglement, more complex communication and cooperation at times. This is why i would much prefer with some efficiency clean-ups & improvements that the turkanis library be used as a basis(implementation-wise) for any of the moreIO features accepted. I like the small raw nature & flexibility of the simpler "building pieces" at the bottom of this IO pyramid I can then compose my complex streams on these "sinks & sources" As to the turkanis library, I have not had as much time to examine it. I am not sure the merit of Carlo Woods remarks but from a 1st perusing I think i agree with him. I also partly agree with Daryl about some efficiency concerns and bloat concerns but i think with everyones assistance those can be hammered out or down in size. ---------------- 2 cents on names for manipulators & readability These comments are due to the disagreement some seem to have over things like newl versus newlines and proliferation of manipulators and whether some should be placed in the library and others left to appl developers to produce on their own. A) Do not have both a singular and plural version instead just opt for the plural version - the singular use could be the default and is typically understood as much by viewing code as well. that is to say dont have a skipline & skiplines instead just opt for skiplines skiplines(x) would skip x lines [thus for one line x==1] B) The use of underscores to break up and make names more readable for classes, vars & funcs is a good thing. Someone tried to bring up the point that the standard streams library was such n such a way well that doesnt just mean it was the best way in terms of its naming schema C) I prefer slightly longer more meaningful names getting away from obfuscation so when it comes to newl or nl i'd really prefer newline. i think most of us can type sufficiently fast and we are thinking "newline" in our heads we can type it with barely any speed difference nor loss in efficiency but it makes for a lot better readability than "nl" (my cognitive web sees "nl" as netherlands these days lol) now that isnt to say this should be taken to the maximum in the opposite direction!! macros/functions like "min" & "max" are plenty self-explanatory. Ok, so that was more like 3 cents on this topic :) ---------------- fsm This library seems & acts more like a "simulator" than an "efficient finite state machine". There is a very high level functionality tied into (possibly) a bare-bones base... not properly tiered or layered approach. Functionality is wrapped into one library rather than separated into add-on libraries with additional functionality. It doesn't seem to scale well even according to the author(s)? (most fsms a computer scientist would deal with encompass 100s to 1000s of states with as many different transitions possible) I would strongly urge something more efficient and bare bones along the lines of http://www.research.att.com/sw/tools/fsm/ take on the mantle of an "fsm" library in boost upon which coarse-grained objects can be simulated from above. would prefer this 'currently proposed' fsm to take on its true stance perhaps attaching the nomenclature of "hlfsm" --> "high level fsm". Due to its greater abstractions, fluffy aggregations and bulk in nature it isnt "efficient". If a library isnt efficient for its purpose or intended purposes then it will be under-utilized. I could never use this one as it is evolving away from an efficient fsm into more of a large coarse simulator. [now if i had need of that it would be great] This library seems more intended for simulation at a higher level than at a more simplistic state/event transition approach. Yes i am biased coming from an algorithms and mathematical background; I am more of a computer scientist who practices softare engineering than most application programmers who just want a product out the door and efficiency goes out the window. Can we adjust this library conceptually into a cleaner bare-bones fsm at one layer and then interject the higher level simulation aspects along with the ability to automatically produce code etc. [Canned automatically generated code is usually not so hot. In fact it rather bites. Any decent programmer can still out-optimize todays compilers based on intimate knowledge of their code & algorithms even on todays complex processors. Given that compilers have thousands of man-years of code & know-how how does one expect this fsm library to generate decent code or is it intended for someone who cannot program?] ------------------ On the subject of move_ptr How about self_consistent_ptr invar_postmove_ptr I must be a freak because "selfish_ptr" grows on me it instantly says "mine mine all mine i can move through proustian circles but it is still my object" moveptr is something i've seen in untold reams of computational code along with leftptr, rightptr, and currptr & variants thereof; so i'm really hoping someone comes up with better nomenclature to settle on :) -------------------- Lastly an area where I dont want to get into because no one manages software well lol The topic of boost - releases, the GABB (gawd almighty boost book), documentation & structure 1) is it possible to go with both a long-term more flexible scheme for release-dates along with a more rigid short-term window as we slide forward into the future? Note the famous M$ vaporware long-term software & release info is not the approach i'm talking about lmao more like "tentative" long range plans [the reiser boys(release 4 is out now) have done a good job with their long term feature & release targets] but sticking to your guns short term with branches rather than how it has been this "special circumstance" time I think most are of the opinion from the app & devel point of view that more "dot" releases are better rather than fewer. (of course within reason!) Presently it seems haphazard with cvs patch check-ins and outs and then fudging jam files, regression test failures and ... seems like by waiting so long between "dot" releases is NOT buying you any more efficiency or salvation from bug issues any more than having more frequent releases would but some of the libs within would be able to rev and get feedback faster. The compilation (syntax) issues are one thing but seems like some of the libs would benefit from hitting the streets when the developers have stomped most of their bugs. It becomes time the hapless consumer of the lib can play their guinea pig role and dive into their playground with much zeal. So i dont see any "good solid" argument as to why releases are so far apart rather than a bit more "reasonably timely" [ok well integration testing when a MAJOR lib-wide wholescale change occurs is one reason why a particular release would take forever :)] NOW if it was an end-user , we all know how psychologically important keeping the release #s down and only coming out with a major rev every few millenia(hic sarcasm hic) is great for business [hmm IE 6 hmmm how many years b4 IE 7? lolol] We devels are made of more sterner stuff though :) 2) Is the flippin book going to be available as ps or pdf since I dont feel the need to kill more trees just to have another book about programming yada yada yada collecting dust on my bookshelves? most programming books are so light on substance and full of wasted verbiage they could do without it isnt funny. [my fave c++ book was the annotated reference can ya tell? lol] If not, would at least the chapter summaries along with toc, glossary, and appendices be available for d/l? I much prefer cutting to the chase to get to information 3) Can someone get the documentation into a more consistent format along with the information at boost.org /boost-consulting site? as someone else pointed out the review information at boost.org was shall we say "slightly dated" 4) Structure ... oh forget it i've rambled enough on this random walk C++ya, xkey

Thomas Xkey wrote:
fsm
This library seems & acts more like a "simulator" than an "efficient finite state machine".
Please define "simulator" and "efficient finite state machine".
There is a very high level functionality tied into (possibly) a bare-bones base... not properly tiered or layered approach.
I believe the non-layered approach is a more or less obvious result of the requirements defined in the rationale. If you agree with this statement then please say exactly which requirements you have a problem with. If you don't agree I'd be happy to hear your proposal how to make boost::fsm more layered and how this layering introduces advantages compared to the current design.
It doesn't seem to scale well even according to the author(s)? (most fsms a computer scientist would deal with encompass 100s to 1000s of states with as many different transitions possible)
Please say where in the docs you think I say that it doesn't scale well.
I would strongly urge something more efficient and bare bones along the lines of http://www.research.att.com/sw/tools/fsm/
Unless I'm mistaken, this is a dynamically configurable fsm. For the reasons explained in the rationale boost::fsm is a static fsm. I don't think these two FSM variants can be compared in a meaningful way.
take on the mantle of an "fsm" library in boost upon which coarse-grained objects can be simulated from above.
You seem to imply that boost::fsm is going to be *the* boost state machine libraray forever (given that it is accepted). I don't think so. boost::fsm does not offer top-notch performance and never will (I don't think it is particularily slow either). boost::fsm is more geared towards projects that need to quickly implement large and complex FSMs drawing on all UML features. I acknowledge that boost::fsm *can* be the wrong choice for some projects and I'm therefore not at all surprised if another static fsm library with a different focus (e.g. along the lines of the MPL FSM) will make it into boost in the not too distant future. I strongly believe that a single best FSM implementation does not exist. The problem domain is just too complex.
would prefer this 'currently proposed' fsm to take on its true stance perhaps attaching the nomenclature of "hlfsm" --> "high level fsm".
I don't like hlfsm, but am otherwise open to suggestions for a better name (I agree that boost::fsm isn't the best possible name).
Due to its greater abstractions, fluffy aggregations
What do you mean with "fluffy aggregations"?
and bulk in nature it isnt "efficient". If a library isnt efficient for its purpose or intended purposes then it will be under-utilized.
What do you think is the intended purpose of this library?
I could never use this one as it is evolving away from an efficient fsm into more of a large coarse simulator. [now if i had need of that it would be great]
What do you use FSMs for?
Can we adjust this library conceptually into a cleaner bare-bones fsm at one layer
As I said earlier: I'll happily consider your proposal on how to do so.
and then interject the higher level simulation aspects along with the ability to automatically produce code etc.
No, this will not happen. One of the goals of boost::fsm was to get rid of the code generation step. I strongly believe that code generation in the FSM domain typically introduces more problems than it solves. Quite a few people argue otherwise and if this means that my library is not accepted then I will take it elsewhere.
[Canned automatically generated code is usually not so hot.
For good reasons.
In fact it rather bites. Any decent programmer can still out-optimize todays compilers based on intimate knowledge of their code & algorithms even on todays complex processors. Given that compilers have thousands of man-years of code & know-how how does one expect this fsm library to generate decent code or is it intended for someone who cannot program?]
Please define what you mean with "decent code". Again, a library that is "decent" in one situation might be bad choice in another. Usability was and always will be the topmost priority for boost::fsm. Yes, this means that the library will never offer top-notch dispatch speed or ultra-low memory footprint. Many projects using FSMs can easily trade some of theoretically possible performance for more important things like faster development. At least a few people seem to agree (TMK, there currently are 3 real-world projects using boost::fsm with at least some success). Regards, Andreas

"Thomas Xkey" <xkey@yahoo.com> wrote:
Review Periods
I would prefer to see a review period of 10-12 days [i like 12 days from a sunday thru the following weeks friday perspective.] The present period is just to short.
Agreed.
----------------------
Documentation !!!
Please utilize & include more diagrams & examples in your library documentation!!!!
Didn't you see http://tinyurl.com/7ylep, http://tinyurl.com/3pjps and http://tinyurl.com/3lvdw?
Compare & contrast with existing implementations that are lacking, State why they are lacking, what they are lacking, with what the proposed library implementation presents to fill the missing pieces, how it is better in some respects to anything it "replaces", and why it replaces some typical
library code currently in widespread use or fills a gaping hole.
I thought it might be interesting to add comparisions with java.io, Crypto++, OpenSLL and the Apache 2.0 filtering framework. But I didn't view this as essential, and just didn't have time.
moreIO & IOstreams
I think if you perhaps had looked into the other IO lib that jason was developing at the time you 2
Do you mean 'Jonathan' here?
coudl have had a meeting of the minds and come up with some great ideas and both done some things differently.
I posted at least four versions of my library, while I was developing it, and received a lot of good criticism. Daryle posted many versions of his library too. I examined Daryles, and I asked him to look at mine. What more do you want?
This library is a software community and with scales ever growing great ideas shouldnt necessarily be pondered on by lone rocket scientists squirreled away with their code and not picking their heads up to look around and talk
See above. I snipped the part of your message about your being a 'pita' but I'm considering adding it back. ;-)
Perhaps you 2 should get together and collaborate. [now there's a concept]
I suggested to Darlye that we combine our submissions, but he was not receptive.
This is why i would much prefer with some efficiency clean-ups & improvements that the turkanis library be used as a basis(implementation-wise) for any of the moreIO features accepted.
Specifics, please.
I like the small raw nature & flexibility of the simpler "building pieces" at the bottom of this IO pyramid I can then compose my complex streams on these "sinks & sources"
Thanks.
As to the turkanis library, I have not had as much time to examine it. I am not sure the merit of Carlo Woods remarks but from a 1st perusing I think i agree with him.
I have not posted my full response to Carlo yet, but I have tentatively concluded that the types of optimizations he is converned about (and the similar concerns of the architects of the Apache 2.0 filtering framework) are only required in limited cases -- roughly, when implementing network proticols or writing web servers. Most of the filters included with the library, and those suggested by reviers, would not benefit much, if at all. In some cases, it could slow things down. In addition, I've concluded that this functionality can be implemented later as an extension to the library, if desired, without modifying the current filter and resource concepts. 'Message aware' filter and resource concepts can be introduced and the internal filtering infrastructure can be reimplemented without modifying the public interface. It might be part of a 'networking library'. Finally, I have mentioned twice a number of optimizations I have not yet applied, because I don't know if they're necessary. (They should have been in the docs.) I'll paste them here again: Jonathan wrote:
There are several optimizations which I have held in reserve which would also minimize copying: a) Allowing resources to advertise that they are streambuf-based, so that i/o is performed directly to the underlying streambuf, with no additional layer of buffering b) Giving special treatment to symmetric filters (used to implement compression/decompression) to allow them to have direct access to the buffers of adjacent streambufs in a chain. c) allowing for a category of 'transparent filters' which simply observe character sequences, forwarding them unchanged. This would allow many useful filters (such as the offsetbuf suggested by David Abrahams) to have essentially zero overhead.
The category of transparent filters could be generalized any filter which can be repesented as a function
char f(char);
so that the length of a sequence of characters is never changed by the filters.
"Thomas Xkey" <xkey@yahoo.com> wrote:
I also partly agree with Daryl about some efficiency concerns and bloat concerns but i think with everyones assistance those can be hammered out or down in size.
Could you repeat Daryle's concerns which you found convincing? I asked him for clarification but I don't think he answered yet. Regards, Jonathan

Thomas Xkey <xkey <at> yahoo.com> writes:
fsm
This library seems & acts more like a "simulator" than an "efficient finite state machine".
I'm wondering if this post is more like a troll... But I'll bite.
It doesn't seem to scale well even according to the author(s)? (most fsms a computer scientist would deal with encompass 100s to 1000s of states with as many different transitions possible)
Not being a computer scientist, I can only assume your generalisation doesn't apply to anything that I've done with fsms. Or do you just mean that the computer scientists use telephones like the rest of us? I haven't tried to implement an fsm with 1000s of states as a single program/module, using boost fsm or anything else. 1000s of transitions is quite reasonable though. I shouldn't think boost fsm would have any particular problem with this.
I would strongly urge something more efficient and bare bones along the lines of http://www.research.att.com/sw/tools/fsm/
Interesting. If that is where you are coming from, I can only agree that boost fsm is not a good tool for eg. NLSR (mind you, I've seen at least one pretty awful tool for that). Conversely, the AT&T fsm tools look to be quite unsuitable for some other applications, and to be focused on the construction and manipulation of fsms for language processing (or other similar recognition problems). There is nothing wrong with that, but it does happen to be completely useless for a project I have jsut been using boost fsm on. There is room for both.
take on the mantle of an "fsm" library in boost upon which coarse-grained objects can be simulated from above.
I don't understand why you think boost fsm is for simulation. I can imagine using it for that (although being a compiled fsm, I'd suggest that would not be a common use) but it certainly has use beyond that.
would prefer this 'currently proposed' fsm to take on its true stance perhaps attaching the nomenclature of "hlfsm" --> "high level fsm". Due to its greater abstractions, fluffy aggregations and bulk in nature it isnt "efficient". If a library isnt efficient for its purpose or intended purposes then it will be under-utilized. I could never use this one as it is evolving away from an efficient fsm into more of a large coarse simulator. [now if i had need of that it would be great]
I don't know what, if anything Andreas intends to add to boost fsm, but I don't see it evolving towards anything bigger - it seems to be complete, given its objectives.
This library seems more intended for simulation at a higher level than at a more simplistic state/event transition approach.
I wouldn't want to force a real computer scientist to confront the possibility that there might actually be real things that are controlled, not simulated, by fsms but - um - there are. If all you want to do with an fsm is to use it as a recogniser taking some lightweight symbol representation as input and processing it until a final state is reached, boost fsm is probably the wrong tool. I am reasonably sure that trying to make one tool, or a shared component, do everything that can be done with fsms is not going to produce the desired/required efficiency.
Yes i am biased coming from an algorithms and mathematical background; I am more of a computer scientist who practices softare engineering than most application programmers who just want a product out the door and efficiency goes out the window.
What class of problem do you think "most application programmers" will apply boost fsm to, which will result in rampant inefficiency? In my experience, those of whom you speak are more likely to write thousands of lines of spaghetti, storing pieces of state in assorted knooks and crannies, rather than using anything as "inefficent" as boost fsm ;-) There is also the class of problem that is more likely to be addressed by boost regex or xpressive - in fact, I'm wondering if xpressive isn't somewhat closer to what you are looking for?
Can we adjust this library conceptually into a cleaner bare-bones fsm at one layer and then interject the higher level simulation aspects along with the ability to automatically produce code etc.
I don't see how to do this (assuming I'm understanding what you are calling "simulation aspects") - do you?
[Canned automatically generated code is usually not so hot.
In fact it rather bites. Any decent programmer can still out-optimize todays compilers based on intimate knowledge of their code & algorithms even on todays complex processors.
What canned code? The transition logic? The whole fsm library? I don't follow your point here at all?
Given that compilers have thousands of man-years of code & know-how how does one expect this fsm library to generate decent code or is it intended for someone who cannot program?]
Does this rant mean that the whole fsm library is futile and that real prog^H^H^H^Hcomputer scientists write in assembler? Not a bad language choice if your entire app is an fsm I'll admit. Regards Darryl.
participants (4)
-
Andreas Huber
-
Darryl Green
-
Jonathan Turkanis
-
Thomas Xkey