Any interest in a library for supporting program reuse?
Hi all, the compulsive coder here again, I've just written in a working prototype of a library for writing C++ programs which can be treated in C++ code like Unix filters, i.e. redirected to and from streams, and other programs. For instance: fstream f("c:\\tmp.txt"); stringstream s; HelloWorldProgram() > f > UpperCaseProgram() > s; Is there any interest in this kind of library, for boost? I am using it for writing test and demonstration programs of libraries, but I think it has a lot of other uses. The source code and a more detailed explanation is at http://www.codeproject.com/useritems/program-objects.asp Christopher Diggins http://www.cdiggins.com http://www.heron-language.com
From: christopher diggins <cdiggins@videotron.ca>
I've just written in a working prototype of a library for writing C++ programs which can be treated in C++ code like Unix filters, i.e. redirected to and from streams, and other programs.
For instance:
fstream f("c:\\tmp.txt"); stringstream s; HelloWorldProgram() > f > UpperCaseProgram() > s;
Is there any interest in this kind of library, for boost? I am using it for writing test and demonstration programs of libraries, but I think it has a lot of other uses.
From what I can see, you use the term "program" to mean an instance of class Program or, more specifically, a derivate
thereof. That isn't what I think of when I see the word "program." You cite an example in your article of piping the output of one *nix command to another using the output redirection operator, ">". The correct operator is the pipe, "|". You seem to have codified that misunderstanding, since you've overloaded ">" and not "|" for Program. Finally, your Program class is too intrusive since one must derive from it. It would be better if there was a make_filter() function template called with a function (object) returning a streamable object. Then, make_filter() can return an object which can invoke the function(object) as necessary and can be chained with other instances to create a pipeline. (Speaking of "pipeline," that is the *nix term for a series of filters piped together. Hence, your PipeChain class would be better named "Pipeline," and its "ChainTo" member function would be better named "PipeTo.") Oh, I should also point you to Jonathan Turkanis' IOStreams Library (recently accepted, but not yet added to Boost). See http://home.comcast.net/~jturkanis/iostreams/ for code and documentation. Look especially at the filters functionality. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;
Hi Rob, ----- Original Message ----- From: "Rob Stewart" <stewart@sig.com>
From what I can see, you use the term "program" to mean an instance of class Program or, more specifically, a derivate thereof. That isn't what I think of when I see the word "program."
What would you suggest the name of the class be instead? It is intended to provide the main entry point for a program, so that the program can be easily reused separately.
You cite an example in your article of piping the output of one *nix command to another using the output redirection operator, ">". The correct operator is the pipe, "|". You seem to have codified that misunderstanding, since you've overloaded ">" and not "|" for Program.
Thank you for pointing that out. I don't actually see a need for two different operators for piping and redirection within the library, except that it would be more appropriate for keeping with unix traditions. Ideally I would have one operator "|>" like in Heron.
Finally, your Program class is too intrusive since one must derive from it. It would be better if there was a make_filter() function template called with a function (object) returning a streamable object. Then, make_filter() can return an object which can invoke the function(object) as necessary and can be chained with other instances to create a pipeline.
Why is it considered better to model a concept rather than inherit from a base class and override a virtual function? It seems to me that it is simpler to explain to a newcomer to inherit from a class, and it is more apparent from looking at the code what is actually happening.
(Speaking of "pipeline," that is the *nix term for a series of filters piped together. Hence, your PipeChain class would be better named "Pipeline," and its "ChainTo" member function would be better named "PipeTo.")
Good point.
Oh, I should also point you to Jonathan Turkanis' IOStreams Library (recently accepted, but not yet added to Boost). See http://home.comcast.net/~jturkanis/iostreams/ for code and documentation. Look especially at the filters functionality.
I am sorry but I don't see the connection with the iostreams library and what I am doing. Could you explain the similarities? Thanks for your feedback. CD
From: christopher diggins <cdiggins@videotron.ca>
From: "Rob Stewart" <stewart@sig.com>
From what I can see, you use the term "program" to mean an instance of class Program or, more specifically, a derivate thereof. That isn't what I think of when I see the word "program."
What would you suggest the name of the class be instead? It is intended to provide the main entry point for a program, so that the program can be easily reused separately.
From what I see, you're packaging functions, not system() calls to executables or separate processes connected by some IPC.
Finally, your Program class is too intrusive since one must derive from it. It would be better if there was a make_filter() function template called with a function (object) returning a streamable object. Then, make_filter() can return an object which can invoke the function(object) as necessary and can be chained with other instances to create a pipeline.
Why is it considered better to model a concept rather than inherit from a base class and override a virtual function? It seems to me that it is simpler to explain to a newcomer to inherit from a class, and it is more apparent from looking at the code what is actually happening.
If I already have a function (object) that provides the desired output, why must I create a class, implement a member function that forwards to that function(object), and then use the class? A simple call to make_filter(f) will do the job without the need to write any code.
Oh, I should also point you to Jonathan Turkanis' IOStreams Library (recently accepted, but not yet added to Boost). See http://home.comcast.net/~jturkanis/iostreams/ for code and documentation. Look especially at the filters functionality.
I am sorry but I don't see the connection with the iostreams library and what I am doing. Could you explain the similarities?
Perhaps if I understood the distinction you're trying to make between a "program" and ordinary functions, we can understand one another better. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;
Why is it considered better to model a concept rather than inherit from a base class and override a virtual function? It seems to me that it is simpler to explain to a newcomer to inherit from a class, and it is more apparent from looking at the code what is actually happening.
If I already have a function (object) that provides the desired output, why must I create a class, implement a member function that forwards to that function(object), and then use the class? A simple call to make_filter(f) will do the job without the need to write any code.
Innheritance is a strongest relationship and shouldn't be used unless really needed. That what we learn from the expirience with modern C++ and from C++ experts (see recent Sutter books for example). Among other things your implementation employ runtime polimorpism for no particular reason IMO. Gennadiy
----- Original Message ----- From: "Rob Stewart" <stewart@sig.com> To: <boost@lists.boost.org> Cc: <boost@lists.boost.org> Sent: Friday, January 07, 2005 2:00 PM Subject: Re: [boost] Any interest in a library for supporting program reuse?
What would you suggest the name of the class be instead? It is intended to provide the main entry point for a program, so that the program can be easily reused separately.
From what I see, you're packaging functions, not system() calls to executables or separate processes connected by some IPC.
I am trying to describe a way to write a C++ program by packaging the entire thing (entry point, global functions, global variable) in a single object. So what I propose is that rather than write a program in a single cpp file as follows: ///////////////// // my_prog.cpp #include <iostream> char const* s = "hello world\n" int main() { std::cout << s << std::endl; return 0; } Instead what I propose is that instead you write the programs in two files as: ///////////////// // my_prog.hpp #include "programs.hpp" #include <iostream> class MyProgram : Program { const char* s; protected: virtual void Main() { std::cout << s << std::endl; } }; char const* MyProgram::s = "hello world\n"; ///////////////// // my_prog.cpp #include "my_prog.hpp" int main() { MyProgram().Run(); return 0; } Now having done that, the entire program can now be treated as a single object which can be redirected or piped. ///////////////// // my_prog3.hpp #include <fstream> #include "programs.hpp" #include "my_prog.hpp" #include "my_prog2.hpp" class MyProgram3 : Program { protected: virtual void Main() { MyProgram() > MyProgram2() > filestream("c:\\tmp.txt"); } }; My desire is to make it trivial to reuse the entire source code from programs, using a simple syntax as if they were shell scripts, and in a completely portable manner from within C++. This foregoes any kind of process call or whatever, because in C++ the notion of threads or process doesn't exist. I have written a small blog entry on the subject as well at http://www.artima.com/weblogs/viewpost.jsp?thread=87459 Is it more clear what I am trying to get at? Am I missing something about Jonathan's iostreams library, that makes it trivial to achieve the same thing? CD
christopher diggins wrote:
----- Original Message ----- From: "Rob Stewart" <stewart@sig.com> To: <boost@lists.boost.org> Cc: <boost@lists.boost.org> Sent: Friday, January 07, 2005 2:00 PM Subject: Re: [boost] Any interest in a library for supporting program reuse?
What would you suggest the name of the class be instead? It is intended to provide the main entry point for a program, so that the program can be easily reused separately.
From what I see, you're packaging functions, not system() calls to executables or separate processes connected by some IPC.
I am trying to describe a way to write a C++ program by packaging the entire thing (entry point, global functions, global variable) in a single object.
Why? Monolithic designs are seldom a good idea. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
----- Original Message ----- From: "David Abrahams" <dave@boost-consulting.com>
christopher diggins wrote:
I am trying to describe a way to write a C++ program by packaging the entire thing (entry point, global functions, global variable) in a single object.
Why? Monolithic designs are seldom a good idea.
All Java and C# programs package global data in a single class, so I don't see a problem with so-called "monolithic" designs. In hind sight there is no reason to require that the global data and functions are in the class, as long as one is uses namespaces conscientiously Christopher Diggins
On Fri, 07 Jan 2005 16:24:10 -0500, christopher diggins <cdiggins@videotron.ca> wrote:
All Java and C# programs package global data in a single class, so I don't see a problem with so-called "monolithic" designs. In hind sight there is no reason to require that the global data and functions are in the class, as long as one is uses namespaces conscientiously
This strikes me as a deficiency in C# and Java. --Matt
christopher diggins wrote:
I am trying to describe a way to write a C++ program by packaging the entire thing (entry point, global functions, global variable) in a single object.
Perhaps I am missing the point, but why not simply do something along the lines of "popen"? This will not put any requirements other than having stdin and stdout on the "base object program"? Roland
----- Original Message ----- From: "Roland Schwarz" <roland.schwarz@chello.at> To: <boost@lists.boost.org> Sent: Saturday, January 08, 2005 5:57 AM Subject: Re: [boost] Any interest in a library for supporting program reuse?
christopher diggins wrote:
I am trying to describe a way to write a C++ program by packaging the entire thing (entry point, global functions, global variable) in a single object.
Perhaps I am missing the point, but why not simply do something along the lines of "popen"? This will not put any requirements other than having stdin and stdout on the "base object program"?
The approach I am proposing, doesn't require the programs to have separate executable files. It also is completely platform agnostic. With something like popen() you have the following requirements: - you have to compile multiple executables - the executables must exist and you know where they are - the operating system supports piping - there is an operating system Multiple executables can be tricky to manage, etc. I want my libraries to have a single C++ program, which runs every test and benchmark, which is platform independant. This makes testing much easier. Christopher Diggins http://www.cdiggins.com http://www.heron-language.com
fstream f("c:\\tmp.txt"); stringstream s; HelloWorldProgram() > f > UpperCaseProgram() > s;
Is there any interest in this kind of library, for boost? I am using it for writing test and demonstration programs of libraries, but I think it has a lot of other uses.
The source code and a more detailed explanation is at http://www.codeproject.com/useritems/program-objects.asp
I gave it a quick look. For one thing naming completely misleading. I expect "Program" word to be used in system calls invocation domain. It we would stick to proper naming, I believe iostream library already doing exactly what you are trying to do here (but much better, since it does not require inheritance! to implement filters) Gennadiy
----- Original Message ----- From: "christopher diggins" <cdiggins@videotron.ca> Newsgroups: gmane.comp.lib.boost.devel Sent: Thursday, January 06, 2005 5:12 PM Subject: Any interest in a library for supporting program reuse?
Hi all, the compulsive coder here again,
I've just written in a working prototype of a library for writing C++ programs which can be treated in C++ code like Unix filters, i.e. redirected to and from streams, and other programs.
For instance:
fstream f("c:\\tmp.txt"); stringstream s; HelloWorldProgram() > f > UpperCaseProgram() > s;
As some has point out, I think that it should not be necessary to derive from a class with a virtual function. Unnecessary coupling. I have take a look at the article on CodeProject and here are my comments: You might instead want to uses CRTP (Curiously Recurring Template Pattern) or probably even better would be to uses boost::function (or something similar). boost::function<void (istream &, ostream &)> In fact, you could have a templatized manager class that would even allows to specify the original and final stream: template <typename IStream, typename OStream> class Manager { public: // Initialize temporary stream for the next piping operation. void SetupStream(); TmpIStreamType &get_istream() const; TmpOStreamType &get_ostream() const; }; template <typename IStream, typename OStream, typename Function> Manager<IStream, OStream> &operator|(Manager &mgr, Function f) { mgr.SetupStream(); f(mgr.get_istream(), mgr.get_ostream()); return mgr; } *** Well, I think this should give an idea of a possible design. In fact, we might need a temporary object so that we would be able to write to the final stream but we can also do it in the destructor (although this might cause problem wwith exception safety if an error occurs while writing to destination. A simple solution would be to have an object that would indicate to flush to the destination. This would even allows to write to the target in part if after writing and flushing the stream we leave it open (and the input stream would be kept at the current position). An exemple: Mgr(params) | Fn1 | Fn2 | Fn3 | Flush | Fn4 | Fn5 | Flush. This would be similar to the uses of ends for STL stream but eventually with the possibility of having more than one.
Christopher Diggins
----- Original Message ----- From: "Philippe Mori" <philippe_mori@hotmail.com>
----- Original Message ----- From: "christopher diggins" <cdiggins@videotron.ca>
As some has point out, I think that it should not be necessary to derive from a class with a virtual function. Unnecessary coupling.
I can live with that. [snip]
*** Well, I think this should give an idea of a possible design.
Thank you for the suggestions and taking the time to look over the code.
In fact, we might need a temporary object so that we would be able to write to the final stream but we can also do it in the destructor (although this might cause problem wwith exception safety if an error occurs while writing to destination.
Oh, this is how I implemented it in my code. I forgot about the dangers of writing code in destructors.
A simple solution would be to have an object that would indicate to flush to the destination. This would even allows to write to the target in part if after writing and flushing the stream we leave it open (and the input stream would be kept at the current position).
An exemple:
Mgr(params) | Fn1 | Fn2 | Fn3 | Flush | Fn4 | Fn5 | Flush.
This would be similar to the uses of ends for STL stream but eventually with the possibility of having more than one.
This is an interesting idea. In order for control (i.e. partial reading and partial writing) to switch back and forth between Functions, it would however require the use of threads. This counters my original design intent, but I am not against doing a separate mini-library which is more appropriate for boost. I think it would be appropriate, if I go in this direction to then use the boost::iostreams library, and write a function to a threaded iostream-filter converter. Then perhaps I could simply overload the pipe operator (|) for iostreams filters? Are we on the same page here? And is this something that has some potential value to boost? Christopher Diggins http://www.cdiggins.com
participants (7)
-
christopher diggins -
David Abrahams -
Gennadiy Rozental -
Matt Austern -
Philippe Mori -
Rob Stewart -
Roland Schwarz