December 2004 - Boost - lists.preview.boost.org

thoughts on a GUI component library
by Reece Dunn 29 Dec '04

29 Dec '04

Hi All, I have been working on revising my GUI library that is currently in the sandbox. The motivation for doing this was Andy Little's comment that most GUI libraries start from a particular operating system and then abstract its interface, providing work-arounds for other GUI systems. There are several considerations to make with a GUI library. For insatance, it should be possible to support GUI interfaces on portable devices (e.g. Palm Pilot/PalmOS) and terminal devices (like is done with the VIM editor). With these devices, code needs to be written to provide common components such as buttons. Also, the target operating system places design problems on creating a generic interface. For example, Windows supports char * and wchar_t * versions of its API, but the version that should be used depends on whether you are targetting Windows 9x or NT based systems. Windows also requires each component type to be registered before you can create an instance of it. Each operating system has different arguments that it passes when an event is generated, containing data relating to that event. Consideration must be made to allow events to be written generically, but also allow you to hook into OS-specific events and get OS-specific data. With all this in mind, how do you design a GUI library for C++? Although a GUI component exists on the screen and is drawn on that screen, the components and graphics elements should be kept seperate, since it is primarily the OSs role to render a component. There are shared elements (mostly the position, size and area types) and the graphics layer should allow for graphics-based events to be handled by the components library. Taking a look at how C++ is designed, it is predominantly data orientated. That is, C++ allows you to create new data types that fit into the language via operator overloading. The STL provides collections of data and algorithms for manipulating those collections. Terminal-based applications are supported by I/O streams that allow you to send data out and read data in. I have taken this as the primary design feature when implementing the GUI components. Thus, each component has an event to signal user interaction (ondatachanged, onselectionchanged, onpressed) and data that is associated with the component. The first thing that comes to mind when reading "data that is associated with the component" is data binding. I think it is a mistake to build data binding directly into the component architecture. The reason for this is twofold: You would need to use templates to bind the different data objects to the control which makes it harder to implement the controls in a platform-independant way and can lead to code bloat. Also, consider having a textbox widget that you use to enter the age. Having: gui::textbox< long > age; How does the textbox perform the conversion? By storing the data as a string, the user can extract the information on data collection like this: long age = boost::lexical_cast< long >( age_input.get_text()); or long age = ::atol( age_input.get_text()); Another thing to take into account is when an input field has a validity based on other component values. For example, the valitity of a dates day field is dependant on month and year. In this instance, it is best to put all the validation logic in the base component and extract the data to a date class such as Boost.DateTime's date. A component has a size and position, as well as visibility (visible or hidden) and enabled/disabled states. The component should make it easy to interoperate with native API. The majority of the current proposals (mine included) have taken the approach of providing native interoperability with the OS-specific data structures. This is very difficult to maintain properly and support generic code because of the different names of the data elements, types used (long or float) and whether an area is (position + size) or (top-left + bottom-right). Also, most OSs have (0,0) being the top-left corner, but MacOS has (0,0) being bottom-left. With this in mind, I have taken a drastic approach: do not provide native interoperability for size, position and area. I have designed them as: struct position{ float x; float y; }; struct size{ float dx; float dy; }; struct area { float top; float left; float width; float height; }; This means that the interface must convert these to the native representations and vise versa. The current implementation (Windows specific at this time) can be found at: http://uk.geocities.com/msclrhd/gui/gui.zip Thoughts? Comments? Regards, Reece

14 36

[iostreams] New version avaiable
by Jonathan Turkanis 29 Dec '04

29 Dec '04

Dear All, The latest version of the Iostreams library is now available: http://home.comcast.net/~jturkanis/iostreams/ I'd like to thank everyone who sent me bug reports after I posted the previous version. I will include you in the acknowledgments when I rewrite the documentation. Contents: - I. Changes - II. Supported platforms - III. Remaining bugs. ----- I. Changes. There are two important changes. First, the code conversion component has been renamed "code_converter", and it now has an optional Codecvt template parameter. Example. #include <boost/iostreams/code_converter.hpp> #include <boost/iostreams/device/mapped_file.hpp> // Perform code conversion using the global locale: typedef code_converter<mapped_file_source> wide_mapped_file_source; // Perform code conversion using utf8_codecvt_facet: typedef code_converter< mapped_file_source, utf8_codecvt_facet<wchar_t> > utf8_mapped_file_source; code_converter has now been tested with all the codecvt facets from the Dinkumware CoreX library. Second, there is now an option to build the libraries zlib and bzip2 from the source, rather than linking to a pre-built version. To build zlib from the source, define the following variables: ZLIB_SOURCE - path to the .co files ZLIB_INCLUDE - path to the .he files, if different from ZLIB_SOURCE To link to a pre-built version, define the follwing variables: ZLIB_INCLUDE - path to the .h files ZLIB_LIBPATH - path to the library or archive file ZLIB_BINARY - name of the library or archive file If ZLIB_LIBPATH is not defined, it is assumed that the library file is in a location where it will be found automatically. If ZLIB_BINARY is not defined, it defaults to "z" on UNIX and Cygwin, and to "zdll" on windows (see http://www.gzip.org/zlib/DLL_FAQ.txt) Similar remarks apply for bzip2. ----- II. Supported platforms The tests have been run on the following platforms: VC7.1 with Dinkumware GCC 3.2 (MinGW) GCC 3.4.2 (MinGW) GCC 3.4.1 (Cygwin) CodeWarrior 8.3 (static runtime only) Intel 7.1 with Dinkumware (static runtime only) Intel 8.0 with Dinkumware Borland 5.6.4 with STLPort Failures: - GCC (MinGW, Cygwin): wide streams not supported - Borland 5.6.4: undiagnosed runtime errors in seekable_file_test.cpp, direct_adapter_test.cpp and code_converter_test.cpp III. Remaining bugs. Although the bzip2 tests pass on all the platforms in II, there is an outstanding problem reported by Michel Decima which I haven't been able to fix yet. Best Regards, Jonathan

1 1

Template Metaprogramming I/O (well, O anyway)
by David Greene 29 Dec '04

29 Dec '04

I asked this question a year or two ago on the list. Has anyone come up with a good way to dump template metaprogramming information at compile time? I'd like to be able to do this both in a "print" sense (compiler warnings) and an "error" sence (compiler abort). I hacked something together last week that sort of does the job. There's a print<Message, FileLine> class that has a constructor with an unused variable declared. gcc gives a nice warning with -Wall. I also have some hacked macros to pass file and line number information to the printer. The printer will dump a warning if its constructor is ever instantiated. The use of all this appears in my last post in the "Named Template Parameters" thread. But this is really terrible. I don't like it much so I'm hoping someone else has a better solution. In particular, it's almost useless if there are any errors in the code because compilation will stop before instantiation happens. Unfortunately, this is _exactly_ the case where one really wants to get this information and figure out what's causing the compiler abort. I know this is useful functionality, practically required for any large metaprogramming project. I'd like to see if Boost can make something really good. -Dave

3 6

Re: Interest in a fast lexical analyser compatible with Spirit
by Dave Handley 28 Dec '04

28 Dec '04

Hartmut Kaiser wrote: >Since tokens have to be copied around in a typesafe manner I'd have >expected >to have something similar to the boost::variant, but maybe I've got you >wrong... How do you plan to achieve this? Copying through a base class >won't >work AFAICT? In my experience, copying around tokens is a major source of slow-down in lexers. The last lexer I wrote, I had to abandon the token class and use a C style union containing ints, doubles and std::string pointers. At the time, I was a little restricted in the amount I could use boost by my company, so I couldn't use boost.variant. Since then I've been putting a lot of thought into this problem and come up with one solution. We haven't completed integration testing with Spirit and other libraries yet using this technique (and I can foresee some issues already :-( ). The solution uses flyweighted tokens. The actual token exists in the rule, and during parsing, if the token has an assigned value, this is copied into the flyweight, then the token is passed around by reference/pointer. When you come to use the token in the parser you have a number of options: 1) Use it and then throw it away as soon as you call the ++ operator on the token_iterator. 2) Get the data out and do something with it - for example put it in a list/deque, etc. 3) Copy the token and use the copied version. For this final option, the token supports a virtual clone function to create copies of the flyweighted version for use elsewhere. An implementation point here is that the iterators contain a pointer to the token so that the iterator can be copied, or have the equals operator called, etc. There are some other potential solutions. However, any other solution leaves the problem of how to allocate and deallocate tokens on the heap very fast. I considered using Alexandrescu's small object allocator from the Loki library, and have also briefly looked at a boost.pool, but I am not comfortable with either of these solutions, because I am convinced that it will add too much of a burden to token creation. In my last lexer I found one or two orders of magnitude of speed improvement when I removed the token class and replaced it with a union. Since our first priority on this project is speed, I do not want the framework to slow down the DFA at all. The critical path in execution should definitely be the DFA. Dave Handley

1 0

Re: Interest in a fast lexical analyser compatible with Spirit
by Dave Gomboc 28 Dec '04

28 Dec '04

> "The Hapy library would not exist if Spirit would generate correct > parsers by default..." > > Whoa. Them's fightin' words! I believe the author is referring to incomplete backtracking (not backtracking past an accepted token when there's a failure later on). It's something the Spirit gurus have taken into consideration for Spirit 2, AFAIK. Dave

2 1

Interest in a fast lexical analyser compatible with Spirit
by Dave Handley 28 Dec '04

28 Dec '04

Since I get the mailing list in a digest, it looks like I have a lot of questions to answer. Sorry for the long(er) post, but I'm answering them all in one post: Darren Cook wrote: >There seems some interest currently in alternatives/improvements to Spirit, >so I thought I'd mention Hapy: > http://hapy.sourceforge.net/ I'll have a look at Hapy and see whether we could interface to that as well. Joel de Guzman wrote: >Some more things to note: >* why use lexeme_d on the lexer stage? >* the comment rule can be rewritten as: > '#' >> *(anychar_p - eol_p) >> eol_p; >* stringLiteral can be rewriten similarly. >* word can be rewritten using chsets. I used lexeme_d because the VRML grammar uses whitespace to separate ints and floats. Without the lexeme_d, ints could merge into one. The other optimisations are all valid, but unfortunately, don't make any difference to the stopwatch. This is because comments, stringLiterals and words don't occur too often in a VRML file - most of the time is spent parsing floats and ints. Joel de Guzman wrote: >Sure, no problem. But watch out, Spirit-2 will have a speed improvement >in the 10X-20X range over classic Spirit ;-) I already have some test >cases, and this does not touch on predictive parsing and lexing. Joel, I would be really keen to try running my VRML test case through Spirit-2. Given that I've now got the Spirit-1 parser running at about 6-7 seconds, if you get a 10x speed improvement on this file, then it could make the case for separate lexing much harder to support! I may also need a bigger file to start doing test cases on - millisecond test timings don't make for accuracy in my view :-) Joel de Guzman wrote: >Anyway, as I said, I'm very interested with your lexer. >Would you be interested in a collaboration? Yes, we would be very interested in a collaboration. Hartmut Kaiser wrote: >How do you specify production rule's? Your above sample specifies how to >use >the recognition part only ;-) >The numbers are the token ID's, I assume? The production rules are wrapped up in the different (and extensible) concept of a token. The token_float for example will match a float, then store the matched number as a float inside the token. The normal token stores nothing since something like a keyword match you don't need the overhead of the original text, but the token_string stores the string that was matched. I am contemplating an interface that allows you to use a functor to perform some basic string manipulation before the token is stored - for example dropping the enclosing quotes or similar. Although this is clearly similar to functionality already available in Spirit. Yes the numbers are token IDs - I have contemplated using const char * template parameters as well, but at present I haven't done that because of the added difficulty of ensuring that your names have external linkage. Perhaps in the final version we can support both. >Is your token type a predefined one or is it possible to use my own token >types instead? >I'm asking, because Wave in principle allows to use any token type for its >work, but it requires the tokens to expose a certain (very small) >interface. In the system as stands, all token types must inherit from a base version. This base version provides lots of useful virtual functions that derived token types can work with. The iterator interface then returns the token_base from its de-reference function. You can define new token types, but accessing the specific interface on the derived token means doing one of: 1) Ensuring the interface is exposed in the token_base. 2) Using polymorphic despatch to get the correct type, through a visitor for example. 3) If only a single token type is used in the lexer then use a static_cast. 4) Using a dynamic_cast. You can also define new rule classes - at present we have written a rule class for conventional tokens, and another rule class for tokens that have a dynamic (run-time) ID rather than the static (compile-time) ID in the template parameter. Dave Handley

3 2

[GUI] some random thoughts on GUI libs
by Christopher D. Russell 28 Dec '04

28 Dec '04

Some random thoughts on GUI libraries: Message filtering, dispatch: Somewhere buried in Joel de Guzman's Spirit or Phoenix docs I recall mention of functional programming techniques being useful for working with graphical user interface library organization. This comment stuck in my head as I generally dislike the organization of message dispatch and filtering code in libraries like the MFC and the WTL. I've been interested in the idea of leveraging Spirit to implement event processing systems for some time (here we parse event streams - not characters, using Spirit's grammar to encode state machine transition logic, and dispatching specific state-bound actions via Spirit's semantic actions). In the context of a GUI library, it's occurred to me that Spirit might be employed to "parse" streams of mouse, keyboard, and graphic subsystem notification events via a custom Sprit scanner. This of course implies that mouse, keyboard, and graphic subsystem hooks are abstracted in some platform independent manner but that's doable. In essence, this is just another way to encode complex state machines. But I think it would be cool to encode the message filtering and dispatch logic in EBNF! Message routing: signals / slots? Window / subwindow hierarchy: Given that typical GUI applications have many "windows" that are typically created and destroyed in response to (a) user input (b) non-graphical subsystem events, a succinct mechanism is needed to keep track of parent / child windows hierarchies. This information is used to calculate / transform drawing coordinates, and affect changes to message dispatching / routing / filtering. I think this information should be modeled using the BGL so we can customize the overall behavior of the system using graph algorithms. One could go a step further even and use a top-level graph to model the windowing hierarchy and another graph associated with vertex properties to encode an arbitrary closed polyline to represent non-rectangular windows. Threads: It's useful to be able to associate a "window" with a thread for high-performance GUI drawing. This is somewhat of a pain under Windows for legacy reasons but it's possible (I've done it within the context of the WTL). Here you simply blast the video subsystem with draw requests from multiple threads and place the onus of optimizing the call sequence on the driver and the underlying hardware. It would be useful to take this into account explicitly and build multithread support directly into the library. Placing the burden of this on the library user will invariably limit the portability of any application written against the lib I think. Just my two cents worth - Chris

6 10

[gui] On Native Controls
by Alan Gutierrez 28 Dec '04

28 Dec '04

* I don't think it matters much, to the user, if the button they click is a Motif button or an Aqua button, so long as their form is processed. Go to Amazon. All the buttons there are images. * Dealing with Win32 GDI controls is pain. Because they consume a Win32 window they are not going to place nice a more complex layout like a grid or a document. They are so few an simple, it is trivial to replicate them. The Win32 GDI is going to be a real problem, and it will put unnecessary design constraints on the GUI library if its poor behavior is enabled. * It is an aspect of the View of MV[PC] and the developer must be able to choose the View that suits their application. I think native controls should be supported in the case of PalmOS, where you can't overlay them anyway. Or in the case of windowing libraries that allow their controls to be clipped, where the native controls will behave properly when overlayed, and thus can take part in a complicated layout. -- Alan Gutierrez - alan(a)engrm.com

7 11

RE: FYI Lambda Question
by Brian Braatz 28 Dec '04

28 Dec '04

> -----Original Message----- > From: Powell, Gary [mailto:powellg@amazon.com] > Subject: RE: FYI Lambda Question > > So I'm checking now but I believe we set a typedef for the use of > "boost::result_of" > http://www.boost.org/libs/utility/utility.htm#result_of > > > -Gary- Thank you for your help that looks like EXCATLY what the doctor prescribed :) We MIGHT have some missing files? I am about to convert to boost 1.32 I am still on 1.31. In the 1.31 zip file I downloaded (and am using), the header does not exist in the utility dir. I grepped the boost tree for result_of and only found one comment in transform_iterator.hpp which is stated as a question ("do we need to adjust the way function_object_result is computed for the standard proposal (e.g. using Doug's result_of)?") (line 64). The link to the web page you sent me points to a HPP file. This file does not seem to exist in utility. For experimenting, I copied the file referenced in the page below to boost\utility. It complained on the line #include <boost/mpl/has_xxx.hpp> That has_xxx.hpp does not exist. Is it possible the file got dropped somehow? (it would be a shame for such a useful template to be "lost") Thanks again for your help. Brian

2 1

Interest in a fast lexical analyser compatible
by Dave Handley 28 Dec '04

28 Dec '04

Hartmut Kaiser wrote: >I'd be willing to write the interfacing stub to plug your library into >Wave. Thanks - I think this should be relatively easy because we currently use a forward iterator across the tokens in order to generate suitable output for Spirit. Clearly one of the requirements for the Boost community to accept this library in general will probably be meeting the Standard requirements for forward iterators - and I note that this is one of your key requirements in Wave. >The two different lexers I was using in the Wave library were a re2c based >lexer (static switch based lexer, extremly fast and compact) and a SLEX >based lexer (runtime generated DFA). I haven't done serious speed >measurements, but the numbers I've got so far showed similar timings for >both with a slight advantage for re2c (as expected). I'd expect your >library >to be very similar in speed as well. But just out of curiousity I'm very >interested in seeing the static DFA generation version :-) Can these lexers be effectively used with any Spirit grammar? I'll download Wave over the next few days and have a look at how much overlap there is between our library and the lexers within Wave. My plan for the static DFA version (which we haven't fully discussed in our internal design discussions yet so may change significantly), is to use the memento pattern to reflect the internal state of the DFA, along with the production rules. These could then be saved or loaded, or serialised to a C++ file, in a rather similar way to flex. This would mean that the lextl classes would be both an inline and an offline tool. In principle this gives some very desirable advantages: 1) The API remains the same in both compile-time and run-time versions. 2) You can very easily swap between run-time and compile-time versions - for example during development you may use run-time creation to speed up the process of developing a complex grammar, then switch to compile-time for your production release. 3) You have a number of options for compile-time versions. The grammar can be in a configuration file, or compiled into the program directly. >at least you could have used the symbol parser ... <snip> I've changed the Spirit grammar to use the symbol parser, and dropped the time from 40-50 seconds to about 6-7 seconds. Thanks for the tip. >Is there any documentation available? No, I'm sorry, we don't have any documentation at present, because we are still only a little beyond a prototyping stage. We still have to refine the API a little, and complete some optimisation work before we publish an early Alpha. As I said in my original post, I'm trying to gauge interest at the moment, and I'm pleased to see that we are generating a little interest. As a flavour, the current API would result in some code looking like: syntax<> mySyntax; mySyntax.add_rule( rule<token<1> >( "Group" ) ); mySyntax.add_rule( rule<token<2> >( "Separator" ) ); mySyntax.add_rule( rule<token<3> >( "Switch" ) ); // Lots more symbols for the rest of VRML... mySyntax.add_rule( rule<token_float<69> >( "([-+]?((([0-9]+[.]?)|([0-9]*[.][0-9]+))([eE][-+]?[0-9]+)?))" ) ); mySyntax.add_rule( rule<token_string<70> >( "#[^\\n\\r]*[\\n\\r]" ) ); mySyntax.add_rule( rule<token_string<71> >( "\"[^\"\\n\\r]*[\"\\n\\r]" ) ); mySyntax.add_rule( rule<token_string<72> >( "[a-zA-Z_][a-zA-Z0-9_]*" ) ); mySyntax.add_rule( rule<token<73> >( "[ \\t\\n\\r]+" ) ); lexer<> myLexer( mySyntax ); std::ifstream fsp( "test.vrml" ); myLexer.set_source( fsp ); // Also it will have a char iterator interface lexer<>::token_iterator iter = myLexer.begin(); for ( ; iter != myLexer.end(); ++iter ) // Do something with the tokens... ; Essentially you create a syntax to which you add rules. These rules explain the token that will be created by the rule, and a regular expression to be matched. Special versions of tokens exist that will automatically generate a float, int or string from the matched expression. You then construct a lexer off the syntax, set a suitable source input, and then iterate over the tokens. The tokens are polymorphic so you can easily access the type and data, they are statically typed so you can use types to match the tokens, or you can dispatch tokens to functions through a visitor framework. You can also use dynamically typed tokens as well if you want - although you lose much of the benefit of strong typing. I especially like the idea of using visitors to distinguish between tokens. This means that in my example of a VRML file, the long float lists can be handled with a visitor that contains visit functions for a float token, a comma token (note that in vector lists in VRML the elements in a 3D vector are separated by whitespace, and the 3D vectors themselves separated by commas) and a list termination token (eg a "]"). The visitor would then automatically store the floats in a suitable data structure - like a list of 3D vectors. Dave

3 2

Interest in a fast lexical analyser compatible
by Dave Handley 28 Dec '04

28 Dec '04

Dick Hadsell wrote: <snip> >disappointed, because I was hoping to use Spirit or something like it, >to give me some independence from Lex/Yacc's dictatorial control of the >input source. >Your project sounds like it would solve the worst of the problems I have >in trying to move to Spirit. The current API we are working with templatises the input so that in theory it will work with any character like input, in much the same way as std::basic_string<>. We are still working on getting the DFA to work generically, rather than just explicitly with char and wchar_t, but I think we should have some success. At present the lexer is strongly typed from this character type in the same way as std::basic_string<> but I don't necessarily see that as a problem. >I broke up the problem into 3 steps. In the first phase the program >uses a Spirit grammar to generate a list of tokens with info similar to <snip> Depending on the type of grammar I think you should easily achieve a 6x or greater performance boost. If the input has long repetitive sections, you could probably further optimise the stage so that the lexer does most of the work - for example if you have long lists of numbers or similar. I'm not sure how well this would work with Spirit until I try it, but it should be possible to switch control part way through a parse to a very quick and efficient parser that just throws the tokens at a visitor until a particular section is finished. I'm sure this could probably be done by writing a new parser type in Spirit. The idea would be to process long lists of numbers or strings or similar, where those lists have a clearly defined end token. Dave Handley

3 2

Interest in a fast lexical analyser compatible with Spirit
by Dave Handley 28 Dec '04

28 Dec '04

Christopher Diggins wrote: >Would you please post the grammar productions you used for Spirit? Would >you >consider testing the yard parser as well, http://yard-parser.sf.net ? Are >there other advantages of your tool over Flex, other than providing a nice >interface to Spirit? The grammar for Spirit was (in a slightly cut down form): keyword = str_p( "Group" ) | str_p( "Separator" ) | //etc.; comment = lexeme_d[ ch_p( '#' ) >> * ( ~chset_p( "\n\r" ) ) >> chset_p( "\n\r" ) ]; stringLiteral = lexeme_d[ ch_p( '\"' ) >> * ( ~chset_p( "\"\n\r" ) ) >> chset_p( "\"\n\r" ) ]; word = lexeme_d[ ( alpha_p | ch_p( '_' ) ) >> * ( alnum_p | ch_p( '_' ) ) ]; floatNum = real_p; vrml = *( keyword | comment | stringLiteral | word | floatNum ); I've cut down the keywords because there are over 60 of them. I would be interested to know if there was any obvious ways to optimise the Spirit parser. Regarding writing a comparitive test for YARD, I will certainly consider it, although I have no experience of using YARD. I've written quite a lot of flex/bison parsers, written a number of Spirit based parsers, and hand coded a number in my time, but I would need to spend a little time learning the YARD framework first. There are a number of advantages over Flex (including a nice interface to Spirit). The key ones are: 1) An extensible object oriented approach to writing the entire system. This can be very useful if you want to handle something like say the parsing of a filename in the lexer. You can simply write a new token type that will split the incoming filename into path, extension, name, etc. This can massively simplify the production of a final parser - allowing you to deal with grammar issues at that stage. 2) DFAs can be created at run-time, or eventually compile-time. 3) The code is considerably less obfuscated than the code produced by flex. Don't get me wrong, I like flex a lot, but the pre-processor directives, and look-up tables in the generated code are pretty unreadable IMHO. Dave Handley

4 3

Interest in a fast lexical analyser compatible with Spirit
by Dave Handley 28 Dec '04

28 Dec '04

A colleague and I are working on a fast lexical analyser designed to work alongside Spirit. We currently have some working prototypes, and as such I thought that I would gauge interest in this library. We plan to produce a DFA based lexical analyser that provides output as a set of iterable polymorphic flyweighted tokens. These could then be provided as input to Spirit instead of character iterators (albeit with the addition of a token_p parser in Spirit). The objectives of our project are as follows: 1) As fast as lex/flex. 2) Simple to use 3) Rules to generate tokens can be provided both statically and dynamically. Static definition would be through an offline pre-process stage much like lex. 4) Easy to interface with Spirit. I understand from Joel's posts below (both on this group and on codeproject) that he is already planning some work to this end. My understanding of expression template based lexers is that they can outperform DFAs under some circumstances, but with other situations DFAs can provide the best performance. I am therefore recommending this in addition to Joel and Eric's work rather than instead of it. Finally, as a taster of the work we are doing, I have performed some rather unscientific and arbitrary performance tests on a 20Mb VRML 1.0 file. The performance of flex, Spirit, and our new library are as follows: flex: < 1 second new library: about 1 second Spirit: 40-50 seconds. These results were produced with no semantic actions to slow down any of the 3 solutions. They are also not conclusive, since we are almost an order of magnitude slower on an Athlon 64 bit machine at present. Dave Handley Relevant posts: Joel de Guzman wrote: >> I guess if you use string literals you'll have this >> problem no matter what, though I'm not sure the use of string literals >> should be a foregone conclusion. If you consider parser generators like >> YACC, they cleanly separate lexical analysis from parsing and the parser >> itself is never working with string literals. > >Again, true. However, one of the original intent of Spirit is to >be usable in micro parsing tasks. When parsing small grammars >like, say, a complex number, you do not want to write a separate >front end lexer. > >Anyway, I do plan to write an optional but separate lexer >sometime. Joel de Guzman wrote: > As for the lexer, I am working with Eric Niebler, author of Greta and >xpressive (google regular expression c++) with the hopes of developing >a powerful expresion template based lexer front end.

6 5

RE: [boost] Interest in a fast lexical analyser compatible
by Dave Handley 28 Dec '04

28 Dec '04

Hartmut Kaiser wrote: >I'm definitely interested to have a look at your library. Besides my >general >interest in Spirit I'd like to try it out as an alternative lexing >component >for Wave. I'm pretty sure this should is interesting for you as well, >because there are implemented already two different lexers, which gives a >good opportunity to compare them in a real environment. I have to confess to not knowing much about Wave, but I would be willing to look in more detail at using this library for Wave. Once we have our first stable version of the software we will be happy to let you have a look at it in more detail. I expect this to happen within the next month or so. >As for a dynamic DFA based lexer Wave already uses the Spirit based SLEX, >but a static DFA based solution is very interesting to look at. Is the DFA >generated at compile time? By static and dynamic, I am meaning compile-time and run-time. The system is designed to generate the DFA at run-time. But we are discussing a method at the moment whereby the same code base could be used to generate code-stubs for compile-time DFA creation. I am keen to support both. >We definitely should try the new upcoming Spirit-2 code base as well, since >it should be a lot faster then the current version. Is it possible to have >a >look at your test code as well? This way we could try to make a comparision >as soon as the Spirit-2 codebase evolves. I would be quite keen to see how Spirit-2 performs on similar tests. If the interest is there, then we will quite happily post the code into the boost yahoo group once we have completed a bit of tidying up in the New Year. We need to put some effort into writing more detailed test cases. At present, we are only directly comparing lexical analysis, and have not looked at the performance of the interface in real detail. We have a desire to properly test the system with a complete parse. To do this I think there are a number of useful test cases: 1) Flex and bison/lex and yacc. 2) Spirit - without any assistance from any lexer. 3) Spirit 2 once available. 4) Our library (called lextl at present) with yacc/bison. 5) Lextl with Spirit. 6) Lextl with Spirit 2. Case 1 would be the control - and the target performance for other cases to achieve. I am confident that case 4 should achieve the same speed, also I think there is a good likelihood of cases 5 and 6 achieving the same speed. The key question is whether 5 outperforms 2 and 6 outperforms 3. If this is the case, then I think we have a viable library. My current test file which is a 20Mb VRML1.0 file reduces to about 2e6 tokens when whitespace is stripped and the file is lexed. By using flyweighted tokens, we are hoping to reduce the overhead for Spirit to parse tokens instead of characters to a minimum, so the reduction from 2e7 to 2e6 input entities should give an order of magnitude speed increase to Spirit. At least that is my hope :-) Dave Handley

3 2

Interest in a fast lexical analyser compatible with Spirit
by Dave Handley 28 Dec '04

28 Dec '04

Christopher Diggins wrote: >Hi David, I will write the grammar for you. I am just finishing up version >2.0 of YARD, and then I will post the equivalent YARD grammar. Thanks. Hi Christopher - that would be excellent. I've had a quick look at YARD now, and consider it perfectly feasible to get our library to work with YARD as well. Essentially, all we require is a generic (ie templatised) interface that expects any form of forward iterator. It seems on first inspection that this is exactly what your library requires. Dave Handley

1 0

Re: [jamboost] Re: [boost] Re: #include <...> not in list of dependants
by Roland Schwarz 27 Dec '04

27 Dec '04

Rene Rivera wrote: > > Looking into this.. I don't see a problem on my setup. Specifically > Oops the thread got totally messed up. It is my fault I think, because I did not manage to subscribe to the correct list at the beginning. I crossposted to boost and jamboost. Sorry for this. This is the _last_ crosspost! Please every follow up to go to jamboost only! Back to the original problem: Steps to repduce the described behaviour: 1) Update boost tree from CVS 2) issuing SET TOOLS=vc-7_1 bjam --prefix=C: install 2) Editing the file boost/boost/thread/detail/config.hpp 3) issuing SET TOOLS=vc-7_1 bjam --prefix=C: install does not rebuild the threading library, altough I expected it to. Am I wrong with this expectation? Roland

1 0

[MPL] zip_view documentation bug?
by David Greene 27 Dec '04

27 Dec '04

I think I've found either a bug in zip_view with gcc or a bug in the zip_view docs. If I try to use a zip_view<> in which the sequences are not all of the same length, the compiler goes into "infinite" recursion instantiating at<> templates until it hits the instantiation limit. According to the MPL reference manual, the size<> of the sequence should be the minimum of the size<>es of all the participating sequences. At first I thought this was only a problem when one of the sequences is empty but I was able to reproduce it with populated sequences of varying length. Just two is enough to expose it. This is a problem with gcc 3.2, 3.3 and 3.4. I can work around the problem by using iterator_range<> and copy<> to truncate the sequences before feeding them to zip_view<>. What's supposed to happen in this case? -Dave

2 1

RE: [boost][regex] Bug in regex 1.32 ?
by Jan Hermelink 27 Dec '04

27 Dec '04

>> boost::regex re("x", boost::regex_constants::icase); > That should be: > boost::regex re("x", boost::regex::perl|boost::regex_constants::icase); Or boost::regex re("x", boost::regex::normal | boost::regex::icase) Ok, I have overlooked this statement in the documentation: "A valid value of type syntax_option_type will always have exactly one of the elements normal, basic, extended, awk, grep, egrep, sed or perl set". My fault - Thank you! Jan

1 0

Properties and boost::any / boost::program_options
by Dirk Gregorius 27 Dec '04

27 Dec '04

Dear group, I have a text file which defines several properties. For example: // Properties.txt "Position" "1.0 2.0 3.0" "Color" "1.0 0.0 0.0" "Intensity" "500" I wonder if it is possible to use the Property structure like defined in the boost::any example: // Property from boost::any help file struct Property { Property(); Property( const char* ) // Convinence c'tor for easy use with std::find Property( const std::string&, const boost::any& ); std::string Key; boost::any Value; } typedef std::vector<Property> Properties_t; // Operators for sorting and finding bool operator==( const Property& lhs, const Property& rhs ) { return lhs. Key == rhs.Key; } bool operator<( const Property& lhs, const Property& rhs ) { return lhs. Key < rhs.Key; } I want to do two things. First I like to parse through my text file and simply push_back() the parsed properties as pairs of strings: // The parse function - basically void ParseProperties( std::ifstream& is, Properties_t& properties ) { std::string key, value; while( is ) { is >> key >> value; properties.push_back( Property( key, value ) ); } std::sort( properties.begin(), properties.end() ); } Then later I would like to be able to process the properties like this: // Process the parsed properties... void ProcessProperties( const Properties_t& properties ) { typedef Properties_t::iterator PIter; PIter pos = find( properties.begin(), properties.end(), "Position" ); // Here we need the const char* c'tor if ( pos != properties.end() ) Vector3 position = any_cast<Vector3>( pos->Value ); } 1. Is it possible to use boost::any in this way? If yes - where and how do I have to implement the conversions? 2. If this is not possible could I use boost::programm_options instead for the value type? How and where do I have to implement the conversion here. 3. Would you suggest to define a simple Value class like this when I can not use any boost classes: class Value { public: Value(); Value( const std::string& ) template<typename T> T As(void ) const; private: std::string m_Value; } Are there any rules how to implement such a class? I have very, very few experience with template code. So do I need a traits class to look up the conversions in the As() function or how should a simple class like this be implemented in order to be nicely extendable by a client? I wish you all a nice christmas and a happy new year! Regards, -Dirk

2 1

[boost][regex] Bug in regex 1.32 ?
by Jan Hermelink 27 Dec '04

27 Dec '04

Hello, I think there is a bug in Boost.Regex 1.32.0 with regex_constants::icase. std::string s("xx"); boost::regex re("x", boost::regex_constants::icase); boost::regex_replace(s, re, "a"); The replace yields "axa"; should be "aa" I think. Without the icase flag the replace yields "aa". Thank you Jan

2 1

Re: [boost] Re: thoughts on a GUI component library
by brangdon＠cix.compulink.co.uk 27 Dec '04

27 Dec '04

In-Reply-To: <cqcjii$hs0$1(a)sea.gmane.org> dave(a)boost-consulting.com (David Abrahams) wrote (abridged): > That's okay for simple things, but for any serious printing work you > need to know you're talking to a printer. For example, you may need to > embed postscript in your output stream. Not all printers support postscript, so you'd need to know not just that you were printing, but what kind of printer it was. Although there probably needs to be some kind of abstraction-busting pass-through mechanism, I see it as the exception rather than the rule. Another issue is transparency. My employer's drawing framework supports semi-transparency in the form of an alpha channel. This is used by us for things like anti-aliasing, and by users for subtle colour blends. Some printers can support transparency directly and some cannot. We found we needed a relatively elaborate architecture to hide that detail from application drawing code. I'm a bit concerned about the scope of this boost project. Our drawing framework is pretty huge. -- Dave Harris, Nottingham, UK

13 44

automatic garbage collecting algorithm and its implementation in c++
by yilin yang 27 Dec '04

27 Dec '04

hi, everybody i've just designed and coded a garbage collecting infrastructure named u_canforget. it was uploaded onto the news group at boost.yahoo. the main idea is that we count the reference to track the life-cycle of the given object. as the new operator was overloaded to statisfy this requirement. the other important concept was the reference object. with the syntax represented by the redefined macros, the codes may be written in a very simple way with relationships automatically constructed, objects are freed without delete operater can be realized as well. i would keep tracking this thread, if there are any thought about this algorithm. thx for your advices. your's yilin. --------------------------------- Do You Yahoo!? 150万曲MP3疯狂搜，带您闯入音乐殿堂美女明星应有尽有，搜遍美图、艳图和酷图 1G就是1000兆，雅虎电邮自助扩容！

1 0

Re: [boost] [gui] Shoulders of Giants
by brangdon＠cix.compulink.co.uk 26 Dec '04

26 Dec '04

In-Reply-To: <20041222211651.GD2616(a)maribor.izzy.net> alan-boost(a)engrm.com (Alan Gutierrez) wrote (abridged): > I'd like to take some time and study what has gone before. > [...] > I've advocated the use of a CSS layout component to render > the View-Controller aspect [...] Are you aware of the variations on MVC, in particular Model-View-Presenter? MVP has been so successful in Smalltalk that I consider MVC obsolete. Taligent's whitepaper is here: http://www.carfield.com.hk/document/software_design/MVP.pdf and there is more material at Object Art's Smalltalk site: http://www.object-arts.com/EducationCentre/Overviews/ModelViewPresenter.ht m -- Dave Harris, Nottingham, UK

3 3

spirit fileiterator always uses std_iterator on posix
by dieMumiee＠gmx.de 26 Dec '04

26 Dec '04

Hi, Found that code in boost/spirit/iterator/file_iterator.hpp template < typename CharT = char, typename BaseIterator = #ifndef BOOST_SPIRIT_FILEITERATOR_WINDOWS fileiter_impl::std_file_iterator<CharT> #else fileiter_impl::mmap_file_iterator<CharT> #endif > class file_iterator; BOOST_SPIRIT_FILEITERATOR_POSIX which is defined a few lines above is never used. Are there any issues with the posix-mmap code? It "worked" here, I modified the code to : template < typename CharT = char, typename BaseIterator = #if defined(BOOST_SPIRIT_FILEITERATOR_WINDOWS) || defined(BOOST_SPIRIT_FILEITERATOR_POSIX) fileiter_impl::mmap_file_iterator<CharT> #else fileiter_impl::std_file_iterator<CharT> #endif > class file_iterator; Regards, Andreas Pokorny -- +++ Sparen Sie mit GMX DSL +++ http://www.gmx.net/de/go/dsl AKTION f�r Wechsler: DSL-Tarife ab 3,99 EUR/Monat + Startguthaben

2 1

[Boost-bugs] [ boost-Support Requests-1091153 ] ublas extendability 2
by SourceForge.net 25 Dec '04

25 Dec '04

Support Requests item #1091153, was opened at 2004-12-25 10:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=207586&aid=1091153&group_… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: ublas extendability 2 Initial Comment: Hi. I want to multiplicate ublas matrices with custom (non- ublas) vectors. Is there some possibility ? Please see attached files and compilation messages. It seems that the job can be done with more knowledge of ublas tunning. Thank you. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=207586&aid=1091153&group_… ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Boost-bugs mailing list Boost-bugs(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/boost-bugs

1 0