[gui] GUI Taxonomy
The OS Direction I'm looking over wxWindows. I'd like to avoid this sort of thing, yet another wrapper library. They seem to have done as Andy Little and Reece Dunn have cautioned against, and started with a particular OS as a model, rather than starting with a model. As I look closer, or stand back and squint, the object heirarchy appears to be a usual-suspects collection of GUI objects, wxPen, wxWindow, wxFocusEvent, et al. Oh, and wxObject. *sigh* Perhaps they didn't start with an particular OS, but they certialy started at the OS and worked their way up, rather than starting at application and working their way back. It is vital to address the design of the primitives. It is a mistake to think of a GUI as merely a collection of primitives. After thirty years, the GUI has begun to develop a set of canonical forms. Any new GUI library should recognize these forms, and cater to the needs of the developer who must implement them. A Taxonomy - The Dialog A collection of widgets, gathered together in a window that usually of a fixed size, usually presented to the user in a modal state. Hence the term dialog, indicating a back and forth between the user and the application. Most dialog systems include a means to specify the layout of a dialog from a resource file. Dialog applications include setting preferences, choosing files, notification of errors, specification of search criteria. - The Grid A collection of widgets, arranged in scrollable rows and columns, in a resizable window, where widgets can be added or removed as the inserts or deletes data. Newer windowing libraries such as Delphi's VCL or Swing provide a grid library, for a time it seemed to be a defining characteristic of a new entry into the GUI arena. Grid heavy applications include spreadsheets, databases, and accounting applications. - The Document Text with a flow direction, broken by whitespace and hyphenation, along with imagry, and object, grouped by blocks, gathered into columns and tables. Most recent application frameworks provide support for the document model by making a component of a web browser, or providing a rich text edit control. Document applications include web browsering, word processing, and reporting. - The Canvas A blank surface onto which shapes are input, usually through manipulation of the mouse, as in drag and drop, and grabbing handles, or else output but still responsive to mouse clicks. There are many shape libraries available, but this is generally the sort of thing that gets written time and again. Canvas applications include illustration, diagraming, charting. - An Example A contemporary personal finance program uses The Dialog to set preferences and to open files, The Grid to display a check register for editing, The Document to display a nicely formatted balance sheet and a hyperlinked start page, and The Canvas to render a pie chart of spending by category. What did I miss? The Application Direction I'm representing this taxonomy since in discussion I'm finding that there are different needs for different problems, and we should know which problems we are addressing. Hit-testing a widget in a dialog or a grid ought to be a trivial operation, hit-testing a line of text in a document somewhat more difficult, and hit-testing a partially exposed polygon on a canvas a real challenge. One should not pay for what one does not need. Validation is a concern for dialogs and grids, but it isn't a concern of a bar chart. Even if when developing a full-blown drawing application, one should not have to pick up the pen and draw out each tool-tip, or call out, when those could be rendered as little documents. -- Alan Gutierrez - alan@engrm.com
Alan Gutierrez wrote:
It seems to be designed around Win32 and OS/2. Likewise, QT uses a special preprocessing stage to handle signals/slots, which should also be avoided. I aim to write a comprehensive comparison as part of my library docs.
I think dialog is the incorrect abstraction. In Windows, a dialog is a pre-built frame that is identified by a resource ID. It constructs the components based on the layout design in that resource. In MacOS, you have a NIB file that describes a frame. In general, you have the concept of a form: a form has a structure specified by a resource file via an ID. This is mapped to the OS-specific concept. I have not yet added support for forms in my Boost.GUI library, but this is something I intend on adding.
- The Grid
A grid is an example of a layout manager. Generic support should be added for layout managers and allow you to add managers such as grid-layouts. Another layout is a flow layout, for example, adding command buttons and placing them at the bottom-left of the frame along the x-axis. I have a working implementation in the latest code and intend to merge the new code into the sandbox.
What about graphics applications? You can have a graphical document/view model with lines, polygons, ellipses, etc. Also, what about PDF documents? It should be possible to support the document/view architecture, but not exclusively like MFC does.
- The Canvas Canvas applications include illustration, diagraming, charting.
I think the canvas abstraction should be used in a more general sense as part of the graphics part of the library. It should have a low level (pens, brushes, fonts, or whatever abstraction is chosen) with high level objects (e.g. vector graphics objects such as ellipses). Doing this would allow you to write turtle graphics applications with the low level objects and an SVG renderer using the high level objects.
What about a table? For example the JTable/TableModel in Java Swing. I personally like this and have a version written using the sandbox version of my library in Windows. I intend to port it over to the new version and add it into the available components. You could use the table in the finance program to view the financial transactions and have a flow layout for the command buttons at the bottom (new transaction, view/edit transaction, save transactions, load transactions). There are other controls/designs such as tabs, e.g. editing multiple documents. What about property sheets (form collection)? How do you switch between the forms in a form collection? Do you use tabs (e.g. property sheets), lists (e.g. Mozilla Thunderbird options), trees? Regards, Reece
* Reece Dunn <msclrhd@hotmail.com> [2004-12-24 06:51]:
I have not yet added support for forms in my Boost.GUI library, but this is something I intend on adding.
I'm not talking about a particular implementation, of corse, but a UI concept, which let's call The Form, but I didn't want to get it confused with XHTML forms, XForms, or PDF forms, which are more in line with The Document below. So, I called it The Dialog, but you're right, there is much more to it, there are applications like Visual Basic that are all about creating Forms, editing Forms, and validating Forms. Still, I think the constraints on this sort of thing are generally that the developer is choosing and arranging a fixed number of controls. If there is a variable number of controls, those are placed in a special form control like a tree, grid, or drop down control. I'm imagining Visual Basic, Power Builder, and Delphi, and thinking of the layout managers in Swing.
- The Grid
A grid is more than a layout manager. It has to scroll. I has to provide for selection of cells, which a layout manager does not have to do. It has to be able to add controls, perhaps as a new row of controls, perhaps as a new cell in a spreadsheet. When I think of a grid application, I'm thinking of the problem address by JTable/TableModel, which I'll get back to below..
It should be possible to support the document/view architecture, but not exclusively like MFC does.
I'm not using the term document in terms of document/view, and I'm not talking about graphics documents. In this taxonomy, graphics applications are falling under the realm of a canvas-oriented application. I'm talking about human language document oriented applications like web browsers, word processors, and reporting. This component of an application is exemplified by the browser controls that are now a *staple* of GUI component libraries. Again, I am not talking about the document/view architecture. I am talking about XHTML + CSS, XForms, PDF Forms, rich text editing, hyperlinking, man pages, etc.
- The Canvas Canvas applications include illustration, diagraming, charting.
Doing this would allow you to write turtle graphics applications with the low level objects and an SVG renderer using the high level objects.
I should have noted, I'm not trying to suggest an architecture. I'm talking about the different sorts of UIs out there. There is obviously going to be a an object that reprensts a drawing surface, but I'm not addressing that here. Again, I'm simply trying to layout the different types of application. When I open up Visio, I see this big, blank Canvas, along with a palette of graphic shapes. This is very different application than when I open up Excel, which is a big Grid, or open up Firefox, which is a Document, or open up System Preferences, which is a bunch of Forms. I agree that the design that is being sorted out, for points, and shapes, is part of the graphics library, and is vital, and that the right design is important for simplifying the creatation Canvas like application. Note that graphics might not really come into play, say on a character console, where one might still want to mode The Document (like man pages, or lynx) or The Form. It would still be nice to be able to use the idioms UI library to assemble a console based Form UI. Also, most UI libraries provide special support for Forms, as in dialog resources, or layout managers. Many provide a Grid library, that let's a developer compose a table using widgets. The Document application becomes simplier, as developers choose to embed KHTML, Mozilla, or IE, or use one of those browsers as a platform. The Canvas is usually left as an exercise for the application developer, but with generic programming, I think that C++ can do better.
I think a table is an example of The Grid. I'd would called The Grid, The Table, but I also think a spread sheet is The Grid, and I don't think of a spread sheet as a table. I think the JTable/TableModel is a very valuable, and effective abstraction. The Swing developers reocognized that there are great many business applications that display information in tables, and provided a good, general-purpose table library. It is more than a layout control, because it needs to respond to scrolling and selection, the columns can be reordered and resized, rows and columns hidden or deleted. It is a very speciliazed concept. I think a new GUI library should good general-purpose document library, which is why I'm putting forward this taxonomy. I'd like to make sure that a GUI library is designed to address the expectations of application developers, and the lack of a rich text library is going to be felt.
Yeah, we are saying the same thing, then. I chose a different name. So, to clarify, I'm not saying, oh, oh, let's have a grid control! Rather, that in the world of 2d UI, a Grid, or Table, is a canonical representation of tabular data, and that developers are going expect a certian set of facilities to support their development of tabluar, or grid UIs.
There are other controls/designs such as tabs, e.g. editing multiple documents.
Mozilla Thunderbird options are implemented using XUL, and the trees in Mozilla Thuderbirds are implemented using The Document (XML + CSS) so when I look at the Advanced pane I see what is very obviously The Document used to arrange controls with collapable paragraphs. When I look at the General pane I see The Form with controls organized into groups. Otherwise, tabs, SDI, MDI, toolbars, menus, docking, drag and drop, the clipboard, might be outside of this taxonomy, since they are framing around an applicatoin, and not an application in themselves. All of the above canoncial forms use windows, and will most likely use menus and cut and paste. The Canvas in this taxonomy, is not the library of 2d privimitives, but a kind of application, like Illustrator, Rational Rose, AudoCad. Providing window dressing that allows a developer to organize their Forms, Grids, Documents, or Canvases into tabs, or have a Form, Grid, Document, and Canvas view of relational database (record entry, tabular entry, reports, and ER diagrams, say) is on the other end of the spectrum from the 2d library of primitives. Here we are talking about windowing, and MV[CP] architectures. In fact, the 2d graphics primitives, on the one hand, and thier framing on the other, are both the areas of integration, where the GUI library abstraction ends, and the interation with the underlying operating system begins. The Revised Taxonomy The Form, The Grid, The Document, & The Canvas. -- Alan Gutierrez - alan@engrm.com
Alan Gutierrez wrote:
The Revised Taxonomy
The Form, The Grid, The Document, & The Canvas.
I think that trying to work out the taxonomy of user interfaces is the wrong way to go. In general, it is not clear how these break down. For example, an application will usually have a main interface frame, such as the web browser. In this case the taxonomy is the Document to use your terminology. The application may also provide a set of options that the user can configure, which is The Form. Also, consider an e-mail manager like Thunderbird. It is a mixed taxonomy application. It has a left pane consisting of the available folders (a tree component), a list of the messages in the folder (a Grid) and the contents of the message (a Document). Then what about games? Media players like WinAMP? These do not fit into one of the above Taxonomy items. The above fit into specialized frame types (the Form), components (the Grid) and content renderers (the Document and Canvas). Regards, Reece
* Reece Dunn <msclrhd@hotmail.com> [2004-12-25 07:03]:
I think that trying to work out the taxonomy of user interfaces is the wrong way to go. In general, it is not clear how these break down.
Really?
Perfect break down.
Yup. Exactly. The tree, it's a common widget, and panes are part of windowing. In the case of Thuderbird, the tree is implemented as an XML document, with paragraphs that are indented, that expand and collapse. (Thunderbird is an amazing example of what you can do with a component like an XML + CSS renderer. All of those controls are, in fact XML documents.)
Then what about games? Media players like WinAMP? These do not fit into one of the above Taxonomy items.
Games don't fit, granted. I'd be hard pressed to come up with a generic basis for Photoshop. WinAMP is a skinnable form, and if all you are is WinAMP you might want to draw your own controls on the drawing surface, fine, but you could implement an WinAMP UI in XUL easily.
The above fit into specialized frame types (the Form), components (the Grid) and content renderers (the Document and Canvas).
I'd say that the framing, tabs, splitters, and the like, are part of content area management. I'm intereseted in providing application developers with building blocks for the content area, and more than wrappers around common controls and graphics primatives. These problems Forms, Grids, Documents, and Canvas are what most applications are made of. This is why web development is so popular, because it focuses not on spliting windows, but filling them. Firefox provides Forms, Grids (tables), Documents, all scriptable, and if you add Flash, you have a generic Canvas application. It think it is important to recognize these content building blocks. -- Alan Gutierrez - alan@engrm.com
* Alan Gutierrez <alan-boost@engrm.com> [2004-12-25 07:50]:
Maybe Taxonomy was too portentious. I don't want to get bogged down defending a "Taxonomy", when I'm trying to learn bjam and Boost.Build in the other window. Content building blocks, I'd like to have 'em. I'd like to talk about them. In addition to the dialog boxes, the tabs and splitters, the graphics primititives, can a Boost.GUI provide some robust strategies for the client area of the application? This is really the discussion like like to have. Widgets + layouts hit a ceiling pretty quick. I'm really impressed with Thuderbird and Firebird, where they've used their XML + CSS renderer, the content renderer, to do pretty much everything else, wigets, splitters, dialgos. I think that shows a lot of forsight. -- Alan Gutierrez - alan@engrm.com
Alan Gutierrez wrote:
Let me clarify. I view the grid and table as a specific data-bound components/widgets that makes requests for data, i.e. row/column information and cell queries. This will fit in with other advanced UI components such as the tree. The way that I am approaching components is to define a specific data+event relationship, for example: * textfield = std::string text; event::ondatachanged(); * push-button = const std::string title; event::onpressed(); * button-group = event::onselectionchanged( long sel ); * calendar = boost::gregorian::date date; event::ondatachanged(); This allows you to bind specific actions to a data component and to query its content. I view a form as a specific type of frame. A frame is a component that supports a frame and will contain other components. Examples include: * main-frame - the top-level application frame with a close button, menu, etc. * form - a frame that has a layout specified by an external resource located by an ID, e.g. Dialogs in Windows and NIBs in Mac. * popup - a popup frame that may not have border decoration. This is used, for example, to host controls beneath a button when the button is pressed (such as a calendar). The Document and Canvas to me are specific renderers that are used to render content on a frames usable area via a graphics canvas. We should not be restrictive on what type of renderers we allow, because you will need a different DOM if you are rendering HTML content (using the W3C DOM), a text editor (using a custom text DOM), a PDF document (using the PDF specification), an SVG application (using either a custom representation, an array of high-level graphics objects or the W3C SVG DOM). What I am trying to say is: provide the framework for rendering content and the interaction between the frame and the graphics canvas, then let the application writer construct the document model that they need.
I am not suggesting a low-level wrapping around the various widgets. What I am suggesting is that we focus on the type of *data* that a widget supplies and support that in the widget implementation. This base data (e.g. std::string in a textfield) can be converted to the required type (e.g. a long) by the programmer when they write the application. If you require data binding, it should be easy to write this on top of the GUI framework, but data binding should be seen as an extension.
Sure, but isn't this what components/widgets are? I know that they don't fit in with providing HTML or PDF content, but this is application-specific data rendering. There are commonly supported content renderers (simple text and rich text editors) that should be provided by the GUI library, but then you get to the issue of whether this is cross-platform and do you also supply HTML+CSS content?
This is really about content rendering models, document/view architecture, MVP architecture, etc.
There are a few issues with providing an XML+CSS layout renderer like Thunderbird/Firefox: * The implementation would require an XML and CSS parser as well as the ability to draw the content to the screen (to support the CSS side of the proposal) - this would make the resulting code base exceptionally large; * Thunderbird/Firefox use this approach because of the nature of the application - they already have a very good XML+CSS parser and content renderer as part of the web browser; * Do we implement the renderer ourselves (expensive in terms of code size, effort and tracking down bugs) or use an external renderer like Geko (Mozilla) where we have licensing issues; * How do you provide the UI, i.e. the event binding and event flow? * What about people who want/need/require native interoperability? * How do you use native controls to get a native L&F? * What about other issues inbuilt to an operating system? * How do you interact with screen readers for blind users? In my opinion, the Boost.GUI library should be as lean as possible, but provide the framework that allows for XML+CSS applications, skinned applications, etc. Regards, Reece
* Reece Dunn <msclrhd@hotmail.com> [2004-12-26 05:57]:
Let me clarify.
I've thought about it some more, and FGDC is a set of rendering strategies. That is my taxonomy.
This allows you to bind specific actions to a data component and to query its content.
A rendering and event strategy.
I view a form as a specific type of frame. A frame is a component that supports a frame and will contain other components. Examples include:
A rendering and event strategy.
I see the Frame as part of the windowing system. It organized content areas. It might have tabs and, spliters. It might be very spare as on PalmOS. It is the work that is supposed to be done for you by the OS or GUI library. I view a Form as a content rendering and event strategy. Rendering Strategy: Simplyfy rendering and event routing by arranging a fixed set of user input "components" onto a client area divided into rectangular regions. The division by region can be done through a composition of layout strategies, or the controls can be positioned at fixed locations on a client area of fixed size. Rendering is performed by rendering borderes and asking each "component" to render itself. Event Strategy: All "components" are on the same z-axis. All "components" are rectangular. Mouse events simplified by rectangular shape of regions and controls. Each "component" responds to OS events individually through some well known behavior. Applications: Dozens of Form Designers, Oracle Forms, Visual Basic, Power Builder, PalmOS. UI Usages: Application preferences, file selection, database record entry, any place where a set of not terribly variable properties are edited. Can contian more complicated rederers. Minimal Library Needs: Axis-aligned boxes. "Component" impelementations. *or* Handles to OS generated resources. I view the Grid as rendering and event strategy. Rendering Strategy: Simplyfy rendering and event routing by arranging a fixed set of user input "components" onto a client area divided into rows and columns. User is able to resize rows and columns. Overflowing text within cells is clipped for overlays the adajcent cells. As information is added to a document, it is rendered as new rows in the grid. Render text with in "cell" or to cell has input focus, delegate rendering to an input "component". Rendering of cells can be buffered. Insertion of cells can be rendered from the buffered images. Cells off screen need not be rendered. Cell off screen can be rendered in advnace of their scrolling into view. Event Strategy: All components and cells on same z-axis. Mouse events mapped to a row and column. A cell responds to an event by displaying an input "component". Additionally a grid provides logic for the selection of ranges of cells. Applications: Excel, Access, SPSS, SAS. UI Usages: Database records, spreadsheets, check registers, e-mail inbox listings, or any other form of tabular data. Grouping of Forms, Documents, or Canvases in repition to create a card listing. Implementation of tables within documents. Minimal Library Needs: Axis-aligned boxes. Fonts. Components. Scrolling. Forms are one way to render content. It is a very simple rendering strategy. Constraints are placed on the capabilities of the UI. The Grid is another content rendering strategy, that was designed to present structured content that is repetitive. Neither of these strategies are more or less worthy of implementation as framework that a Document or a Canvas.
The Document and Canvas to me are specific renderers that are used to render content on a frames usable area via a graphics canvas.
As are The Form and The Grid.
My point exactly. These are all documents. Documents are omnipresent. (I've never spoken of retrictions.) The source of the data is immaterial. The Grid going to come from? The DOM is an abstraction that does not need to be bound to the renderer, any more than the a grid needs to be bound to a flat file.
an SVG application (using either a custom representation, an array of high-level graphics objects or the W3C SVG DOM).
An obvious application of vector graphics. Which is another point to consider. Is a form, grid or a document best defeined in terms of an advance vector grpahics library when most of them can be rendered as axis-aligned boxes?
What I'm trying to explore is a set of core rendering strategies, and their needs, so that might provide an implementation that hits the 80 of 20, or faciliate its implementation. Not throw up our hands and say, that's the application arena, we cannot forsee.
If you require data binding, it should be easy to write this on top of the GUI framework, but data binding should be seen as an extension.
Agreed.
No. I don't see document rendering as widgerty. When it's done as a widget it falls flat. Widget is not the proper classification of the document. It is a rendering strategy. Swing recognised with with JTable/TableModel. That is wasn't just another widget for the widget stable, but a first class citizen. Something that requried special design. The the frames and the widgets in Swing were deisgned to accomodate use within the JTable/TableModel. It isn't a widget. It didn't just happen. JTable/TableModel was not possible in Java's AWT. It is an excellent architecture, and one worth repeating. Let's look now at the rendering strategies used and design the framework to accomdate rendering strategies? Why not make flowing text a similarly first class rendering engine? In this way the frames and widgets can be desinged to accomdate a flowed text rendering strategy, and that bulk you speak of can be shaved.
Let's get into it.
This is really about content rendering models, document/view architecture, MVP architecture, etc.
This is really about rendering models. Maybe about MVP. But, it is /really/ about rendering models. If you say: "provide the framework for rendering content and the interaction between the frame and the graphics canvas, then let the application writer construct the document model that they need." If say say this, you've drawn a boundery between a GUI library and the task of rendering. A boundry that deserves more scrutiny. What's is the 80/20 of the "document model model they need"? I say FGDC. I might be wrong, but I think it is important to say something, so that the UI library really does facilitate the task of rendering content. I'll do what I did above: I view the Document as a rendering and event strategy. Rendering Strategy: Simplify rendering and event routing by arranging the text in accordance with a time-worn, standard-specified, human understanding of the arrangement of text. User is able to enter data by placing caret on areas of document and typing or pasting. Adding text causes the document to "reflow". Alternatively, "components" may be placed within the document, arranged in accordnace with typesetting conventions. Input is thus delegated to a component. Rendering is performed accordance with the grouping of text into blocks. Rendered blocks of text can be buffered for redisplay or shifted during reflow. Blocks of text off screen need not be rendered. Blocks of text can be rendered in advance of their scrolling into view. Event Strategy: Blocks of text may or may not be on the same z-axis. Mouse events are directed to an object block of text or component over which they occur. They event is sent to the enclosing block (paragraph, column, page, etc.) until it reaches a root block representing the document. Any block (at developer discression) may accept the caret and text input. If a component is present in the document, events are delegated to the component, before the are forwarded up the block heirarchy. Additionally, the document provides logic for the selection of text. Applications: Word, XForms, Help, FireFox, IE, Safari, Reader, vi, PostScript, Eudora, Cyrstal Reports, and on and on. UI Usage: Tree controls, online help, hyperlinked start pages, wizzards, text editing, forms, RSS, e-mail, HTML, labels, call-outs, splash pages. Minimal Library Needs: Axis-aligned boxes. Fonts. Scrolling. Caret. As a rendering and event strategy, it is more complicated, but not much more complicated than The Grid.
Widgets + layouts hit a ceiling pretty quick.
There are a few issues with providing an XML+CSS layout renderer like Thunderbird/Firefox:
It wouldn't require XML or CSS. The Mozillans have expressed regret that they bound their renderer to XML DOM. It would support stylized text, and text flow and layout, but the specification would come through a binding. I'd puruse the development of a document renderer. With it I'd develop a non-validating XML parser and CSS parser in Spirit. I believe there is some XML work in Spirit already for serialization. If more powerful XML work was needed, a library user could bind the document renderer to Xerces C++, or they could write their own binding, along the lines of your cell_provider.
Regardless of the reason, it is a successful approach. T/F is cross-platform, i18n, and skinnable. It works. It is easy to script and extend.
Size. A document rendering strategy for Boost would not have to support legacy HTML, which is a huge source of the Mozilla bug count. Nor would it have to support HTTP, FTP, SSL, or JavaScript. If you reduce that to flowing text layout, it is more complicated that a grid renderer, but not so complicated as a web browser. It could choose to support only a subset of document rendering initially, and might even be designed to degrade. I'd like to have the z-axis be a compile time design decision, for example. If you don't need it, compile it out. If you do need it, comiple in an implementation based a Canvas library.
* How do you provide the UI, i.e. the event binding and event flow?
Another reason for positing FGDC. The document, as specified by XML DOM has a very intuitive event model. The event is routed through a tree heirarchy that represents the structure of the document. If you click on a *bold* bit of text, the event is rounted first to the division representing the bold text, then to the enclosing paragraph, then to the column, then to the column. Any one of those objects might have an event handler. Event handling for documents has a well defined pattern to follow, the UI framework should accomdate that. If a mouse click lands on a document, the UI framework should delegate the event routing to the document renderer. It is the same with with the grid model, though. Somewhere their might be a universal event handler that want's to know the cell row and column, information that does not pertain to a raw mouse event. Different rendering strategies have a different view of events. Different rednering strategies route their events using different algorithms.
* What about people who want/need/require native interoperability? * How do you use native controls to get a native L&F?
With Carbon, I think you just use native controls. With Win32 GDI you roll your own controls. I'd use the document renderer to render the GDI text box replacement, and the canvas renderer to render the GDI check box replacement, because that's what the renderers are there for. It becomes the same question in /any/ of the overlay layouts. This is why I raised the question in another thread. If we crack that nut, then the development of x-platform rendering strategies is simple. Regardless of a document renderer, if we don't, then we've left a huge problem unsolved for the end user.
* What about other issues inbuilt to an operating system? * How do you interact with screen readers for blind users?
Well, I'm looking into it. http://www.section508.gov/index.cfm http://bobby.watchfire.com/bobby/html/en/index.jsp http://www.w3.org/TR/WAI-WEBCONTENT/ http://java.sun.com/docs/books/tutorial/uiswing/misc/access.html As native L & F, this is what libraries are for. If you are not going to address accessibility in the framework, then you've left a huge problem unsolved for the end user.
Fine. I'm rasing a discussion of content rendering to keep Boost.GUI as lean as possible. FGDC is about rendering strategies, what they require, and how a library can be designed to faciliate a lean, device specific implementation of these strategies. XUL succeeds as a framework, because the framework is developed in terms of a renderer, it works from the inside out. In closing. I see the Canvas as a rendering and event strategy, but one that goes beyond the need of most applications. It seems odd to build a lot of vector graphics into a drawing surface, when many UI applications will need axis-aligned boxes most of the time, and some won't need it at all. I see the Canvas as a rendering and event strategy that sits parallel to Forms, Grids, and Documents, not a foundation for Forms, Grids, or Documents. -- Alan Gutierrez - alan@engrm.com
Alan Gutierrez wrote:
I've thought about it some more, and FGDC is a set of rendering strategies. That is my taxonomy.
To me, a rendering strategy is something that is implemented by the application using the graphics framework. This covers your Document and Canvas strategies that target two different types of rendering model (one text-based documents, the other, graphical content). When you look at what a form is, you write a resource that describes the window and controls it has and get the OS to construct the window from that resource. In Cocoa/Carbon, that is a NIB file and in Windows it is a Dialog resource. These are different from a rendering strategy because they interact with the operating system, providing a top-level UI frame. I define a widget/component as a *UI object* that is usually provided by the OS. Thus, a table is a widget in Windows (a report-view ListView control). A grid is a tricky area because it is not provided by the OS, but it is a component because it provides a user interface to specific data.
So is the form.
I view a Form as a content rendering and event strategy.
I have the basic structure: layout::layout component frame widget layout::flow where layout contains the basic interface to get minimum size and update layout; component adds OS binding and event handling; frame provides a component that has a content area (drawn to using a rendering strategy); and widget provides data-based components that are placed on a frame.
But the Form and Grid are not drawn by the application and are connected to the OS.
My point exactly. These are all documents. Documents are omnipresent. (I've never spoken of retrictions.)
Okay. I see where you are coming from with this.
This is what I am trying to do as well.
What I meant was what I have described above.
Which is what I am doing with table/grid + cell-provider. The cell-provider is the data model. What I am saying is that there is often OS support for this model and should be used when applicable, thus it is a widget.
This is like what Andy Little has said about treating graphics objects as lightweight components (possibly with events). With the text object you describe, do you clip the text? Do you provide scolling? Do you support editing? What about undo? What starts out as a simple object becomes fairly complex. Providing a font object, allowing you to draw text to a canvas using the font will allow you to implement the above to suit your applications needs. Also, providing a separate scrolling class will allow you to have scrollable content.
A rendering model (how something is drawn) does not encompass things like events and OS interaction.
* What about people who want/need/require native interoperability? * How do you use native controls to get a native L&F?
With Carbon, I think you just use native controls.
With Win32 GDI you roll your own controls.
I think that this is not the right approach. You should only roll your own controls when that control isn't supported by the target OS. The Win32 GDI and Win32 controls/common controls are different things. I need more time to think about how to provide an independant abstraction for graphical elements, since I have been mainly focussed on UI.
Implementing a check box in Win32 as a custom control, how do you handle things like tabstops, using spacebar to check/uncheck the checkbox, provide XP theme support and a whole host of other things provided by the OS?
The question is whether you want a consistent x-platform L&F or a native platform L&F. I think you want a consistent L&F as default, whereas I want a native L&F as default. I want to provide the ability to write lightweight components, native components and custom rendering for native components. Lightweight components does not mean having a lightweight button (you could write one if you want), but means a component that does not consume OS resources (e.g. layout managers).
But the OS provides that native L&F. I don't want to go the Java route of not providing native component intergration. I agree that accessability needs to be addressed in some form or another, but by using native components, you get a lot of that accessability for you provided by the OS. Regards, Reece
Reece Dunn wrote:
Not different enough, IIUC. They're both subject to the same (shared) constraint on maximum number of window handles. To make a windows GUI framework scalable it must provide for that. Not every widget on the screen can afford to be an OS control or window, even when there are appropriate built-ins (consider a grid of spreadsheet cells). One approach might be to make them "ephemeral," i.e. conjure up the actual OS thingy only when you need to draw or process clicks there and then throw it away immediately or soon. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
* David Abrahams <dave@boost-consulting.com> [2004-12-29 13:52]:
Yes. In a more complicated UI, native controls are going to fail the application developer. Materialization is a good strategy, but it doesn't address the problem of rendering. An OS rendered control taking part in the window heirarchy of the OS, displayed within a client area that has an application defined z-axis with overlayed (oh, can I borrow a term?) thingies, well that OS native control stomps all over the application rendering. The application developer is left to roll their own, so what's the library for anyway? Dialog boxes on the one hand, do-it-yourself on the other? Sound's like MFC to me. -- Alan Gutierrez - alan@engrm.com
Alan Gutierrez wrote:
* David Abrahams <dave@boost-consulting.com> [2004-12-29 13:52]:
Can't you image the native controls into an offscreen buffer and then apply whatever transformations, masking, etc., that the framework likes in order to display it in the right context? -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
* David Abrahams <dave@boost-consulting.com> [2004-12-29 16:37]:
Materialization is a good strategy, but it doesn't address the problem of rendering.
Nice. Core point is that the library needs to make rendering strategies generic. If so, materialization is strategy, and there are others as well. Also, what about events & accessibilty? If you render the control off screen on a buffer, and take the result and paint it onto the client surface, the lack of the native OS control, well, I'm sure a Win32 GDI component is counting on a GDI HWND for event notification. So, render off screen, but then you have to figure out how to send events to the hidden control? It is an interesting hybrid approach, but I don't know that it works for the rouge Win32 GDI. Kind regards. -- Alan Gutierrez - alan@engrm.com
"David Abrahams" <dave@boost-consulting.com> wrote
I think the usual approach (eg MS Excel) is to draw the spreadsheet grid and its contents using graphics and text on the client area 'canvas'. Typically only one cell ever has 'focus'. ( Multiple selections are in fact just rendered using graphics, so no multiple windows are required here as you cant be selecting and editing simultaneously) An edit {control ,window, component, widget} is then created at the cell with the focus , using the size of the grid cell as the window size etc. eg pretty much as you describe. This does mean that there must be a general 'sameness' between drawing graphics on the canvas and drawing a window there. Lines and colours should be simple to match and align between the two types of graphics element(dumb and active). This implies to me that graphics_element is the common abstraction , which might be one base class of a window .The same methodology should apply to any graphics element . It should be possible to transform it into an active element. As to how an element is made active, this is surely an implementation detail. regards Andy Little
* Andy Little <andy@servocomm.freeserve.co.uk> [2004-12-30 03:59]:
This implies to me that graphics_element is the common abstraction, which might be one base class of a window
What about when you are not using graphic elements? When you are relying on resource files, you won't need a Boost graphics absraction at all. I don't think graphic_element is a base class, I think it is an implementation. I don't think you can have a grid without one, but you can have forms with out one. -- Alan Gutierrez - alan@engrm.com
"Alan Gutierrez" <alan-boost@engrm.com> wrote
* Andy Little <andy@servocomm.freeserve.co.uk> [2004-12-30 03:59]:
This implies to me that graphics_element is the common abstraction, which might be one base class of a window
I don't think graphic_element is a base class, I think it is an implementation.
Surely its an interface ...? : regards Andy Little
* Andy Little <andy@servocomm.freeserve.co.uk> [2004-12-31 04:22]:
No. On curses there is no "graphic" element. No. On forms-only PalmOS the graphics are not part the libraries concern. It never uses "graphics" only resource handles, any maybe, just maybe, a co-ordinate or two. Are we talking about the same thing? Interface and implentation are overloaded terms. -- Alan Gutierrez - alan@engrm.com
"Alan Gutierrez" <alan-boost@engrm.com> wrote
Are we talking about the same thing? Interface and implentation are overloaded terms.
// interface (ABC) struct graphics_element{ virtual void draw(Context c) =0; }; // gui object.. struct my_wnd : graphics_element, size_handler,move_handler,draw_handler { void draw(Context c); ...}; //simple line definition... template <typename Units> // struct simple_line{ point<Units> start,end;} // dumb drawable line template <typename Units> struct drawable_line : simple_line<Units>, graphics_element{ void draw(Context c); Units thickness; Color color; }; template <typename Units> struct active_line : drawable_line<Units>, user_action_handler, draw_handler{..}; // text struct text_element : graphics_element {..} // edit box struct active_text : text_element,user_action_handler, draw_handler{..}; regards Andy Little
* Andy Little <andy@servocomm.freeserve.co.uk> [2004-12-31 07:37]:
"Alan Gutierrez" <alan-boost@engrm.com> wrote
Are we talking about the same thing? Interface and implentation are overloaded terms.
// text struct text_element : graphics_element {..}
Except that on PlamOS, when using resource files, I don't need a draw function for my text boxes. I know they are draw somehow, but not by the Boost.GUI library. Could this be what I'm saying? struct graphics_element{ virtual void draw(Context c) =0; }; struct text_element { virtual void display(std::string s); } template <typename IdType> struct resource_binder { IdType resource_id; } // PalmOS edit box. struct resource_edit_box : text_element, resource_binder, user_action_handler, draw_handler{..}; // Drawn edit box. struct graphic_edit_box : text_element, graphics_element, user_action_handler, draw_handler{..}; -- Alan Gutierrez - alan@engrm.com
"Alan Gutierrez" <alan-boost@engrm.com> wrote in message news:20041231150348.GA20515@maribor.izzy.net...
Hmm I see what you are saying ... The os privately owns the draw function for some (system) elements. So it doesnt need to be part of the interface except for user drawn elements regards Andy Little
* Andy Little <andy@servocomm.freeserve.co.uk> [2004-12-31 13:36]:
Or library drawn elements (a nit). Also, I'm not creating boundries across applications, or platforms, such specialization might occur within a single application on a single platform. I might write a bitty spread sheet for Palm OS that uses Boost rendered elements on a grid, but still use resource bound elements for all other forms. Even though I've pulled the logic in for the spread sheet, it is going to be unused in Palm OS rendered forms. -- Alan Gutierrez - alan@engrm.com
"Alan Gutierrez" <alan-boost@engrm.com> wrote
* Andy Little <andy@servocomm.freeserve.co.uk> [2004-12-31 13:36]:
Yes.... on reflection the draw() function in the previous post was a red herring, and is closer to implementation as you say though IMO could also be classified as a particular type of graphics_element based on who owns its graphics real-estate. Nevertheles Who does the drawing is probably a key part of the make up of a particular graphics_element, and presumably it should be possible in an ideal world to change an element at run_time, from say a dumb rectangle ( explicitly drawn), to a window (drawn by the OS) to an embedded curses application (though see final paragraph),. The main idea behind the graphics_elements is to provide a single mechanism to express the spatial relationship between diverse entities in some 2D space(s).There is also something like a 3rd dimension (the Z-order) to consider in this ( as has been mentioned in another post) . I conjecture that there should be some common framework to express this. One needs surely to be able to measure distances and know extents of graphics_elements, regardless of their darker purpose, in order to implement a layout manager for example.
As far as the resource-bound elements that you mention goes I have to admit that I'm not too sure what these are. 'Resource' could mean many things, however it does sound like this is the same issue of sorting out the graphics real-estate using some 'spatial-framework' as sketched above and then providing some means of trading ownership between the application, the OS and other applications, but doesnt this make the boost:GUI itself start to look very much like an OS itself.? I was hoping that the scope of the project would be kept small enough to be realistic to achieve without a Herculean effort. Bottom line is all I really considered on gui was client area graphics,. ( client area meaning the bit I as the user own.... the canvas ) though the ability to make elements in the client area 'active' would be a bonus. Of course once I started to make active_elements, I would probably be well on the way to creating my own simple gui (which would also work on very basic OSes , with windows etc though restricted to the client area of the app IOW I lease the client area.from whatever OS(Sound familiar .... Java) . The advantage of this approach is that I am not relying on an OS to supply eg my button. I can construct it entirely from scratch, though can presumably replace my construction with something suitable supplied by an OS where possible. The bottom line requirements on the os are basically some drawing primitives and user input feedback etc. Hmmm... I suppose it really boils down to what the boost::gui actually should be, and what is realistic. regards Andy Little
* Andy Little <andy@servocomm.freeserve.co.uk> [2005-01-01 05:41]:
Consider a draw method for a spread sheet. Is there a graphics element for each cell? If I load 100x10,000 workbook, do I create 1,000,000 cell objects so that each of them can draw themselves? No. The grid draws the cells. Even if you had a grid flyweight, and create a cache of cell styles, it is still easier to draw the cells by simply drawing rows and columns. This is more than a nit. It is easy to talk about grids being an application problem. But if the library is modeled soley as an object tree, it gets cumbersome when it come across a rendering task that cannot be so modeled. When looking at windows, or a form like a dialog box, nested aa-boxes in a tree in memory works, and that's what we are modeling here, a graphics_element draws itself, but for grid, document, and canvas applications, this model breaks down. Which is why, I think there is some other mixin. A document needs to define how a button can participate in document layout. If the button cannot partipiate in the z-axis defined by the documet, because it participates in the z-axis of the windows system for example, their is a compile time error. An interface that makes for a document_component, grid_component, or form_component, so that the boundries between rendering strategies are not artificial ones like nested boxes.
Layout manager? What is your definition of layout manager, for clarification.
This project will be unlike anything else at Boost. I'm starting to feel that what I want to do does not belong here, but the holidays are not over yet. I can draw a dialog box using a resource designer in Win32, layout my buttons and textboxes, attach a number, and then hand a dialog id off to windows and saw, ShowDialog, and it will. Then I'd like to bind event handling to each control on the dialog, also identified by an ID. This event handling ought to be uniform with all other library event handling.
I see that a lot of people here feel the way you do. Primitives and events. There is probably a need for something so simple. I'd like to continue to participate in this discussion, but I don't know that it is going to ultimately be something I want to work on. This doesn't mean that I don't want to share work, research, or ideas with Boost. I am working from the application back to the OS, and I don't think my direction will meet with an OS direction, because I always see the OS forward direction fall short. I don't think that there'll be a Golden Spike. -- Alan Gutierrez - alan@engrm.com
"Alan Gutierrez" <alan-boost@engrm.com> wrote in message news:20050101122432.GA24017@maribor.izzy.net...
Only the visible part of the spreadsheet would require to be represented graphically of course. Even if each cell was a graphics_element this would typically only amount to (say) 1000 elements. In practise I guess lines representing the grid might be required and rectangular fills where individual cells or blocks are in different colours and of course text. All rendered (if it is out of date) when a draw request is made, So typically the lifetime of a graphics_element is at minimum the period between display updates. But I would consider the above decisions to be the job of the application designer, who hopefully would find his tools in the boost::gui library.
Yes.. the GUI is only providing some particular graphical View on data at one instant of time, using the artifice of available graphical facilities... a triumph of artifice over substance.(but without this mechanism to provide feedback to the user computers would be useless... so its such an important area).
An alternative to a tree representation would be to make them nodes in a graph (ala Boost Graph Library). The spatial relationship between graphics_elements would then be represented in an graph-edge property. To make the thing generic I guess the spatial relationship might be represented as a transform ( eg a homogeneous transform matrix). The transform between non-adjacent nodes could presumably also be worked out from the above properrties. I think this is quite similar idea to a scene-graph in 3D graphics.
Who draws a graphical entity ( the application or the system etc) is definitely a fundamental part of its character. Usually the user is given the task by the OS of drawing in the client area when prompted to do so( thus defining client_area). In that sense the user 'owns' the client area. The system owns ( and therefore draws) everything else. That may be a useful distinction for 'generic' graphics elements.
I am not sure what you are saying with the z-axis terminology.
The term document doesnt seem to fit here. I usually understand it to mean a data container, without particular direct graphical display significance.
One example would be typesetting in a word-processor. The text and and formatting information is essentially held in a serial container, but is constrained to fit the paper dimensions by the layout manager. Each time the data is modified the layout manager recomputes the absolute position that each text character appears. And of course images and windows may also participate in the layout. The layout manager takes care of flowing the text around them.
True... I am certainly not representative of all the views on boost, in fact I am probably very unrepresentative. ;-)
Thats true. However AFAIK the two requirements so far on boost::gui seem to be 1) That is implementable in standard C++ code. 2) that it is operating_system independent, although that is a loose term and could I guess mean pretty much anything in practise.
Something like what Reece Dunn has layed out in the [gui] Big Picture thread. Sounds like this is more low level than you would like, however The bottom up approach of designing the lower level components first although slower is more robust IMO. Test the low level components first and then build the high level stuff on top of that. I dont really like this, but the problem is that there is no common set of low level components across platforms,. Without this I suspect that we would be swimming in macro jam.
I guess that a project like this will require all the help it can and then some, and working code demonstrating the concepts is going to be the most persuasive factor in the direction that this library would take.
I don't think that there'll be a Golden Spike.
nor a magic bullet... :-) regards Andy Little
Hi, (and merry holidays to those that celebrate them :)
I would like to take the concept of layout managers a lot further. Basically a layout is nothing but a particular strategy to calculate the size and position of a set of widgets. The grid layout is one of them. Others are docking mechanisms, or autohide/unhide windows as you see them in Visual Studio for example: a window that shows itself when you hover with the mouse pointer over its tab for a while and that hides itself again when a different window gets the focus. These too are size/position handlers. So I'd rather think along positioning strategies rather than have a 'Grid'. A Grid is too low an abstraction. Regards Hajo -------------------------------------------- Lit Window Library - Speed up GUI coding 10x http://www.litwindow.com/library?src=ml wxVisualSetup - integrate wxWidgets into Visual Studio .NET http://www.litwindow.com/Products/products.html?src=ml BugLister - Defect Tracker http://www.litwindow.com/buglister?src=ml Tips & Tricks for wxWidgets & MS Visual Studio http://www.litwindow.com/Knowhow/knowhow.html?src=ml
Hajo Kirchhoff wrote:
Yes.
This would also cover strategies such as scrolling, which allows you to move about a space that is larger than the owning frame. These should be kept separate: that is, there should not be explicit support for docking toolbars or frames, but a dockable manager that adds docking support. Perhaps then layout manager is too specific a term. How about "UI manager"? There are two things that can be controlled: components and renderers, where a renderer is something that draws content (such as a vector graphic) onto a frames usable area. This would encompass: * layout managers (that control the layout of the component that they own), * docking (that provides dockable support to the component it owns), * scrolling (that provides scrolling functionality to a component or renderer), * splitters (that allow two components and/or renderers to be positioned next to each other with a bar that allows their sizes to be affected) and split-views (a specialised splitter that works for renderers to allow two active views of some provided content), * other advanced user interfaces.
A grid in layout manager terms allows you to position components at (x,y) positions, for example creating a calculator. There are other layout strategies (flow/aligned, e.g. buttons along the bottom right corner along the y-axis) that should be supported. The Grid that Alan was referring to is a data control that serves a large amount of data split into cells such as a spreadsheet. This is a valid component and needs supporting. In this model, we have a "Cell Provider" that is something like: struct cell_provider { long get_row_size() const = 0; std::string get_row_name( long r ) = 0; long get_column_size() const = 0; std::string get_column_name( long c ) = 0; bool is_cell_editable( long r, long c ) = 0; std::string get_cell( long r, long c ) = 0; void set_cell( long r, long c ) = 0; }; This then serves a grid component that is used to provide a spreadsheet style control, using both column *and* row name. It also serves a table component to implement things like the list of e-mails in an e-mail manager or the list of songs on an album (with additional details such as track number, duration, etc.). This table component is akin to the JTable/TableModel in Java Swing and does not make use of row name. Happy Holidays, Reece
* Hajo Kirchhoff <mailinglists@hajo-kirchhoff.de> [2004-12-25 05:47]:
I would like to take the concept of layout managers a lot further.
Basically a layout is nothing but a particular strategy to calculate the size and position of a set of widgets. The grid layout is one of them.
Hajo I agree, as far as positioning strategies go. Layout management can be far richer. I was not putting forward The Grid as a positioning strategy, and I think both you and Reece mistook the point of the OP. The point of the OP (me) was to lay out a taxonomy of what goes into the client window of an application. I was attemping a highest-level look at the sort of interfaces developers must regularly assemble, so that a Boost GUI library would faciliate the those tasks. My post was to note, that in common business applications repeate four different canonical UI forms in their /client/ areas, The Form, The Grid, The Document, or The Canvas. By The Grid, I'm thinking of applications like Excel, Access, or SASS, where information is presented in a tabular format. Information information is mapped to a tabular data structure, or a sparse matrix, rows and columns are added and deleted, ranges are selected, cells are referenced, etc. I'm noting that this is such a common display of data, that it should be considered as a special strategy for the client area. A grid, in this sense, is far more than just a layout. But, to address your concern directly, layout is far more than just a grid. The layout and arrament of client areas is a whole other area of discussion. Reece goes on to talk about a desire for something along JTable/TableModel, so I think we are talking about the same thing in different terms. It would be disappointing to see another library that was, like MFC or Java AWT, a whole lot of framing, but for the client area you have a set of stock widgets on the one hand, or take the pen and draw it yourself on the other. When I look at xwWindows, I see a library went from the OS, frames, widgets, and primtives, and on the way towards the application and got lost. I put forward what I thought were the four basic types of application that a library design could keep it's eyes on the prize. Also, so we can consider how to compartmentalize the library. If I'm developing a vector graphics application, I don't need a validation library, if I'm merely tracking the mouse over a grid, I don't need much in the way of hit-testing. -- Alan Gutierrez - alan@engrm.com
* Michael Walter <michael.walter@gmail.com> [2004-12-27 04:52]:
Which is why I was talking about "hit" testing, since I was trying to look at the needs of an event router, and opposed to the renderer. Visibility testing is not necessary, or rather should not be necessary, for the an implementer of a form or grid renderer. The components within the form or grid are on the same z-axis. A dialog box might be obscuring the control, but that, "visibility testing", is done by a windowing system. Unlike a document renderer that implements CSS positioning, where there is an z-axis property, or a canvas renderer that renders a Visio diagram. An implementation of a form, or grid renderer depends on the windowing system for "visibility testing" and "hit testing" is made simply by dividing the drawing surface up aligned to the x and y axis. -- Alan Gutierrez - alan@engrm.com
* Michael Walter <michael.walter@gmail.com> [2004-12-27 15:29]:
Yes. I know. Thank you. Am I not saying thank you enough? More smilies? Also, my point wasn't really clear to me, until now. The term "visibility testing" cleared up the distinctions. Thank you. :^) -- Alan Gutierrez - alan@engrm.com
On Mon, Dec 27, 2004 at 03:33:10PM -0500, Alan Gutierrez wrote:
Also, my point wasn't really clear to me, until now. The term "visibility testing" cleared up the distinctions.
Yes, the intention wasn't clear to me after the first read, therefore my misguided correction. Marcelo
participants (7)
-
Alan Gutierrez -
Andy Little -
David Abrahams -
Hajo Kirchhoff -
Marcelo E. Magallon -
Michael Walter -
Reece Dunn