[rfc] a library for gesture recognition, speech recognition, and synthesis
Hello, For the past few years I've been working on the AME Patterns library - a generic library for modeling, recognition and synthesis of sequential patterns. So far it's been applied mostly to gesture recognition (e.g., recognition of mouse gestures, full body gestures from video or motion capture data, accelerometer gestures), but recently it has also shown some good results on speech datasets. It can also be used to synthesize the same kinds of patterns it can recognize (e.g., you could use it to synthesize a gesture or speech, or purely numerical patterns). The library provides support for hidden Markov models and other related models. The design is concept-based, and the library uses the Boost Graph Library, Boost.Range (including the RangeEx extension), Boost.Fusion, Boost.Math, Boost.Random, and other boost libraries. The documentation for the library is available here: http://ame4.hc.asu.edu/amelia/patterns/ However, the documentation is lacking, so I'm curious to know which parts of the functionality (if any) are of interest to the boost community, so I know what parts of the documentation to focus on. I am also considering proposing a presentation of this library to BoostCon. The library is currently released under the GPL, which I realize is not acceptable to some. Please let me know your thoughts. If anyone is interested in using this library, please let me know what you'd like to use it for and I'd be happy to help you get started. Kind regards, Stjepan
On Wed, Oct 21, 2009 at 3:53 PM, Stjepan Rajko <stjepan.rajko@gmail.com> wrote:
However, the documentation is lacking, so I'm curious to know which parts of the functionality (if any) are of interest to the boost community, so I know what parts of the documentation to focus on. I am also considering proposing a presentation of this library to BoostCon.
We would be interested if it could make something like this (video links to demonstration of multi-touch touch table): http://www.youtube.com/watch?v=X2pPeW4cUgU easier to code for. Currently the gesture recognition is pretty primitive, and none of it is generic. Would love to hear that this library could help. --Michael Fawcett
On Fri, Oct 23, 2009 at 10:56 AM, Michael Fawcett <michael.fawcett@gmail.com> wrote:
On Wed, Oct 21, 2009 at 3:53 PM, Stjepan Rajko <stjepan.rajko@gmail.com> wrote:
However, the documentation is lacking, so I'm curious to know which parts of the functionality (if any) are of interest to the boost community, so I know what parts of the documentation to focus on. I am also considering proposing a presentation of this library to BoostCon.
We would be interested if it could make something like this (video links to demonstration of multi-touch touch table):
Nice video. Did you work on that project or are you working on something similar?
easier to code for. Currently the gesture recognition is pretty primitive, and none of it is generic. Would love to hear that this library could help.
You could use the library in it's current state to provide a richer set of gestures. Here are some examples showing a vocabulary of gestures being trained and then recognized: http://www.youtube.com/watch?v=LAX5qgzYHjU (iPhone gesture classification) http://www.youtube.com/watch?v=mjjwhK4Dxt4 (mouse gesture on-line recognition) You could do something similar in the system shown in your video, at least with single-touch gestures (maybe they could be used as shortcuts to some of the functions otherwise accessible through the menu). There was also a nuicode GSoC project this year that used the library for multi-touch gestures: http://nuicode.com/projects/gsoc-gesture-models http://code.google.com/p/multitouch-gestr/ The system works pretty well, but I don't think there is much documentation at this point. Would any of this be useful to you? Best, Stjepan
On Sat, Oct 24, 2009 at 5:39 PM, Stjepan Rajko <stjepan.rajko@gmail.com> wrote:
Nice video. Did you work on that project or are you working on something similar?
I worked with the software and hardware shown in the video, but I did not work on that particular demonstration.
You could use the library in it's current state to provide a richer set of gestures. Here are some examples showing a vocabulary of gestures being trained and then recognized: http://www.youtube.com/watch?v=LAX5qgzYHjU (iPhone gesture classification) http://www.youtube.com/watch?v=mjjwhK4Dxt4 (mouse gesture on-line recognition)
You could do something similar in the system shown in your video, at least with single-touch gestures (maybe they could be used as shortcuts to some of the functions otherwise accessible through the menu).
There was also a nuicode GSoC project this year that used the library for multi-touch gestures: http://nuicode.com/projects/gsoc-gesture-models http://code.google.com/p/multitouch-gestr/
The system works pretty well, but I don't think there is much documentation at this point.
Would any of this be useful to you?
Yes, I had seen the nuicode project, but didn't follow it very closely. It's good to hear that your library would be useful in this context. Thanks! --Michael Fawcett
participants (2)
-
Michael Fawcett -
Stjepan Rajko