No not at present, but I've been playing with the attached (very simple) code for this purpose,

Ha! Forgot the attachment, here we go.... John.

I have some simple stuff in the sandbox under the "stat" directory. You can view its usage here:

Otto: first off we really need something like this, in Boost and in the std as well IMO. It really galls me that my desktop calculator has more math functions than <cmath> has, and wait for it: my calculator is 25 years old !!!!!!!! However, I'm not sure that either of us has the right interface yet: I'm particularly concerned that you're making these algorithms, it means that if you want to access more than one statistic you have to make multiple passes over the data. The advantage of the "make it an object" approach is that pretty much all stats you could want are accessible after a single pass over the data. More than that you can: * Pause at any time and read off the stats, and then continue adding more data if you want. * An extension of the above would be to make the stats object serialisable. * Two or more objects can be "added" together to obtain the stats for the combined data without reaccessing the original data: imagine a weather station gathering temperature data over time: hourly stats can be combined into daily or weekly stats without going back to the original data - which may be either discarded (unlikely) or stored in offline storage. Unfortunately: this method is prone to numerical overflow/underflow :-( Knuth has some fancy algorithms that I believe avoid that, but then you loose the simplicity and checkpointing/additive behaviour of the "scorecard" based system. So as usual, there ain't no free lunch! John.

John Maddock wrote:

...

I have some simple stuff in the sandbox under the "stat" directory. You can view its usage here:

Otto: first off we really need something like this, in Boost and in the std as well IMO. It really galls me that my desktop calculator has more math functions than <cmath> has, and wait for it: my calculator is 25 years old !!!!!!!!

I don't disagree with that :-) I dont' plan to work on this myself in the next year; I have enough to do. We should have an official list of volunteers that where looking for ways to contribute.

However, I'm not sure that either of us has the right interface yet:

I'm not satisfied with my own use of tuple as return-values since it so easy to forget which tuple element is what. I would prefer named tuples or simply template< class T > struct least_square_result { T slope, intersection, correlation; }; etc.

I'm particularly concerned that you're making these algorithms, it means that if you want to access more than one statistic you have to make multiple passes over the data.

Right. It's a trade-off. OTOH, if you only call one algorithm, you don't want to pay for accumulation that is not used. So maybe we need algorithms that takes iterators and algorithms that take some kind of accumulator object.

The advantage of the "make it an object" approach is that pretty much all stats you could want are accessible after a single pass over the data. More than that you can:

* Pause at any time and read off the stats, and then continue adding more data if you want.

Right, this could be very useful.

* An extension of the above would be to make the stats object serialisable. * Two or more objects can be "added" together to obtain the stats for the combined data without reaccessing the original data: imagine a weather station gathering temperature data over time: hourly stats can be combined into daily or weekly stats without going back to the original data - which may be either discarded (unlikely) or stored in offline storage.

Unfortunately: this method is prone to numerical overflow/underflow :-(

How can this be different than just accumulating it all from scratch? (Or is it the accumulator method in general that is error-prone?) -Thorsten

John Maddock wrote:

...

I have some simple stuff in the sandbox under the "stat" directory. You can view its usage here:

Otto: first off we really need something like this, in Boost and in the std as well IMO. It really galls me that my desktop calculator has more math functions than <cmath> has, and wait for it: my calculator is 25 years old !!!!!!!!

However, I'm not sure that either of us has the right interface yet: I'm particularly concerned that you're making these algorithms, it means that if you want to access more than one statistic you have to make multiple passes over the data. The advantage of the "make it an object" approach is that pretty much all stats you could want are accessible after a single pass over the data. More than that you can:

* Pause at any time and read off the stats, and then continue adding more data if you want.

I have been working on a statistical library based on an incremental calculation model. The syntax is like this: // define an accumulator set for calculating mean and max // of a sequence of doubles accumulator_set > acc; // push some data into the accumulator acc(1.2); acc(2.3); acc(3.4); // fetch some intermediate results std::cout << mean(acc) << std::endl; /* push more data, etc. */ Some statistics may depend on the results of other statistics, in which case they are calculated automatically and in the correct order. The set of statistical accumulators is extensible, of course. I hope to have a version available online Real Soon Now. -- Eric Niebler Boost Consulting www.boost-consulting.com

I've now put what I have in the Boost File Vault. You can find it under the name stats.tar.gz at the following location:

http://www.boost-consulting.com/vault/index.php?directory=Math%20-%20Numerics&

No docs, yet -- working on that right now. But the tests might give some inkling of how to use it.

Very cool, go boy go! :-) I haven't done much more than quickly browse the docs and a couple of test programs, but it looks like you have everything covered to me. John.

Thorsten Ottosen wrote:

John Maddock wrote:

...

...

I've now put what I have in the Boost File Vault. You can find it under the name stats.tar.gz at the following location:

http://www.boost-consulting.com/vault/index.php?directory=Math%20-%20Numerics&

No docs, yet -- working on that right now. But the tests might give some inkling of how to use it.

Very cool, go boy go! :-)

I haven't done much more than quickly browse the docs and a couple of test programs, but it looks like you have everything covered to me.

Can one calculate covariance and simple linear regression (correlation, intersection, slope)?

Not yet, but adding new statistical accumulators is pretty easy. Patches would be graciously accepted. -- Eric Niebler Boost Consulting www.boost-consulting.com

Mike Gibson wrote:

On 1/12/06, Eric Niebler wrote:

...

...

Can one calculate covariance and simple linear regression (correlation, intersection, slope)?

Not yet, but adding new statistical accumulators is pretty easy. Patches would be graciously accepted.

I'm working on converting a median accumulator algorithm I wrote a little while back to your framework.

Great!

It does however require some parameters to optimize the performance. How is that handled in your framework?

I'm actively working on the docs, updated daily at http://boost-sandbox.sf.net/libs/accumulators. In particular, have a look at http://tinyurl.com/dpzmt. Basically, you'll use named parameters from the Boost.Parameters library. Have a look at how order_impl is implemented to accept the cache_size named parameter. I need to add an Extensibility section to the docs that describes this in detail. -- Eric Niebler Boost Consulting www.boost-consulting.com

First, I'm strongly interested in such library for statistics calculations in std or boost. I'm going to test frequently what you're working on.

Second, I think it may be interesting to take a look at the R Project http://www.r-project.org. May be, instead of inventing the wheel, we could provide a kind of interoperability between C++ and R. I have in mind something like Boost.Python. R is a very rich statistical computing package what could shorten the time-to-market of statistical package for boost.

What do you think?

I think it needs someone to step up and volunteer to write it :-) John.