Hi, any plans of implementing the inverse gamma distribution as part of the distributions library ? What about multivariate (in particular the multivariate normal and t distributions and Wishart and Dirichlet ) ? thanks, Thomas
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Thomas Mang Sent: Friday, July 30, 2010 4:29 PM To: boost-users@lists.boost.org Subject: [Boost-users] [distributions]: Inverse Gamma
Hi,
any plans of implementing the inverse gamma distribution as part of the distributions library ?
This looks possible - but I'm curious about applications - you obviously have one, but Wikipedia doesn't mention any http://en.wikipedia.org/wiki/Inverse-gamma_distribution But you obviously have one ;-)
What about multivariate (in particular the multivariate normal and t distributions and Wishart and Dirichlet ) ?
A previous offer to do some multivariate distributions seems have fizzled out - perhaps because it is *much* more difficult, particularly with the templated and 'policied ' (to control the troublesome parameter cases) structure used for the Boost.Math library. There may be no analytic expressions for some like inverse and CDF. So some strong motivation and support would be needed to embarge on this. Paul --- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
On 30.07.2010 18:51, Paul A. Bristow wrote:
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Thomas Mang Sent: Friday, July 30, 2010 4:29 PM To: boost-users@lists.boost.org Subject: [Boost-users] [distributions]: Inverse Gamma
Hi,
any plans of implementing the inverse gamma distribution as part of the distributions library ?
This looks possible - but I'm curious about applications - you obviously have one, but Wikipedia doesn't mention any
http://en.wikipedia.org/wiki/Inverse-gamma_distribution
But you obviously have one ;-)
Yes I truly have one ;) The inverse gamma distribution and its special case, the scaled inverse chi-square distribution, is the conjugate prior to the normal distribution variance parameter in Bayesian statistics. Pretty much as uncommon and unheard of as it is outside Bayes world [to the best of my knowledge], it's very much central to Bayesian stats and appears in every textbook right after the introduction chapter ;) http://en.wikipedia.org/wiki/Scaled_inverse_chi-square_distribution http://en.wikipedia.org/wiki/Conjugate_prior Hence I wonder it has not been requested so far - but being a Bayesian C++ / booster I definitely want / need it :). @John: Yes it is a transformation deviate of the gamma, and an easy so. And it should be fairly easy to implement IMHO. Is contribution on my side expected (can be done just notice I am a [heavy !] user of the stats library only, not familiar with code / numerical stability issues).
What about multivariate (in particular the multivariate normal and t distributions and Wishart and Dirichlet ) ?
A previous offer to do some multivariate distributions seems have fizzled out - perhaps because it is *much* more difficult, particularly with the templated and 'policied ' (to control the troublesome parameter cases) structure used for the Boost.Math library. There may be no analytic expressions for some like inverse and CDF.
So some strong motivation and support would be needed to embarge on this.
Paul
I think the multivariate normal is fairly obvious, use for the others equally arises in Bayesian statistics. But I understand that everything is much more complicated. What about 'vegetarian' versions offering say only the pdf (that is needed a lot. Personally I have never had the need for CDFs / inverse CDFs as these are truly a big mess. But the pdf - yes would be cool). best, Thomas
--- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Thomas Mang Sent: Tuesday, August 03, 2010 3:07 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [distributions]: Inverse Gamma
any plans of implementing the inverse gamma distribution as part of the distributions library ?
This looks possible - but I'm curious about applications - you obviously have one, but Wikipedia doesn't mention any
http://en.wikipedia.org/wiki/Inverse-gamma_distribution
But you obviously have one ;-)
Yes I truly have one ;) The inverse gamma distribution and its special case, the scaled inverse chi-square distribution, is the conjugate prior to the normal distribution variance parameter in Bayesian statistics. Pretty much as uncommon and unheard of as it is outside Bayes world [to the best of my knowledge], it's very much central to Bayesian stats and appears in every textbook right after the introduction chapter ;)
http://en.wikipedia.org/wiki/Scaled_inverse_chi-square_distribution http://en.wikipedia.org/wiki/Conjugate_prior
Hence I wonder it has not been requested so far - but being a Bayesian C++ / booster I definitely want / need it :).
@John: Yes it is a transformation deviate of the gamma, and an easy so. And it should be fairly easy to implement IMHO.
The pdf and pdf etc looks fairly straightforward (only uses exp, pow and gamma?) so I might be persuaded to do these. But the inverses (qhantiles) may prove more troublesome if have to be done by brute force numerically R library does it numerically http://rss.acs.unt.edu/Rdoc/library/pscl/html/igamma.html - or are there analytic expressions for these? Or can it use the inverse of the gamma distribution?
Is contribution on my side expected (can be done just notice I am a [heavy !] user of the stats library only, not familiar with code / numerical stability issues).
I'm not mathematician enough to deal with this - but I can deal with the obfuscated code (by templating and policies) if you can provide the equations. (And there is the question of testing - some parameters and value combinations (preferably exact) are needed for sanity and accuracy checks). Are there use cases for the inverses? Paul --- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
On 04.08.2010 10:37, Paul A. Bristow wrote:
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Thomas Mang Sent: Tuesday, August 03, 2010 3:07 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [distributions]: Inverse Gamma
any plans of implementing the inverse gamma distribution as part of the distributions library ?
This looks possible - but I'm curious about applications - you obviously have one, but Wikipedia doesn't mention any
http://en.wikipedia.org/wiki/Inverse-gamma_distribution
But you obviously have one ;-)
Yes I truly have one ;) The inverse gamma distribution and its special case, the scaled inverse chi-square distribution, is the conjugate prior to the normal distribution variance parameter in Bayesian statistics. Pretty much as uncommon and unheard of as it is outside Bayes world [to the best of my knowledge], it's very much central to Bayesian stats and appears in every textbook right after the introduction chapter ;)
http://en.wikipedia.org/wiki/Scaled_inverse_chi-square_distribution http://en.wikipedia.org/wiki/Conjugate_prior
Hence I wonder it has not been requested so far - but being a Bayesian C++ / booster I definitely want / need it :).
@John: Yes it is a transformation deviate of the gamma, and an easy so. And it should be fairly easy to implement IMHO.
The pdf and pdf etc looks fairly straightforward (only uses exp, pow and gamma?) so I might be persuaded to do these. But the inverses (qhantiles) may prove more troublesome if have to be done by brute force numerically
R library does it numerically http://rss.acs.unt.edu/Rdoc/library/pscl/html/igamma.html
- or are there analytic expressions for these?
Or can it use the inverse of the gamma distribution?
Yes pdf just uses exp, pow and gamma function, for the cdf / complementary cdf also the incomplete gamma function is needed - all in the library available. For quantiles yes you can use quantiles of the gamma distribution. Indeed all of pdf, cdf , complementary cdf and quantiles can be derived using calls to the corresponding functions of the gamma distribution: If we define both the gamma and inverse gamma distribution with the shape and scale parameter (as is done for the boost implementation of the gamma) and in that constructor order, then the relations are [should be] as follows: typedef gamma_distribution<...> gamma_dist; typedef inverse_gamma_distribution<...> inv_gamma_dist; pdf(inv_gamma_dist(a, b), x) == pdf(gamma_dist(a, 1/b), 1/x) * 1/(x*x); cdf(inv_gamma_dist(a, b), x) == 1 - cdf(gamma_dist(a, 1/b), 1/x); quantile(inv_gamma(a, b), p) == 1 / quantile(gamma_dist(a, 1/b), 1/x); where for the pdf the extra term 1/(x*x) is the density transformation Jacobian term while for the probabilities no equivalent term is needed. Note that I just wrote "1" as nominator to save space and avoid a newsgroup-linebreak; it should be a floating point type in real code of course. And my "[should be]" refers to that I just worked that out and tested it in R, but not in C++. But it will be double-checked anyway after it's implemented. Why R evaluates the quantile numerically is beyond my understanding. Whether you implement the pdf / cdf directly or go for above relations is probably an implementation issue, in terms of performance it shouldn't make a big difference. If you go for implementation that would be great and very much appreciated with lots of thanks. Could you then also implement the inverse chi-square and scaled inverse chi-square distribution as well (they are just special inverse gamma distributions, but their names might ring more bells and I'd find it appealing to offer them straight away, similar to Chi-square and Gamma distribution cases) ?
Is contribution on my side expected (can be done just notice I am a [heavy !] user of the stats library only, not familiar with code / numerical stability issues).
I'm not mathematician enough to deal with this - but I can deal with the obfuscated code (by templating and policies) if you can provide the equations.
(And there is the question of testing - some parameters and value combinations (preferably exact) are needed for sanity and accuracy checks).
Are there use cases for the inverses?
What is "usually" (with my subjective definition of "usually") needed is pdf and, fairly important actually for various applications, random numbers. Out of memory not a single application for cdf or quantiles comes to mind. Of much greater importance are spherical properties such as Mahalanobis distance and area for confidence ellipses etc. To the best of my knowledge more advanced matrix algebra is required however (inverses, decompositions). I have to check with the uBLAS library what they offer as base, and what is needed. best, Thomas
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Thomas Mang Sent: Wednesday, August 04, 2010 4:02 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [distributions]: Inverse Gamma
On 04.08.2010 10:37, Paul A. Bristow wrote:
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Thomas Mang Sent: Tuesday, August 03, 2010 3:07 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [distributions]: Inverse Gamma
any plans of implementing the inverse gamma distribution as part of the distributions library ?
This looks possible - but I'm curious about applications - you obviously have one, but Wikipedia doesn't mention
any
http://en.wikipedia.org/wiki/Inverse-gamma_distribution
But you obviously have one ;-)
Yes I truly have one ;) The inverse gamma distribution and its special case, the scaled inverse chi-square distribution, is the conjugate prior to the normal distribution variance parameter in Bayesian statistics. Pretty much as uncommon and unheard of as it is outside Bayes world [to the best of my knowledge], it's very much central to Bayesian stats and appears in every textbook right after the introduction chapter ;)
http://en.wikipedia.org/wiki/Scaled_inverse_chi-square_distribution http://en.wikipedia.org/wiki/Conjugate_prior
Hence I wonder it has not been requested so far - but being a Bayesian C++ / booster I definitely want / need it :).
@John: Yes it is a transformation deviate of the gamma, and an easy so. And it should be fairly easy to implement IMHO.
The pdf and pdf etc looks fairly straightforward (only uses exp, pow and gamma?) so I might be persuaded to do
these.
But the inverses (qhantiles) may prove more troublesome if have to be done by brute force numerically
R library does it numerically http://rss.acs.unt.edu/Rdoc/library/pscl/html/igamma.html
- or are there analytic expressions for these?
Or can it use the inverse of the gamma distribution?
Yes pdf just uses exp, pow and gamma function, for the cdf / complementary cdf also the incomplete gamma function is needed - all in the library available. For quantiles yes you can use quantiles of the gamma distribution. Indeed all of pdf, cdf , complementary cdf and quantiles can be derived using calls to the corresponding functions of the gamma distribution:
If we define both the gamma and inverse gamma distribution with the shape and scale parameter (as is done for the boost implementation of the gamma) and in that constructor order, then the relations are [should be] as follows:
typedef gamma_distribution<...> gamma_dist; typedef inverse_gamma_distribution<...> inv_gamma_dist;
pdf(inv_gamma_dist(a, b), x) == pdf(gamma_dist(a, 1/b), 1/x) * 1/(x*x);
cdf(inv_gamma_dist(a, b), x) == 1 - cdf(gamma_dist(a, 1/b), 1/x);
quantile(inv_gamma(a, b), p) == 1 / quantile(gamma_dist(a, 1/b), 1/x);
where for the pdf the extra term 1/(x*x) is the density transformation Jacobian term while for the probabilities no equivalent term is needed. Note that I just wrote "1" as nominator to save space and avoid a newsgroup-linebreak; it should be a floating point type in real code of course. And my "[should be]" refers to that I just worked that out and tested it in R, but not in C++. But it will be double-checked anyway after it's implemented. Why R evaluates the quantile numerically is beyond my understanding. Whether you implement the pdf / cdf directly or go for above relations is probably an implementation issue, in terms of performance it shouldn't make a big difference.
If you go for implementation that would be great and very much appreciated with lots of thanks. Could you then also implement the inverse chi-square and scaled inverse chi-square distribution as well (they are just special inverse gamma distributions, but their names might ring more bells and I'd find it appealing to offer them straight away, similar to Chi-square and Gamma distribution cases) ?
OK that seems to be what I would need to know. I'll give this a try soon and tell you when a draft is available.
(And there is the question of testing - some parameters and value combinations (preferably exact) are needed for sanity and accuracy checks).
Any *exact* pdf and cdf values from *exact* parameters would be helpful - if you know any. But I'll look more closely at the equations to try to find some too. Do you have easy access to R that you could produce some test values (to double precision only of course)? Paul --- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
(And there is the question of testing - some parameters and value combinations (preferably exact) are needed for sanity and accuracy checks).
Any *exact* pdf and cdf values from *exact* parameters would be helpful - if you know any. But I'll look more closely at the equations to try to find some too.
Sorry, no exact ones directly available.
Do you have easy access to R that you could produce some test values (to double precision only of course)?
Yes I have, can produce pretty much whatever you would need. R and C++ also have a language interface. If it helps I should be able to write a loop which evaluates some million or so calls of the functions using both R and the boost implementation and compares them. If you just need some R evaluations output to a file, no problem as well. Let me know what you would need. best, Thomas
Yes I truly have one ;) The inverse gamma distribution and its special case, the scaled inverse chi-square distribution, is the conjugate prior to the normal distribution variance parameter in Bayesian statistics. Pretty much as uncommon and unheard of as it is outside Bayes world [to the best of my knowledge], it's very much central to Bayesian stats and appears in every textbook right after the introduction chapter ;)
http://en.wikipedia.org/wiki/Scaled_inverse_chi-square_distribution http://en.wikipedia.org/wiki/Conjugate_prior
Hence I wonder it has not been requested so far - but being a Bayesian C++ / booster I definitely want / need it :).
@John: Yes it is a transformation deviate of the gamma, and an easy so. And it should be fairly easy to implement IMHO.
The pdf and pdf etc looks fairly straightforward (only uses exp, pow and gamma?) so I might be persuaded to do these. But the inverses (qhantiles) may prove more troublesome if have to be done by brute force numerically
R library does it numerically http://rss.acs.unt.edu/Rdoc/library/pscl/html/igamma.html
- or are there analytic expressions for these?
Or can it use the inverse of the gamma distribution?
Implementation would be the same as the gamma distro, all the information needed is on Wikipedia - cdf is simply: gamma_q(alpha, beta/x); so gamma_q_inv can be used for the quantile. PDF is just a call to gamma_p_derivative.
Is contribution on my side expected (can be done just notice I am a [heavy !] user of the stats library only, not familiar with code / numerical stability issues).
I'm not mathematician enough to deal with this - but I can deal with the obfuscated code (by templating and policies) if you can provide the equations.
(And there is the question of testing - some parameters and value combinations (preferably exact) are needed for sanity and accuracy checks).
Paul, I don't mind filling in the implementation details if you could take care of docs and tests - if that helps? Do we need the scaled-inverse-chi-squared as well? John.
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of John Maddock Sent: Wednesday, August 04, 2010 4:32 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [distributions]: Inverse Gamma
Yes I truly have one ;) The inverse gamma distribution and its special case, the scaled inverse chi-square distribution, is the conjugate prior to the normal distribution variance parameter in Bayesian statistics. Pretty much as uncommon and unheard of as it is outside Bayes world [to the best of my knowledge], it's very much central to Bayesian stats and appears in every textbook right after the introduction chapter ;)
http://en.wikipedia.org/wiki/Scaled_inverse_chi-square_distribution http://en.wikipedia.org/wiki/Conjugate_prior
Hence I wonder it has not been requested so far - but being a Bayesian C++ / booster I definitely want / need it :).
@John: Yes it is a transformation deviate of the gamma, and an easy so. And it should be fairly easy to implement IMHO.
The pdf and pdf etc looks fairly straightforward (only uses exp, pow and gamma?) so I might be persuaded to do these. But the inverses (qhantiles) may prove more troublesome if have to be done by brute force numerically
R library does it numerically http://rss.acs.unt.edu/Rdoc/library/pscl/html/igamma.html
- or are there analytic expressions for these?
Or can it use the inverse of the gamma distribution?
Implementation would be the same as the gamma distro, all the information needed is on Wikipedia - cdf is simply:
gamma_q(alpha, beta/x);
so gamma_q_inv can be used for the quantile. PDF is just a call to gamma_p_derivative.
Is contribution on my side expected (can be done just notice I am a [heavy !] user of the stats library only, not familiar with code / numerical stability issues).
I'm not mathematician enough to deal with this - but I can deal with the obfuscated code (by templating and policies) if you can provide the equations.
(And there is the question of testing - some parameters and value combinations (preferably exact) are needed for sanity and accuracy checks).
Paul, I don't mind filling in the implementation details if you could take care of docs and tests - if that helps? Do we need the scaled-inverse-chi-squared as well?
John - I was just about to start to do the inverse gamma implementation - since it doesn't appear to need anything fancy like root finding, it might just within my skill_set/pay_grade ;-) I'd need more guidance on the scaled-inverse-chi-squared (since we are doing an upgrade, it might be worth doing this too). But I'll certainly do the grunt work on docs and testing anyway. Thomas - could you provide an *example* of using this distribution(s) for our docs? (IMO this is especially important because the presentation of the distributions is unlike what your average FORTRAN or R user would expect). Paul --- Paul A. Bristow Prizet Farmhouse Kendal, UK LA8 8AB +44 1539 561830, mobile +44 7714330204 pbristow@hetp.u-net.com
any plans of implementing the inverse gamma distribution as part of the distributions library ?
Not till now, of course submissions are always welcome ;-) Isn't this one just a transformation of the gamma distribution though?
What about multivariate (in particular the multivariate normal and t distributions and Wishart and Dirichlet ) ?
They've been asked for, but there's no progress in implementing them I'm afraid. Regards, John.
participants (4)
-
John Maddock -
Paul A. Bristow -
Thomas Mang -
Thomas Mang