Preliminary submission: generic linear algebra algorithms library Boost LA

I remember some past discussions on this mailing list about the need for similar library in Boost, so I hope it is OK to skip the interest query and request a preliminary review for Boost LA. Boost LA provides a set of generic linear algebra algorithms, primarily for working with vectors and matrices of static size. The library also defines vector and matrix data types, however it allows users to introduce their own types by specializing the vector_traits and matrix_traits templates. Documentation and source code available at http://www.revergestudios.com/boost-la, released under the Boost Software License. Thanks, Emil Dotchevski

Hello Emil,
Boost LA provides a set of generic linear algebra algorithms, primarily for working with vectors and matrices of static size.
I don't quite get what you mean by "generic linear algebra algorithms". Some curriculum for Mathematics contain a two semester course on "Linear Algebra", but there is very little overlap between what is thought in such a course and your proposed library. The final piece to fully confuse me are the files "matrix_determinant.hpp" and "matrix_inverse.hpp". Are they meant as examples of how the library can be used? Then they should be part of the examples and not part of the library. Or do they also represent some "generic linear algebra algorithm"? But in this case, I wonder why you used an algorithm of complexity O(n!) for computing the determinant, when it's easy to implement an algorithm of complexity O(n^3) for computing the determinant. (The complexity of O(n^2*!(n-1)) for computing the inverse is also not better.) My conclusion from this is that you don't really consider "matrix_determinant.hpp" and "matrix_inverse.hpp" to be part of the library, because I doubt that you would propose a totally inefficient library for review. Regards, Thomas

Thomas Klimpel wrote:
Hello Emil,
Boost LA provides a set of generic linear algebra algorithms, primarily for working with vectors and matrices of static size.
I don't quite get what you mean by "generic linear algebra algorithms". Some curriculum for Mathematics contain a two semester course on "Linear Algebra", but there is very little overlap between what is thought in such a course and your proposed library.
Perhaps it should be 'generic small-sze linear algebra algorithms', where n is usually less than 10, often 3, 4 or 6. mvh /Marcus

On Tue, Oct 20, 2009 at 2:41 AM, Thomas Klimpel <Thomas.Klimpel@synopsys.com> wrote:
The final piece to fully confuse me are the files "matrix_determinant.hpp" and "matrix_inverse.hpp".
They are part of the library interface, not examples.
But in this case, I wonder why you used an algorithm of complexity O(n!) for computing the determinant, when it's easy to implement an algorithm of complexity O(n^3) for computing the determinant. (The complexity of O(n^2*!(n-1)) for computing the inverse is also not better.)
You are correct that the current implementations are not very efficient for large size matrices, where "large" is perhaps as small as 5x5 for some values of "very". :) (However the overloads that kick in automatically for 2x2, 3x3 and 4x4 matrices are efficient.)
I doubt that you would propose a totally inefficient library for review.
My efforts have been focused on getting the interface correct, and on decoupling algorithms from data types. There is more work to be done. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski wrote:
I remember some past discussions on this mailing list about the need for similar library in Boost, so I hope it is OK to skip the interest query and request a preliminary review for Boost LA.
Boost LA provides a set of generic linear algebra algorithms, primarily for working with vectors and matrices of static size. The library also defines vector and matrix data types, however it allows users to introduce their own types by specializing the vector_traits and matrix_traits templates.
Documentation and source code available at http://www.revergestudios.com/boost-la, released under the Boost Software License.
Just looked through the tutorial. Boost.LA looks easy and simple yet very capable, i.e. plain awesome! :) Hope it gets through. Cheers, /Marcus

I'm presuming that these are different than http://www.osl.iu.edu/research/mtl/ and boost multi_array as well as other systems. It would be interesting if the introduction included a comparison to these and other alternatives. Robert Ramey Emil Dotchevski wrote:
I remember some past discussions on this mailing list about the need for similar library in Boost, so I hope it is OK to skip the interest query and request a preliminary review for Boost LA.
Boost LA provides a set of generic linear algebra algorithms, primarily for working with vectors and matrices of static size. The library also defines vector and matrix data types, however it allows users to introduce their own types by specializing the vector_traits and matrix_traits templates.
Documentation and source code available at http://www.revergestudios.com/boost-la, released under the Boost Software License.
Thanks, Emil Dotchevski _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Tue, Oct 20, 2009 at 4:57 PM, Robert Ramey <ramey@rrsd.com> wrote:
I'm presuming that these are different than
http://www.osl.iu.edu/research/mtl/
and boost multi_array
... and what are the advantages over the Boost.uBLAS library? Cheers, -- Marco

On Tue, Oct 20, 2009 at 7:57 AM, Robert Ramey <ramey@rrsd.com> wrote:
I'm presuming that these are different than
http://www.osl.iu.edu/research/mtl/
and boost multi_array
as well as other systems. It would be interesting if the introduction included a comparison to these and other alternatives.
Good point. To answer briefly, both MTL and (Boost) LA define generic algorithms to work with matrices and vectors, but MTL is perhaps more similar to BLAS/uBLAS/cuBLAS in terms of problem domain. Boost multi_array is relevant as it is trivial to define matrix_traits specialization that allows (Boost) LA to treat it as a matrix. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski wrote:
Boost multi_array is relevant as it is trivial to define matrix_traits specialization that allows (Boost) LA to treat it as a matrix.
You lost me again. What do you mean by "trivial"? How should I be able to treat an n-dimensional multi_array of variable size (where n is a template parameter, hence static) as a matrix of static size (where a matrix is a sort of 2-dimensional multi_array)? I didn't saw the "static size" before Marcus Lindblom's answer to my initial question. I think "LA" or "Linear Algebra" is simply the wrong name for a static size matrix library, but names will only become important later. What would be more important would be to improve the description of what the library "wants to be", i.e. its intended scope.
(However the overloads that kick in automatically for 2x2, 3x3 and 4x4 matrices are efficient.)
I agree for 2x2 and 3x3, but the overload for 4x4 is probably already problematic. Regards, Thomas

Some final words for the start: I didn't really intended to review this library. I was just confused by the description about the scope of the library, so I followed the link and tried to conclude what the library intended to do. The matrix_inverse and matrix_determinant looked misplaced to me, so I skimmed over them and was left even more puzzled. At that point, I decided to write a potentially quite unfriendly mail explaining why I felt puzzled. The answer of Marcus Lindblom made me realize that the proposed library was actually quite polished, and probably took Emil quite some time. However, the question how to interpret "trivial" in the statement "Boost multi_array is relevant as it is trivial to define matrix_traits specialization that allows (Boost) LA to treat it as a matrix." was probably also quite unfriendly. So I decided to review the library so that I can also comment on the positive things. I noticed that my review is not free from unfriendly remarks, but there are hopefully enough friendly words to compensate for these. Emil Dotchevsk wrote:
so I hope it is OK to skip the interest query and request a preliminary review for Boost LA.
So, let's try to review the library:
Please always state in your review, whether you think the library should be accepted as a Boost library!
Good question. I have no real opinion on this question, so I will not participate in the real review. However, I don't like the name Boost.LA and I think that the description "Boost LA provides a set of generic linear algebra algorithms, ..." is not really helpful. And I really miss a clear statement with respect to the scope of the library.
- What is your evaluation of the design?
I haven't really tried to evaluate the design. But everything I have seen looked OK, and some things where extremely nice done. A possible improvement might be more explicit about the requirements of the value types of the matrices.
- What is your evaluation of the implementation?
The implementation looks quite polished. I was impressed by the complete suite of unit-tests and the code generators. I've seen that there were even code generators for matrix_determinant and matrix_inverse. However, I think that these two code generators are false friends, because they implement an analytic formula that is neither efficient nor robust against rounding errors.
- What is your evaluation of the documentation?
The generated documentation looks nice from an optical point of view, but this is probably the place that still needs improvement.
- What is your evaluation of the potential usefulness of the library?
The library in its current form seems to be concerned with fixed size vectors and matrices. In a certain sense, I like libraries with a clear scope. Previous discussions indicated the need for such a library, so why not? The C++ language has limited built in support for fixed sized multi_arrays like double[2][3][4]. The library could be used to wrap types like double[2] and double[2][3], but not types like double[2][3][4]. But then we are back to the question of the scope of the library. The Boost.Array library is essentially a wrapper around T[N] and is considered useful. Now vec.hpp (la::vec<T, D>) provides a wrapper around T[D] and mat.hpp (la::mat<T, Rows, Cols>) provides a wrapper around T[Rows][Cols]. The traits system of the library probably even allows to adapt the raw T[Rows][Cols] type without using the la::mat<T, Rows, Cols> wrapper class. And the library provides quite some functionality in addition to simple wrapping, so I guess I have to conclude that the library is useful. Providing wrappers for T[D1][D2][D3] and T[D1][D2][D3][D4] , ..., T[D1][D2][D3][D4][D5][D6][D7][D8][D9] might also be useful sometimes (and the code generators might help here...), but this is the question of the scope of the library again.
- Did you try to use the library? With what compiler? Did you have any problems?
No, I didn't try to use the library.
- How much effort did you put into your evaluation? A glance? A quick reading? In-depth study?
A glance.
- Are you knowledgeable about the problem domain?
No, I don't even know what problem domain the library tries to address. Regards, Thomas

On Thu, Oct 22, 2009 at 3:15 PM, Thomas Klimpel <Thomas.Klimpel@synopsys.com> wrote:
So I decided to review the library so that I can also comment on the positive things. I noticed that my review is not free from unfriendly remarks, but there are hopefully enough friendly words to compensate for these.
Thanks for your time, I do appreciate it.
A possible improvement might be more explicit about the requirements of the value types of the matrices.
I realize that this page got buried in the documentation, even I had trouble finding it yet I know I wrote it. :) I'll have to fix this but here's a direct link: http://www.revergestudios.com/boost-la/scalar_requirements.html This area definitely needs work, because I'm not 100% sure those requirements are entirely correct. They could be wrong because while I have been using a previous version of this library for quite a while, I've not implemented a custom scalar type that's compatible with the library, so that part has never been tested.
- What is your evaluation of the implementation?
The implementation looks quite polished. I was impressed by the complete suite of unit-tests and the code generators. I've seen that there were even code generators for matrix_determinant and matrix_inverse. However, I think that these two code generators are false friends, because they implement an analytic formula that is neither efficient nor robust against rounding errors.
For determinants, I wanted to have at least the 2x2 3x3 and 4x4 versions done efficiently because those are used in 2D and 3D graphics. The formula is certainly not efficient, but (perhaps a bit naively?) I relied on compiler optimizations: the generated functions pull all scalars in local variables, which should untie the optimizer to figure out which series of multiplication/addition operations are identical and can be computed only once.
The traits system of the library probably even allows to adapt the raw T[Rows][Cols] type without using the la::mat<T, Rows, Cols> wrapper class.
The wrappers are necessary to support return-by-value operations (you can't copy an array.) That said, none of the supported operations require their *arguments* to be copyable, so you can use any of them with plain C arrays out of the box. Other non-copyable types are used by the view proxy system (in fact those types can't even be instantiated much less copied.)
Providing wrappers for T[D1][D2][D3] and T[D1][D2][D3][D4] , ..., T[D1][D2][D3][D4][D5][D6][D7][D8][D9] might also be useful sometimes (and the code generators might help here...), but this is the question of the scope of the library again.
I'm not sure why we're talking about 3D arrays, those can't possibly represent vectors or matrices. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski wrote:
For determinants, I wanted to have at least the 2x2 3x3 and 4x4 versions done efficiently because those are used in 2D and 3D graphics. The formula is certainly not efficient, but (perhaps a bit naively?) I relied on compiler optimizations: the generated functions pull all scalars in local variables, which should untie the optimizer to figure out which series of multiplication/addition operations are identical and can be computed only once.
FWIW, I read somewhere (can try to dig up reference if desired) that 4x4 (and 5x5) determinants are most efficiently calculated via dynamic programming, so you might consider explicitly implementing that rather than relying on the compiler. Also, signed volumes of small matrices might be useful, and I believe they can be implemented more efficiently than simply taking the determinant of a matrix adjoined with 1's.
I'm not sure why we're talking about 3D arrays, those can't possibly represent vectors or matrices.
I just have to take issue with the strong "can't" here ;) It is certainly imaginable that a 3D array could represent a linear operator from 2D arrays to 1D arrays, or from 1D arrays to 2D arrays. Using multidimensional arrays as your vectors might be more convenient than flattening them for a given application. - Jeff

On Thu, Oct 22, 2009 at 11:13 PM, Jeffrey Hellrung <jhellrung@ucla.edu> wrote:
Emil Dotchevski wrote:
For determinants, I wanted to have at least the 2x2 3x3 and 4x4 versions done efficiently because those are used in 2D and 3D graphics. The formula is certainly not efficient, but (perhaps a bit naively?) I relied on compiler optimizations: the generated functions pull all scalars in local variables, which should untie the optimizer to figure out which series of multiplication/addition operations are identical and can be computed only once.
FWIW, I read somewhere (can try to dig up reference if desired) that 4x4 (and 5x5) determinants are most efficiently calculated via dynamic programming, so you might consider explicitly implementing that rather than relying on the compiler.
Also, signed volumes of small matrices might be useful, and I believe they can be implemented more efficiently than simply taking the determinant of a matrix adjoined with 1's.
I'm not sure why we're talking about 3D arrays, those can't possibly represent vectors or matrices.
I just have to take issue with the strong "can't" here ;) It is certainly imaginable that a 3D array could represent a linear operator from 2D arrays to 1D arrays, or from 1D arrays to 2D arrays. Using multidimensional arrays as your vectors might be more convenient than flattening them for a given application.
Oh from this point of view yes, the library does allow you to get a vector out of a given column or row of a matrix, you just go m|col<2> and you get a vector type (without actually creating a temporary.) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski wrote:
Oh from this point of view yes, the library does allow you to get a vector out of a given column or row of a matrix, you just go m|col<2> and you get a vector type (without actually creating a temporary.)
What's the trick behind that ? -- ___________________________________________ Joel Falcou - Assistant Professor PARALL Team - LRI - Universite Paris Sud XI Tel : (+33)1 69 15 66 35

On Fri, Oct 23, 2009 at 11:27 AM, joel <joel.falcou@lri.fr> wrote:
Emil Dotchevski wrote:
Oh from this point of view yes, the library does allow you to get a vector out of a given column or row of a matrix, you just go m|col<2> and you get a vector type (without actually creating a temporary.)
What's the trick behind that ?
The op| overload takes m by reference, and returns the same reference reinterpret_casted to reference to an instance of this template: template <int Col,class OriginalMatrix> class col_; //non-copyable Then there is this partial specialization of the vector_traits template: template <int Col,class OriginalMatrix> struct vector_traits< col_<Col,OriginalMatrix> > { ... }; The element access functions in that specialization reinterpret_cast the col_<> reference they get back to OriginalMatrix reference and then forward the call to the access functions found in matrix_traits<OriginalMatrix>. There is a bunch of such op| overloads in (Boost) LA, and I wouldn't be shy about adding more: http://www.revergestudios.com/boost-la/view_proxy.html. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

This is mind blowing at the power of awesome. I had some similar features but my view are mere Range with different access methods. I have to investigate this in my own code. -- ___________________________________________ Joel Falcou - Assistant Professor PARALL Team - LRI - Universite Paris Sud XI Tel : (+33)1 69 15 66 35

Other question regarding implementation. Did you measure real performance increase by specifying those forceinline directives ? I tend to use them but I have hard time justifying this usage -- ___________________________________________ Joel Falcou - Assistant Professor PARALL Team - LRI - Universite Paris Sud XI Tel : (+33)1 69 15 66 35

On Sun, Oct 25, 2009 at 2:44 AM, joel <joel.falcou@lri.fr> wrote:
Other question regarding implementation. Did you measure real performance increase by specifying those forceinline directives ? I tend to use them but I have hard time justifying this usage
What specific usage do you refer to? Note that any use of forceinline is through other documented, possibly user-defined macros. The relevant macro is BOOST_LA_INLINE_CRITICAL. It is unspecified which functions use it but it is a safe bet that to get any reasonable performance, those functions should end up inlined. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski wrote:
Thomas Klimpel wrote:
Providing wrappers for T[D1][D2][D3] and T[D1][D2][D3][D4] , ..., T[D1][D2][D3][D4][D5][D6][D7][D8][D9] might also be useful sometimes (and the code generators might help here...), but this is the question of the scope of the library again.
I'm not sure why we're talking about 3D arrays, those can't possibly represent vectors or matrices.
I completely agree that a 3D array can't possibly represent vectors or matrices. The wrapper for "T[D1][D2][D3]" would be "la::ten3<T, D1, D2, D3>", and its traits would be ten3_traits, and the wrapper for T[D1][D2][D3][D4][D5][D6][D7][D8][D9] would be "la::ten9<T, D1, D2, D3, D4, D5, D6, D7, D8, D9]>". The reason we are talking about 3D arrays is that the C++ language has fixed-size higher dimensional multi_arrays, but only very limited language support for them. Now I have no strong opinions about whether there is a need for such wrappers or not. I searched my source code for occurrences of higher dimensional fixed size arrays, and found only very few occurrences. The occurrences were of the type float points[2][2][2] and float derivatives[2][2][2][3], and represented values in the corners of the unit-cube. Just tell me that higher dimensional tensors are clearly out of scope of the library, and everything is fine. Regards, Thomas

On Fri, Oct 23, 2009 at 3:28 AM, Thomas Klimpel <Thomas.Klimpel@synopsys.com> wrote:
Just tell me that higher dimensional tensors are clearly out of scope of the library, and everything is fine.
Ah I see. The honest answer is, I don't know, I have never used tensors. You can ask another similar question: what if someone needs vectors but not matrices? This is not an abstract question, I have written such programs. I have been very careful in organizing the header files -- if you don't include matrix headers from (Boost) LA, you don't get any matrix stuff in your code (and if this statement is ever found to be wrong, I would treat that situation as a bug.) I would say that *if* tensors can be implemented without adding *any* weight to a use case where they are not needed, then perhaps they should be part of the library. From this abstract point of view, I am not sure I have a reasonable answer to the question "why stop at 2." However I don't feel qualified to make that call. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski escribió:
I remember some past discussions on this mailing list about the need for similar library in Boost, so I hope it is OK to skip the interest query and request a preliminary review for Boost LA.
Boost LA provides a set of generic linear algebra algorithms, primarily for working with vectors and matrices of static size. The library also defines vector and matrix data types, however it allows users to introduce their own types by specializing the vector_traits and matrix_traits templates.
Documentation and source code available at http://www.revergestudios.com/boost-la, released under the Boost Software License.
Thanks, Emil Dotchevski
I'm considering this library for use in a future project. Mainly in order to operate with C arrays as vectors. I try using it for a few minutes and got the following concerns, sorry if they were brought up already: · Operators don't work unless boost::la namespace is bring into scope, couldn't ADL kick in? · Could swizzling 'placeholders' be placed on a different namespace, so I could write `using namespace boost::la::swizzling;` or similar. Everything else looks nice so far. Hope you continue working on it whether it gets into boost or not, its a really useful library! Thank you. Agustín K-ballo Bergé.-

2009/11/21 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
I'm considering this library for use in a future project. Mainly in order to operate with C arrays as vectors. I try using it for a few minutes and got the following concerns, sorry if they were brought up already:
· Operators don't work unless boost::la namespace is bring into scope, couldn't ADL kick in?
No, ADL can not kick in because the overloads are in namespace boost::la and the user-defined types they are typically called with are in other namespaces. You can safely bring boost::la into scope globally, the overloads this will introduce will not clash with anything.
· Could swizzling 'placeholders' be placed on a different namespace, so I could write `using namespace boost::la::swizzling;` or similar.
This is a good idea. I am planning to separate the operator overloads in a different namespace from everything else, since the operator overloads are always safe to bring into scope with using, yet ADL doesn't work for them.
Everything else looks nice so far. Hope you continue working on it whether it gets into boost or not, its a really useful library!
Thanks, I am hoping to get some more feedback, then I think I'll do a what will probably be a very light refactoring. After that I'll request a review and we'll see. :) Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski escribió:
2009/11/21 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
· Operators don't work unless boost::la namespace is bring into scope, couldn't ADL kick in?
No, ADL can not kick in because the overloads are in namespace boost::la and the user-defined types they are typically called with are in other namespaces. You can safely bring boost::la into scope globally, the overloads this will introduce will not clash with anything.
I should have been more specific, I was expecting the operators to just work for la::vref and la::mref. And probably for any other view too. Agustín K-ballo Bergé.-

2009/11/22 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
Emil Dotchevski escribió:
2009/11/21 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
· Operators don't work unless boost::la namespace is bring into scope, couldn't ADL kick in?
No, ADL can not kick in because the overloads are in namespace boost::la and the user-defined types they are typically called with are in other namespaces. You can safely bring boost::la into scope globally, the overloads this will introduce will not clash with anything.
I should have been more specific, I was expecting the operators to just work for la::vref and la::mref. And probably for any other view too.
Oh, vref/mref and other views will find the operator overloads through ADL. If you have a case where this doesn't work, please post some code and I'll take a look. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode

Emil Dotchevski escribió:
2009/11/22 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
I should have been more specific, I was expecting the operators to just work for la::vref and la::mref. And probably for any other view too.
Oh, vref/mref and other views will find the operator overloads through ADL. If you have a case where this doesn't work, please post some code and I'll take a look.
#include <boost/la/all.hpp> int main() { //using namespace boost::la; float vector[3] = { 1, 2, 3 }; boost::la::vref( vector ) *= 2; ( boost::la::vref( vector ) | boost::la::X ) = 0; return 0; } Both operations fail unless `using namespace boost::la`. ADL has nothing to do since vref returns a `boost::la::la_detail::vref_<T>` and operators are defined at `boost::la`. I'm using the version refered to by your original post, is there an update somewhere? Agustín K-ballo Bergé.- http://talesofcpp.blogspot.com

2009/11/22 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
Emil Dotchevski escribió:
2009/11/22 Agustín K-ballo Bergé <kaballo86@hotmail.com>:
I should have been more specific, I was expecting the operators to just work for la::vref and la::mref. And probably for any other view too.
Oh, vref/mref and other views will find the operator overloads through ADL. If you have a case where this doesn't work, please post some code and I'll take a look.
#include <boost/la/all.hpp>
int main() { //using namespace boost::la;
float vector[3] = { 1, 2, 3 };
boost::la::vref( vector ) *= 2; ( boost::la::vref( vector ) | boost::la::X ) = 0;
return 0; }
Both operations fail unless `using namespace boost::la`. ADL has nothing to do since vref returns a `boost::la::la_detail::vref_<T>` and operators are defined at `boost::la`.
Ah, yes! I'm sorry I had a brain fart. I'll think what's a good way to fix this, it seems desirable for ADL to lead to the generic operator overloads in this case.
I'm using the version refered to by your original post, is there an update somewhere?
No you're doing everything right. Thanks for the feedback. Emil Dotchevski Reverge Studios, Inc. http://www.revergestudios.com/reblog/index.php?n=ReCode
participants (9)
-
Agustín K-ballo Bergé
-
Emil Dotchevski
-
Emil Dotchevski
-
Jeffrey Hellrung
-
joel
-
Marco Guazzone
-
Marcus Lindblom
-
Robert Ramey
-
Thomas Klimpel