On 1/9/2012 12:09 PM, Simonson, Lucanus J wrote:

I think that numerical optimization is more general than its application to machine learning. While you argue that you'd like to see the library go deep in machine learning I don't see anything wrong with a general purpose numerical optimization library. However, DE is more general than numerical optimization, while the library seems to be an application of DE to numerical optimization. I'm not sure that DE is even one of the best ways to do numerical optimization (I'm pretty sure genetic algorithm is a bad way, so I'm skeptical), and I know I'd like a more general interface to a DE engine than one designed for numerical optimization problems. I'd like to see the library go wide for max applicability rather than deep. Hopefully that will keep it out of all the different cans of worms that could be opened in each application domain. Put numerical optimization and machine learning applications in the example code of the library and submit just the core engine as the library itself. Narrowing the scope of a library submission is usually the path to success.

I forwarded Ken Price (one of the DE co-authors) some of the comments I received for his view on things and here's his reply, which hopefully should clarify some of the issues that have been raised: "The skeptic is right in believing that GAs have traditionally done poorly at numerical optimization, primarily because they rely on general data manipulation operations (bit flips, exchanges, rotations, insertions, replacements...) that are not appropriate for real-parameter optimization. DE, however, was specifically designed for numerical optimization (although it has since been applied with limited success to combinatorial problems). Instead of relying on general data manipulation operations, DE's unique differential mutation operation is a real-valued vector operation that is appropriate for the numerical optimization problem domain. More than just appropriate, the differential mutation operation at the core of DE is effective because it exploits the tendency of a population to distribute itself around function contours. Randomly sampling the difference in the location of population members when they are distributed along a function's level surfaces turns out to be a very effective way to deal with some of the most challenging aspects of numerical optimization, e.g. multiple optima, parameter-dependence, slow-down at high resolution and disparate parameter sensitivities. As evidence of DE's utility as a numerical optimizer, I would offer that it has been part of Mathematica's numerical optimization toolbox for over a decade. It helped the Mathematica team win the SIAM 100-digit challenge (ten problems to ten-digit accuracy) by solving what the contest participants agreed was the contest's most difficult (numerical optimization) problem. (DE also solved one of the other problems in the contest). In the many other contests that have been held to compare numerical optimizers, DE has consistently placed at or near the top. DE is also part of the OPTIMUS package, which is highly respected and in wide use by several automobile manufactures. Alternatively, skeptics can do an Internet search and discover for themselves the enormous range of engineering problems the DE has successfully solved, nearly all of which are numerical optimization problems, often with multiple, non-linear constraints. DE has also proven effective in the domain of multi-objective numerical optimization. Finally, I would suggest that since DE is so simple and easy to implement, skeptics ought to just try it. Most people are surprised by how effective something as simple as DE can be."