[parameter] efficiency testing
Last week, we were considering using the parameter library in some client software that demands high performance (they are doing massively parallel statistical analysis), so I sat with Matthias Troyer, an expert in high-performance C++, and developed the test that's now checked in at libs/parameter/test/efficiency.cpp. The test was carefully designed to avoid deceptive effects due to dead code elimination, constant propagation, and CPU architecture, and the process of developing it was highly instructive to me personally. We left lots of comments in the code so I'd be able to remember the issuse involved. I recommend reviewing it for anyone interested in how to measure abstraction penalty. One set of results is checked in at libs/parameter/test/timings.txt, and are reproduced below. As you can see, at least on that platform, you don't need to worry about Boost.Parameter slowing your code down; it might even speed your code up! We'd be very interested in seeing results produced with other compilers and/or platforms at the highest optimization levels. ----------------- Test #1 ---------------- Testers: Matthias Troyer, Michael Gauckler, David Abrahams Date: 2005-09-09 Compiler: Cygwin g++-4.0.0 Command lines: g++ -c -o efficiency.o -ftemplate-depth-100 -funroll-loops \ -O3 -finline-functions -Wno-inline -DNDEBUG efficiency.cpp g++ -o efficiency.exe efficiency.o -Wl,--strip-all efficiency && efficiency && efficiency Machine: Dell Inspiron 9300 Intel(R) Pentium(R) M processor 2.00GHz Running on 230 volts AC power Timings: Run #1 results discarded per standard practice Run #2 plain time: 0.75 named parameter time: 0.734 Run #3 plain time: 0.781 named parameter time: 0.766 ----------------------------------------- -- Dave Abrahams Boost Consulting www.boost-consulting.com
On Mon, Sep 12, 2005 at 06:25:18PM +0200, David Abrahams wrote:
Last week, we were considering using the parameter library in some client software that demands high performance (they are doing massively parallel statistical analysis), so I sat with Matthias Troyer, an expert in high-performance C++, and developed the test that's now checked in at libs/parameter/test/efficiency.cpp. The test
Interesting. Thanks for sharing it.
We'd be very interested in seeing results produced with other compilers and/or platforms at the highest optimization levels.
Here's a test from a machine I have handy: Testers: Chris Frey Date: 2005-09-17 Compiler: Linux kernel 2.4.27 gcc version 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8) Command lines: g++ -c -o efficiency.o -ftemplate-depth-100 -funroll-loops \ -O3 -finline-functions -Wno-inline -DNDEBUG efficiency.cpp g++ -o efficiency.exe efficiency.o -Wl,--strip-all Machine: IBM Thinkpad 770Z, running on AC power Intel Pentium II 366Mhz Timings: Run #1 results discarded per standard practice Run #2 plain time: 6.42 named parameter time: 7.34 Run #3 plain time: 6.42 named parameter time: 7.34 - Chris
Chris Frey <cdfrey@netdirect.ca> writes:
We'd be very interested in seeing results produced with other compilers and/or platforms at the highest optimization levels.
Here's a test from a machine I have handy:
Compiler: Linux kernel 2.4.27 gcc version 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8)
Timings: Run #1 results discarded per standard practice
Run #2 plain time: 6.42 named parameter time: 7.34
Run #3 plain time: 6.42 named parameter time: 7.34
Looks like GCC's optimizer got better at some point. I just tested with gcc-3.4.4 (on Cygwin) and got exactly the same time for both the plain and named parameter tests. However, everything's about twice as fast with gcc-4.0.0 :) -- Dave Abrahams Boost Consulting www.boost-consulting.com
On 9/12/05, David Abrahams <dave@boost-consulting.com> wrote:
Last week, we were considering using the parameter library in some client software that demands high performance (they are doing massively parallel statistical analysis), so I sat with Matthias Troyer, an expert in high-performance C++, and developed the test that's now checked in at libs/parameter/test/efficiency.cpp. The test was carefully designed to avoid deceptive effects due to dead code elimination, constant propagation, and CPU architecture, and the process of developing it was highly instructive to me personally. We left lots of comments in the code so I'd be able to remember the issuse involved. I recommend reviewing it for anyone interested in how to measure abstraction penalty. One set of results is checked in at libs/parameter/test/timings.txt, and are reproduced below.
As you can see, at least on that platform, you don't need to worry about Boost.Parameter slowing your code down; it might even speed your code up!
We'd be very interested in seeing results produced with other compilers and/or platforms at the highest optimization levels.
<snip>
-- Dave Abrahams Boost Consulting www.boost-consulting.com
Dave - I've just run this on an Athlon 64 3500+ laptop (on AC power) under Windows XP SP2, with Visual C++ 7.1 & 8 (beta 2) and gcc 3.4.2. I only used the -O2 (maximize speed) option on VC++ because that's meant to gather up the 'best' set of optimisations. I used the command line you usd for gcc. Stuart Dootson ---------- RESULTS ---------- Visual C++ 7.1 : command line: cl -O2 efficiency.cpp -EHsc -I\lib\boost\include\boost-1_33 Results: plain time: <ignored> named parameter time: <ignored> plain time: 1.453 named parameter time: 1.437 plain time: 1.453 named parameter time: 1.453 Visual C++ 8.0: command-line: cl -O2 efficiency.cpp -EHsc -I\lib\boost\include\boost-1_33 Results: plain time: <ignored> named parameter time: <ignored> plain time: 1.438 named parameter time: 1.453 plain time: 1.438 named parameter time: 1.437 Mingw gcc 3.4.2: command-line: g++ -c -o efficiency.o -ftemplate-depth-100 -funroll-loops -O3 -finline-functions -Wno-inline -DNDEBUG efficiency.cpp -I\lib\boost\include\boost-1_33 g++ -o efficiency.exe efficiency.o -Wl,--strip-all Results: plain time: <ignored> named parameter time: <ignored> plain time: 1.14 named parameter time: 1.422 plain time: 1.125 named parameter time: 1.406
Stuart Dootson <stuart.dootson@gmail.com> writes:
Dave - I've just run this on an Athlon 64 3500+ laptop (on AC power) under Windows XP SP2, with Visual C++ 7.1 & 8 (beta 2) and gcc 3.4.2. I only used the -O2 (maximize speed) option on VC++ because that's meant to gather up the 'best' set of optimisations. I used the command line you usd for gcc.
Stuart Dootson
Thanks, Stuart! I checked your results into timings.txt -- Dave Abrahams Boost Consulting www.boost-consulting.com
participants (3)
-
Chris Frey -
David Abrahams -
Stuart Dootson