On Feb 17, 2016 5:21 AM, "Michael Marcin" wrote:

FWIW I can't reproduce your results on Visual Studio 2015 Update 1. Here are my results if you're interested.

Thanks for checking! (You are the first to compile it on Windows, I'm surprised it worked) Looking at your results: although now std::vector wins, std::deque awfully falls behind. I don't know the std library supplied by MSVC, but I suspect this isn't the expected result. I matched the segment size to libstdc++, maybe dinkumware uses smaller segments and allocates more. Thanks, Benedek

I think there is definitely something not quite right. One thing I notice is that the line: sample.push_back(get_clock() - base_clock); Is being timed. Which could potentially access a new memory page I think. In pin_thread you have _MSVC_VER instead of _MSC_VER Hacking up std::vector and adding a push_back_reserved that does no bounds checks I get consistently very strange timings. (Code included at the bottom to make it obvious that the latter should be no slower). X std::vector::push_back 0 0 100 765.360000 200 1537.260000 400 3037.950000 800 6008.280000 1600 11905.150000 3200 23950.320000 6400 48064.600000 12800 96178.470000 25600 190637.270000 51200 379025.420000 102400 757784.880000 204800 1514584.120000 409600 3026480.680000 819200 6050654.130000 X std::vector::push_back_reserved 0 0 100 938.090000 200 1936.380000 400 3704.400000 800 7274.170000 1600 14307.310000 3200 28550.140000 6400 56835.310000 12800 113579.430000 25600 226906.070000 51200 453802.040000 102400 911835.760000 204800 1819723.520000 409600 3639619.650000 819200 7281897.560000 My simpler, and I'm sure in many ways inferior benchmark, shows much more believable results. http://codepad.org/SBZ4rbqo Running 1 test case... std::vector::push_back_reserved 4.88 std::vector::push_back 6.31 bst::devector::push_back 5.37 bst::vector::push_back 7.22 void push_back(value_type&& _Val) { // insert by moving into element at end if (_Inside(_STD addressof(_Val))) { // push back an element size_type _Idx = _STD addressof(_Val) - this->_Myfirst(); if (this->_Mylast() == this->_Myend()) _Reserve(1); _Orphan_range(this->_Mylast(), this->_Mylast()); this->_Getal().construct(this->_Mylast(), _STD forward(this->_Myfirst()[_Idx])); ++this->_Mylast(); } else { // push back a non-element if (this->_Mylast() == this->_Myend()) _Reserve(1); _Orphan_range(this->_Mylast(), this->_Mylast()); this->_Getal().construct(this->_Mylast(), _STD forward(_Val)); ++this->_Mylast(); } } void push_back_reserved( value_type&& _Val ) { this->_Getal().construct( this->_Mylast(), _STD forward( _Val ) ); ++this->_Mylast(); }

On 2/18/2016 7:22 AM, Thorsten Ottosen wrote:

On 18-02-2016 08:51, Michael Marcin wrote:

...

I think there is definitely something not quite right.

One thing I notice is that the line: sample.push_back(get_clock() - base_clock); Is being timed. Which could potentially access a new memory page I think.

In pin_thread you have _MSVC_VER instead of _MSC_VER

I don't understand why its necessary with all this complicated stuff. AFAICT, a single threaded program runs on one CPU.

FWIW http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-...

On 2/19/2016 12:48 AM, Benedek Thaler wrote:

On Thu, Feb 18, 2016 at 8:51 AM, Michael Marcin wrote:

...

I think there is definitely something not quite right.

One thing I notice is that the line: sample.push_back(get_clock() - base_clock); Is being timed. Which could potentially access a new memory page I think.

In pin_thread you have _MSVC_VER instead of _MSC_VER

Good catch, fixed this.

...

Hacking up std::vector and adding a push_back_reserved that does no bounds checks I get consistently very strange timings. (Code included at the bottom to make it obvious that the latter should be no slower).

My simpler, and I'm sure in many ways inferior benchmark, shows much more believable results.

Still, there are huge differences between essentially same programs.

...

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-...

I gave a spin to rdtscp, no success, but I continue looking at this doc.

Thanks, Benedek

FWIW (probably not much this time) I took a shot at porting my test to the google/benchmark library. Mostly just to learn the library but also to see if there were any glaring discrepancies. Not sure I'm using it right but the relative results seem in line with my expectations. results: http://codepad.org/U66O82X8 source: http://codepad.org/rii3BHly

On Wed, Feb 17, 2016 at 5:34 PM, Thorsten Ottosen wrote:

Some more questions: A. is the lambda optimized away?

It is. I got the same results without any helper functions.

B. Do the test run the same number of times? Is it asserted at runtime?

It does, just checked it manually.

C. Do you compile via bjam? I mean, is all the right flags present, e.g. NDEBUG=1?

I use bjam. I have -DNDEBUG on the compiler invocation line. I also have -O3. I wonder why the release build have -O3 instead of -O2 by default.

D. How does the actual code between std::vector and devector compare? Maybe there could be hints to the difference.

I found nothing indicative.

I don't understand why its necessary with all this complicated stuff. AFAICT, a single threaded program runs on one CPU.

A single threaded program can be preempted, and migrated to an other CPU later. Since I'm using TSC and processor isolation, it's important to always run the program on the same CPU core.

If you do a benchmark without reserve, you should configure devector to use the same growth factor/initial size as the std/boost equivalent.

Currently, vector tests use reserve, the growth policy is not used.

This is platform dependent, so one test per library is needed.

Regarding queues, I matched the segment size to my platform (512 elems), after the issue above is resolved, we can have tests for different libs, yes. Thanks for checking, Benedek