[boost] n1803/n1982 “Simple Numeric Access” proposal is broken

18 May 2006

      n1803/n1982 “Simple Numeric Access” proposal is broken

I’ve implemented “long stol(string& str, int base = 10)” function from 
n1982 proposal (see code at the bottom of this message) and faced some 
issues:

issue 1: non-const version (“string& str”) breaks code like
long l = stol("10");
usage is limited to:
string s = "10";
long l = stol(s);

issue 2: "function call strtol(str.c_str(), 0, base)" requirement can 
not be fulfilled.
it conflicts with "erases the characters from the front of str that were 
converted to get the result" and with "Throws: invalid_argument if 
strtol, strtoul, strtoll, or strtoull reports that no conversion could 
be performed"
requirements (error checking prevents usage of '0' for 2nd argument)

issue 3: simple performance test shows that the function proposed is 
1000 times slower compared to iostreams solution (number of samples for 
test were 100000)
Platform: WinXP VC71 STLPort 4.6.2

all these issues (except for 2) could be solved by removing the "... 
erase the characters from the front of str that were converted to get 
the result. ..." requirement.
issue 2 has a trivial solution: replace "function call 
strtol(str.c_str(), 0, base)" with "function call strtol with 
appropriate arguments".

the fixed code is given in “long stol_fixed(string const& str, int base 
= 10)” function.

this stol_fixed function is less powerful than the originally version 
proposed (if we forgot about issues mentioned). it disallows parsing of 
multiple numbers from one long string. the C-library’s strtol function 
allows it but if all error conditions should be checked the code becomes 
too complex for simple parsing task.

On the other side, all issues with the stol function proposed comes from 
an attempt to merge in one ‘simple’ interface both needs: (1) parsing of 
multiple numbers from one string; (2) parsing one number from string.
The (2) can be accomplished with simple function interface, but the (1) 
should keep a state (pointer to beginning of a char-sequence to parse 
from). In the n1982 proposal it was established be erasing parsed 
characters, but it is too expensive, because of memory [de]allocations 
caused.

It seems that stream-like interface would be good for the (1), for example:
std::string s(“10 20 30 40”);
cistream cs(s);
long l;
cs >> l;
int i;
cs >> i;
long arr[10];
for(int i = 0; i < 10; ++i) {
   cs >> arr[i]; // exception would be thrown (read after eof)
}

or, better:
std::string s(“10 20 30 40”);
cistream cs(s);
long l = cs.read_long(); //convenience function
int i = cs.read_int(); //convenience function
long arr[10];
for(int i = 0; i < 10; ++i) {
   cs >> arr[i]; // exception would be thrown (read after eof)
}

This solution would support:
cistream, costream, ciostream
wcistream, wcostream, wciostream

One more issue with n1982 proposal and the one I’ve proposed in this 
group is that both make invisible for C++ programmer that he uses 
wrappers around C-library I/O functions, that are incompatible with C++ 
locales. The solution would be to code this in components names (like 
the ‘c’ symbol in names above).

NOTE: with from_string function that was proposed earlier by me
the “parsing one number from string” task with use of C-library 
functions can be implemented as from_string_c function.
It can not be safely merged with from_string function, because of C vs. 
C++ locale issues. Same applies to (w)string_from function. it’s 
C-wrapper counterpart would be (w)string_from_c function. Better naming 
suggestions are welcome.
(_byc suffix?)

Best,
Oleg Abrosimov.

// code begins

#include <locale>
#include <cstdlib>
#include <cerrno>
#include <string>

// 1) "12 23 34 34 56 78" - stream-like interface is the best in C++
// 2) " 12" - conversion function is appropriate
namespace std { namespace tr2 {

   // issue 1: non-const version breaks code like
   // "long l = stol("10");"
   // usage is limited to:
   // string s = "1 2";
   // long l = stol(s);
   // long l1 = stol(s);
   //long stol(string const& str, int base = 10)
   long stol(string& str, int base = 10)
   {
       char* endptr = 0;
       const char* nptr = str.c_str();
       long res = strtol(nptr, &endptr, base);
       if (endptr == nptr) {
           throw std::invalid_argument("stol invalid argument. str = '" 
+ str + "'");
       }
       if (errno == ERANGE) {
           switch(res) {
               case LONG_MAX :
                   throw std::overflow_error("stol overflow. str = '" + 
str + "'");
                   break;
               case LONG_MIN :
                   throw std::underflow_error("stol underflow. str = '" 
+ str + "'");
                   break;
           }
       }
       // performance killer !!!!!!!
       str.erase(0, endptr - nptr);
       return res;
   }
   // issue 2: "function call strtol(str.c_str(), 0, base)" requirement 
can not be fulfilled
   // it conflicts with "erases the characters from the front of str 
that were converted to get the result"
   // and with "Throws: invalid_argument if strtol, strtoul, strtoll, or 
strtoull reports that no conversion could be performed"
   // requirements (error cheking prevents usage of '0' for 2nd argument)

   long stol_fixed(string const& str, int base = 10)
   {
       char* endptr = 0;
       const char* nptr = str.c_str();
       long res = strtol(nptr, &endptr, base);
       if (endptr == nptr) {
           throw std::invalid_argument("stol invalid argument. str = '" 
+ str + "'");
       }
       if (errno == ERANGE) {
           switch(res) {
               case LONG_MAX :
                   throw std::overflow_error("stol overflow. str = '" + 
str + "'");
                   break;
               case LONG_MIN :
                   throw std::underflow_error("stol underflow. str = '" 
+ str + "'");
                   break;
           }
       }
       return res;
   }
}}

// performance test code begins
// it uses profiler code by Christopher Diggins
#include <iostream>
#include <sstream>
#include <limits>
#include <vector>
#include <boost/profiler.hpp>
#include <boost/lexical_cast.hpp>

#ifdef max
#undef max
#endif
#ifdef min
#undef min
#endif

// 1) create a string of long values
// 2) read em using std::stringstream
// 3) same with stol()
int main()
try {
   // 1) create a string of long values
   const long lMax = std::numeric_limits<long>::max();
   const long lCount = 1000000L;
   const long lMin = std::numeric_limits<long>::max() - lCount;
   std::string sLongs;
   std::vector<std::string> vecLongs;
   vecLongs.reserve(lCount);
   for (long l = lMax; l > lMin; --l) {
       std::string s = boost::lexical_cast<std::string>(l);
       sLongs += (s + '\t');
       vecLongs.push_back(s);
   }
   sLongs = sLongs.substr(0, sLongs.length() - 1);

   // 2) read em using std::stringstream
   {
       boost::prof::profiler p(": read em using std::stringstream");
       std::istringstream ss(sLongs);
       long l;
       volatile vl;
       while (!sLongs.empty() && ss.good() && !ss.eof()) {
           ss >> l;
           volatile const char* nptr = sLongs.c_str();
           // uncomment it to simulate std::tr2::stol timings
           //sLongs.erase(0, 11);
           vl = l;
       }
   }

   // 3) same with stol_fixed()
   // 3.6 times faster then (2)
   {
       boost::prof::profiler p("same with stol_fixed()");
       long l;
       volatile vl;
       for(long i = 0; !sLongs.empty() && i < lCount; ++i) {
           l = std::tr2::stol_fixed(vecLongs[i]);
           vl = l;
       }
   }

   // 4) same with stol()
   // 1000 times slower then iostreams solution
   {
       boost::prof::profiler p("same with stol()");
       long l;
       volatile vl;
       while (!sLongs.empty()) {
           l = std::tr2::stol(sLongs);
           vl = l;
       }
   }

} catch (std::exception& ex) {
   std::cerr << ex.what() << std::endl;
} catch (...) {
   std::cerr << "Unknown exception occured" << std::endl;
}