[system] patch for access violation on FormatMessage failure

I ran across a case this morning where the FormatMessage Win32 call in boost::system_error_category::message() fails (the error message table did not have an entry for the error code that was passed). Unfortunately, when this happens lpMsgBuf is not initialized to anything and trying to use it can result in an access violation. The attached patch adds checks to catch situations where FormatMessage fails. Thanks, -Dave Index: error_code.cpp =================================================================== --- error_code.cpp (revision 40686) +++ error_code.cpp (working copy) @@ -347,7 +347,7 @@ { # ifndef BOOST_NO_ANSI_APIS LPVOID lpMsgBuf; - ::FormatMessageA( + DWORD retval = ::FormatMessageA( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, @@ -358,11 +358,14 @@ 0, NULL ); + if (retval == 0) + return std::string(); + std::string str( static_cast<LPCSTR>(lpMsgBuf) ); ::LocalFree( lpMsgBuf ); // free the buffer # else // WinCE workaround LPVOID lpMsgBuf; - ::FormatMessageW( + DWORD retval = ::FormatMessageW( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, @@ -373,10 +376,13 @@ 0, NULL ); + if (retval == 0) + return std::string(); int num_chars = (wcslen( static_cast<LPCWSTR>(lpMsgBuf) ) + 1) * 2; LPSTR narrow_buffer = (LPSTR)_alloca( num_chars ); - ::WideCharToMultiByte(CP_ACP, 0, static_cast<LPCWSTR>(lpMsgBuf), -1, narrow_buffer, num_chars, NULL, NULL); + if (::WideCharToMultiByte(CP_ACP, 0, static_cast<LPCWSTR>(lpMsgBuf), -1, narrow_buffer, num_chars, NULL, NULL) == 0) + return std::string(); std::string str( narrow_buffer ); ::LocalFree( lpMsgBuf ); // free the buffer

David Deakins wrote:
I ran across a case this morning where the FormatMessage Win32 call in boost::system_error_category::message() fails (the error message table did not have an entry for the error code that was passed).
Grrr... That's what I get for taking some Microsoft example code at face value.
Unfortunately, when this happens lpMsgBuf is not initialized to anything and trying to use it can result in an access violation. The attached patch adds checks to catch situations where FormatMessage fails.
Thanks! Much appreciated! Before apply the patch, I'll try to figure out a a test case or two. Out of curiosity, what was the code that failed? Was it actually a valid code? --Beman
participants (2)
-
Beman Dawes
-
David Deakins