[Iostreams] Can't open mapped_file data file on Windows
Hi all, I'm trying to use mapped_file from the Boost 1.35.0 Iostreams library to open a data file for reading and writing. My code works fine under Mac OS X, which is posix, of course. But under Windows (built with VS2005), I get an access error. Here's a code sample that works on Mac OS X and fails under Windows: --snip-- /** Test boost iostream mapped_file. */ static int BoostMappedFileTest(void) { int result = 0; std::cout << "\nRunning BoostMappedFileTest..." << std::endl; // Initialize our test directory fs::path testDirPath( "BoostMappedFileTest" ); create_directory(testDirPath); // Initialize a test file buffer with some data const size_t kTestFileSize = 4096; boost::scoped_array<char> fileBuffer( new char[kTestFileSize] ); char* pCur = fileBuffer.get(); char curChar = 0; for( size_t i = 0; i < kTestFileSize; ++i ) { *pCur++ = curChar++; } // Write the data to a test file. Be sure to explicitly close it. fs::path testFilePath( testDirPath / "test_mapped_file" ); boost::filesystem::ofstream testFile( testFilePath, ios::trunc | ios::binary ); testFile.write(fileBuffer.get(), kTestFileSize); testFile.close(); // Try and open the test file as a memory mapped file. string testFilePathStr = testFilePath.file_string(); mapped_file mappedFile(testFilePathStr); // We never get here under Windows with boost 1.35.0. The file fails to // open with an access error, although it throws with a more obscure error // due to a clean up bug. // Compare the mapped file data to our original buffer const char* mmConstData = mappedFile.const_data(); bool compareSucceeded = (0 == memcmp(fileBuffer.get(), mmConstData, kTestFileSize)); if (!compareSucceeded) { throw runtime_error("The boost memory mapped file data doesn't match our buffer!"); } return result; } --snip-- The problem seems to come from passing GENERIC_ALL as the "dwDesiredAccess" parameter to the CreateFileA call. If I change the implementation to pass GENERIC_READ | GENERIC_WRITE, then it works in my test scenario. Please note that I haven't tried pointing the current code at an actual executable to see if it works with GENERIC_ALL. If you assume a strict interpretation of the mapped_file API, the mode of BOOST_IOS::in would map to GENERIC_READ in the Windows world, and BOOST_IOS::out would map to GENERIC_WRITE. There is no corresponding "execute" flag that I can see in this API. Since I'm only interested in mapping data files, I could certainly modify the boost 1.35.0 sources so that only GENERIC_READ and GENERIC_WRITE are passed to the Windows API depending on what BOOST_IOS mode was specified. But I'm not sure of what dependancies may exist between the mapped_file implementation and other boost libraries. The interprocess library, for example, hasn't yet fallen under my sphere of research ;-) Could someone please advise me as to the best course of action that will allow me to open memory mapped data files on Windows without adversely impacting the rest of the boost libraries? BTW, there's a secondary problem where if an error happens at CreateFileA, the clean up code fails and throws an invalid handle exception. This is because the mapped_handle_ member variable is initialized to NULL instead of INVALID_HANDLE_VALUE (which is what the clean up test is looking for). After code inspection of the svn trunk, I *think* this secondary issue has already been resolved. However, the latest sources look like they're still using GENERIC_ALL as the access parameter to the CreateFile, so my reported problem is still relevant and needs to be resolved. Thanks in advance for any suggestions. Best, -- Allen Cronce
No one responded to my email regarding problems using mapped_file with data files under Windows. But I need to move forward, so I've worked around the problem by changing the implementation of mapped_file.cpp in our boost 1.36.0 vendor drop: --snip-- //--------------Open underlying file--------------------------------------// // GENERIC_ALL causes an access error opening existing data files for writing. // PAC 31-Aug-2008. DWORD dwAccess = readonly ? GENERIC_READ : (GENERIC_READ | GENERIC_WRITE); pimpl_->handle_ = ::CreateFileA( p.path.c_str(), dwAccess, FILE_SHARE_READ, NULL, (p.new_file_size != 0 && !readonly) ? CREATE_ALWAYS : OPEN_EXISTING, readonly ? FILE_ATTRIBUTE_READONLY : FILE_ATTRIBUTE_TEMPORARY, NULL ); --snip-- As indicated before, I don't know what other libraries, if any, depend on mapped_file, so this fix might not be the right one. I'm hoping that whoever maintains Iostreams will look into this issue and comment. Further, I think that the unit test for Iostreams should be extended to catch this kind of problem. Basically the test code should do something like the code from my original post. That is, it should create a test data file, then open it as a mapped file for writing. I'm kind of new to boost, so I'm not clear on the process for dealing with bugs or what my role should be. Should I report this issue on <http://svn.boost.org/trac/boost/report
? Or is the expectation that I should be more proactive and fix it myself in a bleeding edge branch?
Thanks in advance for any suggestions. Best, -- Allen Cronce On Aug 28, 2008, at 8:31 PM, Allen Cronce wrote:
Hi all,
I'm trying to use mapped_file from the Boost 1.35.0 Iostreams library to open a data file for reading and writing. My code works fine under Mac OS X, which is posix, of course. But under Windows (built with VS2005), I get an access error.
Here's a code sample that works on Mac OS X and fails under Windows:
--snip--
/** Test boost iostream mapped_file. */ static int BoostMappedFileTest(void) { int result = 0;
std::cout << "\nRunning BoostMappedFileTest..." << std::endl;
// Initialize our test directory fs::path testDirPath( "BoostMappedFileTest" ); create_directory(testDirPath);
// Initialize a test file buffer with some data const size_t kTestFileSize = 4096; boost::scoped_array<char> fileBuffer( new char[kTestFileSize] ); char* pCur = fileBuffer.get(); char curChar = 0; for( size_t i = 0; i < kTestFileSize; ++i ) { *pCur++ = curChar++; }
// Write the data to a test file. Be sure to explicitly close it. fs::path testFilePath( testDirPath / "test_mapped_file" ); boost::filesystem::ofstream testFile( testFilePath, ios::trunc | ios::binary ); testFile.write(fileBuffer.get(), kTestFileSize); testFile.close();
// Try and open the test file as a memory mapped file. string testFilePathStr = testFilePath.file_string(); mapped_file mappedFile(testFilePathStr);
// We never get here under Windows with boost 1.35.0. The file fails to // open with an access error, although it throws with a more obscure error // due to a clean up bug.
// Compare the mapped file data to our original buffer const char* mmConstData = mappedFile.const_data(); bool compareSucceeded = (0 == memcmp(fileBuffer.get(), mmConstData, kTestFileSize)); if (!compareSucceeded) { throw runtime_error("The boost memory mapped file data doesn't match our buffer!"); }
return result; }
--snip--
The problem seems to come from passing GENERIC_ALL as the "dwDesiredAccess" parameter to the CreateFileA call. If I change the implementation to pass GENERIC_READ | GENERIC_WRITE, then it works in my test scenario. Please note that I haven't tried pointing the current code at an actual executable to see if it works with GENERIC_ALL.
If you assume a strict interpretation of the mapped_file API, the mode of BOOST_IOS::in would map to GENERIC_READ in the Windows world, and BOOST_IOS::out would map to GENERIC_WRITE. There is no corresponding "execute" flag that I can see in this API.
Since I'm only interested in mapping data files, I could certainly modify the boost 1.35.0 sources so that only GENERIC_READ and GENERIC_WRITE are passed to the Windows API depending on what BOOST_IOS mode was specified. But I'm not sure of what dependancies may exist between the mapped_file implementation and other boost libraries. The interprocess library, for example, hasn't yet fallen under my sphere of research ;-)
Could someone please advise me as to the best course of action that will allow me to open memory mapped data files on Windows without adversely impacting the rest of the boost libraries?
BTW, there's a secondary problem where if an error happens at CreateFileA, the clean up code fails and throws an invalid handle exception. This is because the mapped_handle_ member variable is initialized to NULL instead of INVALID_HANDLE_VALUE (which is what the clean up test is looking for).
After code inspection of the svn trunk, I *think* this secondary issue has already been resolved. However, the latest sources look like they're still using GENERIC_ALL as the access parameter to the CreateFile, so my reported problem is still relevant and needs to be resolved.
Thanks in advance for any suggestions.
Best, -- Allen Cronce
Allen Cronce wrote:
As indicated before, I don't know what other libraries, if any, depend on mapped_file, so this fix might not be the right one. I'm hoping that whoever maintains Iostreams will look into this issue and comment.
Further, I think that the unit test for Iostreams should be extended to catch this kind of problem. Basically the test code should do something like the code from my original post. That is, it should create a test data file, then open it as a mapped file for writing.
I'm kind of new to boost, so I'm not clear on the process for dealing with bugs or what my role should be. Should I report this issue on http://svn.boost.org/trac/boost/report? Or is the expectation that I should be more proactive and fix it myself in a bleeding edge branch?
I would recommend creating a new ticket which describes the problem at: http://svn.boost.org/trac/boost Then I would attach a patch of your proposed changes to the ticket with your proposed solution (and test code).
participants (2)
-
Allen Cronce -
eg