[boost] [nowide] Request for interest (nowide unicode support for windows)

13 Jun 2010

      Hello All,

I've recently written a small library that allows writing platform
independent Unicode aware applications transparently.

Problem, basic stuff like opening a file, deleting it, is generally
accepted for granted and indeed, STL provides std::fstream and C library
provides FILE* API like std::fopen, std::remove, std::rename.

However, this API is broken under Windows any time we talk about some
basic localization like using Unicode file names.

Unlike all other Windows development moved to redesigning API to wide
characters instead of adapting backward compatible UTF-8 locales into
the core system.

Result: it is total nightmare to write any kind of Unicode aware cross
platform programs.

Even when trying to use wide strings and calling _wfopen or _wremove
where needed is not enough, as there is no simple replacement for
std::fstream. MSVC provided non-standard extension where
std::fstream::open() receives wide string, but this is not accepted by
many other compilers including MinGW gcc that has shared libstdc++
over multiple platforms. Not talking about that standard does not
define std::fstream::open(wchar_t const *,...) (and not in Tr1 as well).

So... proposed solution (in short):

namespace boost {
  namespace nowide {
     #if !defined(BOOST_WIN32)

     using namespace std;

     #else // Windows Wide API

     std::wstring convert(std::string const &);
     std::string convert(std::wstring const &);

     FILE *fopen(char const *,char const *);
     FILE *freopen(char const *,char const *,FILE *);
     int remove(char const *);
     int rename(char const *,char const *);

     template<typename Char,typename Traits ...>
     basic_filebuf {
        ...
        open(char const *,...);
        ...
     };

     template<typename Char...>
     basic_istream {...};
     template<typename Char...>
     basic_ostream {...};
     template<typename Char...>
     basic_fstream {...};
  }
}

When working on non-win32 platform it would use native 
API (and most POSIX OSes use UTF-8 nativly)

On whindows each of these classes and functions assumes UTF-8 strings as
input and map underlying functions to _w* alternatives or
in case of basic_filebuf implements basic_filebuf over FILE *api and 
_wfopen.

This would allow much easier writing cross platform application
using unified and standard API instead of non-standard wide API.

Note, functions boost::nowide::convert would allow adapt any library
transparently use of widely used UTF-8 API instead of WIN32 one.

I had implemented this for my own projects, I'm asking if boost is 
interested in something like that at all.

Artyom