fstream open utf8 filename fails x86_64-w64-mingw32

[spoo@…]

boost::filesystem stream classes won't open files with non-ascii filenames provided as utf-8 strings. This happens with and without imbuing a locale as specified in the boost doc page "Default Encoding under Microsoft Windows". My example is basically the example on that page, but with an ifstream.

I compiled boost on Arch Linux using the AUR mingw-w64 packages. I reproduced the issue on Windows 7 x64, Windows 7 x64 VM, and in x64 Wine.

I confirmed that the file exists and is named properly by making a parallel test program using _wopen (the file opened successfully).

[spoo@…]

Test code

[spoo@…]

Here's a WINEDBG=+relay trace snip for a working example using _wopen:

0024:Call msvcrt._wfopen(00404012 L"\6e90\5c4b.txt",0040400c L"rb") ret=00401532
0024:Call KERNEL32.CreateFileW(00404012 L"\6e90\5c4b.txt",80000000,00000003,0032f950,ffffffff00000003,00000080,00000000) ret=7fa609f2952b
0024:Ret  KERNEL32.CreateFileW() retval=0000004c ret=7fa609f2952b               
0024:Ret  msvcrt._wfopen() retval=7fa60a1a0c60 ret=00401532

Here's the trace snip for the broken example with boost (fstream.cxx):

0024:Call msvcrt.fopen(00049628 "\xe6\xba\x90\xe5\xb1\x8b.txt",004d0778 "r") ret=00486efb
0024:Call ntdll.RtlAllocateHeap(00010000,00000000,00000016) ret=7f3fa25e446c    
0024:Ret  ntdll.RtlAllocateHeap() retval=00049740 ret=7f3fa25e446c              
0024:Call KERNEL32.MultiByteToWideChar(00000000,00000001,00049628 "\xe6\xba\x90\xe5\xb1\x8b.txt",0000000b,00049740,0000000b) ret=7f3fa25cd130
0024:Ret  KERNEL32.MultiByteToWideChar() retval=0000000b ret=7f3fa25cd130       
0024:Call ntdll.RtlAllocateHeap(00010000,00000000,00000004) ret=7f3fa25e446c    
0024:Ret  ntdll.RtlAllocateHeap() retval=000435d0 ret=7f3fa25e446c              
0024:Call KERNEL32.MultiByteToWideChar(00000000,00000001,004d0778 "r",00000002,000435d0,00000002) ret=7f3fa25cd130
0024:Ret  KERNEL32.MultiByteToWideChar() retval=00000002 ret=7f3fa25cd130       
0024:Call KERNEL32.CreateFileW(00049740 L"\00e6\00ba\0090\00e5\00b1\2039.txt",80000000,00000003,0032f4c0,00000003,00000080,00000000) ret=7f3fa25dc52b
0024:Ret  KERNEL32.CreateFileW() retval=ffffffffffffffff ret=7f3fa25dc52b       
0024:Call ntdll.RtlFreeHeap(00010000,00000000,00049740) ret=7f3fa25e058f        
0024:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7f3fa25e058f                  
0024:Call ntdll.RtlFreeHeap(00010000,00000000,000435d0) ret=7f3fa25e0597        
0024:Ret  ntdll.RtlFreeHeap() retval=00000001 ret=7f3fa25e0597                  
0024:Ret  msvcrt.fopen() retval=00000000 ret=00486efb

I'm not sure what the call equivalencies are but I think that's all from the second example. If I had to hazard a guess, it looks like the problem might be that it's not converting the string to wide characters and using the corresponding msvcrt call.

[Dominik Schmidt <boost@…>]

Looks like a duplicate of ​https://svn.boost.org/trac/boost/ticket/5769