Boost C++ Libraries: Ticket #3332: boost::filesystem::path will get trobule in locale Chinese_Taiwan.950 (windows)

attachment set

Tue, 11 Aug 2009 09:30:39 GMT

attachment → code.txt

try to fix the problem but not really work.

component changed; owner set

Tue, 11 Aug 2009 09:52:41 GMT

owner set to Beman Dawes
component None → filesystem

at code line 22 can be deleted. this is over paste

attachment set

Tue, 11 Aug 2009 10:13:31 GMT

attachment → test-code-and-data.zip

a minial test app and test folder

Tue, 11 Aug 2009 10:32:00 GMT

In test-code-and-data.zip contains: test.cpp and test_folder. the test.cpp use recursive_directory_iterator to travsal test_folder. but the recursive_directory_iterator cannot travsal it completely because of the wrong path converted.

test_folder contains this structure:

test_folder\
    功/
        foo.txt
    功能總覽/
        a.txt
        b.txt
    另一個資料夾/
        a.txt
        b.txt

the path name contains 「功」 is a chinese character which means 'function'. it last byte is 0x5c ('
'), when the path try to convert the '
' to '' will make this word broken and the semi-word have the chance to become antoher chinese character with the first byte in 「功」. that is reason why we got problem in cp950 at win os.

the test app output is:

C:\demo-room-workspace\native.impl>test
test_folder\功 [directory]
boost::filesystem::basic_directory_iterator constructor: 系統找不到指定的路徑。:
 "test_folder\功功功能總覽"
0.00 s

the message said: system cannot found the path "test_folder\功功功能總覽". when path object try to invoke remove_filename() it will calculate the last of '\' but 「功」 also contains '\' so that remove_filename() got test_folder\功 (half of 功) not test_folder.

then, path add next path '\' will make the half of 功 become the whole Big5 char 「功」. the path will be "test_folder\功" not "test_folder\"

status changed

Beman Dawes — Tue, 18 Aug 2009 01:57:51 GMT

status new → assigned

nemg2004@… — Fri, 16 Mar 2012 02:25:37 GMT

您好，請問現在的filesystem可以支持中文路徑了嗎？

status changed; resolution set

Beman Dawes — Wed, 24 Dec 2014 13:22:00 GMT

status assigned → closed
resolution → worksforme

Sorry for the 5 year delay in closing this.

The problem does not reproduce with current versions of boost.filesystem. The path has already been converted to UTF-16 by the time operations begin, so the C5 character in cp950 is immaterial.

Here is an updated test program, using the codepage 950 codecvt facet that ships with recent versions of VC++:

#include <boost/filesystem.hpp>
#include <cvt/cp950>
#include <iostream>
#include <string>
#include <locale>
namespace fs = boost::filesystem;
int main(void) {
  std::locale global_loc = std::locale();
  std::locale loc(global_loc, new stdext::cvt::codecvt_cp950<wchar_t>);
  fs::path::imbue(loc);
  std::cout <<
    "HEADS UP! PIPE OUTPUT TO FILE AND INSPECT WITH HEX OR CP950 EDITOR.\n"
    "WINDOWS COMMAND PROMPT FONTS DON'T SUPPORT CHINESE,\n"
    "EVEN WITH CODEPAGE SET AND EVEN AS OF WIN 10 TECH PREVIEW." << std::endl;
  fs::recursive_directory_iterator end;
  fs::recursive_directory_iterator iter
    ("C:/boost/modular/develop/libs/filesystem/test/issues/3332/test_folder");
  while (iter != end)
  {
    if (fs::is_directory(*iter))
    {
      std::cout << "[directory] " << iter->path().generic_string() << std::endl;
    }
    else if (fs::is_regular(*iter))
    {
      std::cout << " [file] " << iter->path().generic_string() << std::endl;
    }
    ++iter;
  }
  return 0;
}

A hex dump of the output shows that it does correctly handle the Big5 characters.

I've also tested a UTF-8 version of the above, and checked those with a UTF-8 aware text editor.

Thanks,

--Beman