Boost C++ Libraries: Ticket #5928: FileSystem runtime error: locale::facet::_S_create_c_locale name not valid

midnight-runner@… — Fri, 21 Oct 2011 14:07:23 GMT

I can confirm this, even though the problem is kind of strange ...

I'm using boost 1.47 and gcc 4.6.1 and doing static compilation as well. To run the program, I send the comandline (with parameters) to a cluster running the (sun) grid engine. Hence, I'm not in an interactive shell. If LC_ALL isn't set to 'C' in this shell (export LC_ALL=C) I will end up with "Error: locale::facet::_S_create_c_locale name not valid." However, if I log in to the maschine and execute the program without(!) setting LC_ALL to 'C', I can execute the program without problems.

Here's the system: Linux PC_NAME 2.6.38-10-server #46~lucid1-Ubuntu SMP Wed Jul 6 20:19:32 UTC 2011 x86_64 GNU/Linux

midnight-runner@… — Fri, 21 Oct 2011 14:26:01 GMT

Update: Oh I just realized i used the wrong machine to do the interactive test ... and now it turns out that the programs fails in the interactive sessions as well, as long LC_ALL isn't set to 'C'.

System: Linux mphy-s01 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux

Beman Dawes — Sun, 22 Jan 2012 16:00:45 GMT

Replying to midnight-runner@…:

Update: Oh I just realized i used the wrong machine to do the interactive test ... and now it turns out that the programs fails in the interactive sessions as well, as long LC_ALL isn't set to 'C'.

System: Linux mphy-s01 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux

I'm going to need a little test program that demonstrates this problem. My tests here all work perfectly, running fine if the environment is valid, otherwise throwing an exception after main() has started. So it would be very helpful to see your code that is failing, and be able to test with it here.

Thanks,

--Beman

midnight-runner@… — Wed, 25 Jan 2012 14:33:38 GMT

Replying to bemandawes:

Replying to midnight-runner@…:

Update: Oh I just realized i used the wrong machine to do the interactive test ... and now it turns out that the programs fails in the interactive sessions as well, as long LC_ALL isn't set to 'C'.

System: Linux mphy-s01 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux

I'm going to need a little test program that demonstrates this problem. My tests here all work perfectly, running fine if the environment is valid, otherwise throwing an exception after main() has started. So it would be very helpful to see your code that is failing, and be able to test with it here.

Thanks,

--Beman

Ok I have prepared a few test cases with two different systems (different compilers). First of all, the problem occurs only if the programs are linked statically and only if codesample 1 is used. Codesample 2 works fine.

System 1 (Ubuntu 11.10): Linux XXX 3.0.0-15-server #25-Ubuntu SMP Mon Jan 2 19:14:55 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

System 2 (Ubuntu 09.10): Linux XXX 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)

Boost with version 1.48 was employed and both systems use the same boost includes and libs (via nfs shares).

Codesample 1 (won't work if compiled statically with system 1 and running on system 2 with "incorrec LC-ALL"):

#include <boost/filesystem.hpp>
int main ( int argc, char *argv[] ) {
  boost::filesystem::path path;
  std::string             filename("readme.txt");
  path = filename;     // this isn't working
//  path = "readme.txt"; // this isn't working
  std::cout << "OK" << std::endl;
  return 0;
}

Codesample 2 (works all the time):

#include <boost/filesystem.hpp>
int main ( int argc, char *argv[] ) {
  boost::filesystem::path path("readme.txt");
  std::cout << "OK" << std::endl;
  return 0;
}

Compiling codesample 1 with system 1:
g++ main.cpp -o test_SYS-1_dyn -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH
g++ main.cpp -o test_SYS-1_sta -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH -static

Compiling codesample 1 with system 2:
g++ main.cpp -o test_SYS-2_dyn -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH
g++ main.cpp -o test_SYS-2_sta -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH -static

Running the programs:

With system 1:
and export LC-ALL='C':
test_SYS-1_dyn:OK
test_SYS-1_sta:OK
test_SYS-2_dyn:OK
test_SYS-2_sta:OK

and export LC-ALL='en_GB.UTF-8':
test_SYS-1_dyn:OK
test_SYS-1_sta:OK
test_SYS-2_dyn:OK
test_SYS-2_sta:OK

With system 2:
and export LC-ALL='C':
test_SYS-1_dyn:OK
test_SYS-1_sta:OK
test_SYS-2_dyn:OK
test_SYS-2_sta:OK

and export LC-ALL='en_GB.UTF-8':
test_SYS-1_dyn:OK
test_SYS-1_sta: terminate called after throwing an instance of 'std::runtime_error'

what(): locale::facet::_S_create_c_locale name not valid

Aborted test_SYS-2_dyn:OK
test_SYS-2_sta:OK

Thanks,

Christian

Mon, 06 Feb 2012 07:06:27 GMT

Replying to midnight-runner@…:

Replying to bemandawes:

Replying to midnight-runner@…:

Update: Oh I just realized i used the wrong machine to do the interactive test ... and now it turns out that the programs fails in the interactive sessions as well, as long LC_ALL isn't set to 'C'.

System: Linux mphy-s01 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux

I'm going to need a little test program that demonstrates this problem. My tests here all work perfectly, running fine if the environment is valid, otherwise throwing an exception after main() has started. So it would be very helpful to see your code that is failing, and be able to test with it here.

Thanks,

--Beman

Ok I have prepared a few test cases with two different systems (different compilers). First of all, the problem occurs only if the programs are linked statically and only if codesample 1 is used. Codesample 2 works fine.

System 1 (Ubuntu 11.10): Linux XXX 3.0.0-15-server #25-Ubuntu SMP Mon Jan 2 19:14:55 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

System 2 (Ubuntu 09.10): Linux XXX 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)

Boost with version 1.48 was employed and both systems use the same boost includes and libs (via nfs shares).

Codesample 1 (won't work if compiled statically with system 1 and running on system 2 with "incorrec LC-ALL"):
#include <boost/filesystem.hpp>
int main ( int argc, char *argv[] ) {
  boost::filesystem::path path;
  std::string             filename("readme.txt");
  path = filename;     // this isn't working
//  path = "readme.txt"; // this isn't working
  std::cout << "OK" << std::endl;
  return 0;
}
Codesample 2 (works all the time):
#include <boost/filesystem.hpp>
int main ( int argc, char *argv[] ) {
  boost::filesystem::path path("readme.txt");
  std::cout << "OK" << std::endl;
  return 0;
}
Compiling codesample 1 with system 1:
g++ main.cpp -o test_SYS-1_dyn -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH
g++ main.cpp -o test_SYS-1_sta -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH -static

Compiling codesample 1 with system 2:
g++ main.cpp -o test_SYS-2_dyn -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH
g++ main.cpp -o test_SYS-2_sta -L$LIB_PATH -lboost_system -lboost_filesystem -I$INCLUDE_PATH -static

Running the programs:

With system 1:
and export LC-ALL='C':
test_SYS-1_dyn:OK
test_SYS-1_sta:OK
test_SYS-2_dyn:OK
test_SYS-2_sta:OK

and export LC-ALL='en_GB.UTF-8':
test_SYS-1_dyn:OK
test_SYS-1_sta:OK
test_SYS-2_dyn:OK
test_SYS-2_sta:OK

With system 2:
and export LC-ALL='C':
test_SYS-1_dyn:OK
test_SYS-1_sta:OK
test_SYS-2_dyn:OK
test_SYS-2_sta:OK

and export LC-ALL='en_GB.UTF-8':
test_SYS-1_dyn:OK
test_SYS-1_sta: terminate called after throwing an instance of 'std::runtime_error'

what(): locale::facet::_S_create_c_locale name not valid

Aborted test_SYS-2_dyn:OK
test_SYS-2_sta:OK

Thanks,

Christian

I experienced similar problem when I linked statically to boost and runtime libraries; but the reason was not a Boost.

AFAIK the line
path = filename; // this isn't working
internally calls to some STL's functions. One of them eventually calls to codecvt() constructor which in turn calls to the newlocale(...) function from glibc and throws the locale::facet::_S_create_c_locale exception if newlocale() fails (i.e. returns 0).

The system where I built the application is Slackware 13.37 (like System 1 in your case) with glibc-2.13. But the system where the application was intended to run is Slackware 12.0 (System 2) with glibc-2.7

Trying to strace the application on System 2 shows that (as a result of a call to newlocale()) statically linked glibc-2.13 opens /usr/lib/locale/ru_RU.koi8r/LC_CTYPE file (I guess it would be /usr/lib/locale/en_GB.utf8/LC_CTYPE in your testcase) it stumbles upon incompatible version of this file (from glibc-2.7).

Exact file which is opened by call to glibc's newlocale() depends on LC_ALL (or LANG if no LC_ALL set) environment variable; but if LC_ALL/LANG is set to standard intrinsic locale (e.g. LC_ALL=C or LC_ALL=POSIX) then glibc-2.13 constructs the locale without opening any file, so the newlocale() call is successful in this case. If one manages to link application to runtime libraries dynamically (regretfully this is not my case) then the call to newlocale() would be successful too.

Hope this helps. Good luck.

hugh.m.bright@… — Sat, 07 Jul 2012 23:51:13 GMT

I can reproduce this as well, the only difference beteween my "works" machine and "doesnt works" machine is the version of libstdc++ involved.

Following http://stackoverflow.com/questions/10354636/how-do-you-find-what-version-of-libstdc-library-is-installed-on-your-linux-mac

strings /usr/lib/libstdc++.so.6 | grep GLIBCXX

Machine that doesn't work: list stops with GLIBCXX_3.4.13 Machine that does work: list stops with GLIBCXX_3.4.16

Test program to demonstrate crash:

#include <boost/filesystem.hpp> main() { boost::filesystem::path fp = boost::filesystem::current_path(); }

koliasvskj@… — Sun, 08 Jul 2012 11:22:14 GMT

Replying to hugh.m.bright@…:

I can reproduce this as well, the only difference beteween my "works" machine and "doesnt works" machine is the version of libstdc++ involved.

Following http://stackoverflow.com/questions/10354636/how-do-you-find-what-version-of-libstdc-library-is-installed-on-your-linux-mac

strings /usr/lib/libstdc++.so.6 | grep GLIBCXX

Machine that doesn't work: list stops with GLIBCXX_3.4.13 Machine that does work: list stops with GLIBCXX_3.4.16

Test program to demonstrate crash:

#include <boost/filesystem.hpp> main() { boost::filesystem::path fp = boost::filesystem::current_path(); }

But what does it tell about glibc version in your case? Please, try also

strings /usr/lib/libstdc++.so.6 | grep GLIBC

for this...

Thanks.

dzinn@… — Thu, 09 Aug 2012 22:01:29 GMT

I can confirm the bug. Is there a patch available?

My GLIBC version goes up to 3.4.16. Other details of my system are:

libboost_filesystem.so.1.49.0

gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

Ubuntu 11.10

Linux 3.1.0-1-amd64

strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX | grep 3.4.16

GLIBCXX_3.4.16

status changed; resolution set

Beman Dawes — Fri, 22 Feb 2013 13:24:24 GMT

status new → closed
resolution → fixed

Boost.Filesystem's implementation of path locale and codecvt handling has been rewritten with much simpler, more portable, and hopefully more robust code. The interface has not changed. The rewrite has been applied to svn trunk as of revision 83062. An added reference documentation section (boost-root/libs/filesystem/doc/reference.html#path-Usage) describes class path usage concerns, such as thread data races.

These changes should resolve this issue. If you believe it has not been resolved satisfactorily, please open a new issue rather than reopening this issue. But before you do that, please reread boost-root/libs/filesystem/doc/reference.html#path-Usage to be sure that the problem you are seeing is not a manifestation of one of the those concerns. If you do open a new issue, please be specific and supply a test case and lots of details. Just saying something "doesn't work" or "crashes" is not enough!

Thanks,

--Beman

sleary@… — Wed, 16 Jul 2014 10:36:11 GMT

Which release is revision 83062?

I see this behaviour on Solaris on 1.49. A call to unique_path() when LC_ALL is not set to C causes a crash as described above. Will attempt to verify on a later version.

orangetinyterror@… — Fri, 18 Jul 2014 11:58:14 GMT

For what it's worth, I was hitting this with version 1.51.0 when passing a char[] to any API that implicitly constructs a path. There is a template constructor that seems to jump into the locale-resolution stuff (codecvt, etc...).

Casting to or creating a const char * or a std::string seems to bypass that works as expected.

For example:

char dir[] = "/some/path";
boost::filesystem::exists( dir ); // runtime exception, perhaps dependent on env. locale
boost::filesystem::exists( std::string( dir ) ); // works

There's a comment in path.hpp about this, so I'm not sure it should be called a bug or not...

Hope this helps.