id,summary,reporter,owner,description,type,status,milestone,component,version,severity,resolution,keywords,cc
9827,"Missing support for some code page(e.g 949, 950) in windows conversion with std backend",hucaonju@…,Artyom Beilis,"There is a table windows_encoding all_windows_encodings[] in wconv_codepage.ipp. It contains several code page definitions. However, it misses some code pages, such as the Korean code page(949) or Traditional Chinese Big5 code page(950), which will cause an invalid_charset_error when running in that windows for the following code:

{{{
// Assuming we are using the std backend so it supports ansi encodings
boost::locale::generator gen;
gen.use_ansi_encoding(true);

std::locale loc(gen(""""));
// Throws invalid_charset_error when running in Korean windows but OK in English windows.
// The charset is ""windows949"" in Korean windows, which is not in the table.
std::string us = boost::locale::conv::to_utf<char>(""abcdefg"", loc);
}}}

The root cause of this exception is that the generated code page string is not in the table. When the locale generator with std backend in windows platform generates a locale, it calls boost::locale::util::get_system_locale(bool use_utf8). This function will use the following code to generate the locale string(in default_locale.cpp):
{{{
if(GetLocaleInfoA(LOCALE_USER_DEFAULT,LOCALE_IDEFAULTANSICODEPAGE,buf,sizeof(buf))!=0) {
    if(atoi(buf)==0)
        lc_name+="".UTF-8"";
    else {
        lc_name +="".windows-"";
        lc_name +=buf;
    }
}
}}}
So the encoding part of the lc_name is windows-(code page). In a system with Korean(949) or Traditional Chinese(950) code page, this will generate an encoding string like ""windows-949"" or ""windows-950"". However, when wconv_from_utf::open() initializes, it tries to search ""windows949"" or ""windows950"" in array all_windows_encodings[]. Obviously it will not find the string, and the open() fails, then the exception is thrown.

For a quick fix, I suggest adding the missing code page to the table:
{{{
{ ""cp949"",      949, 0 }, // Korean
{ ""uhc"",        949, 0 }, // From ""iconv -l""
{ ""windows949"",         949, 0 }, // Korean
// ""big5"" already in the table
{ ""windows950"",         950, 0 }, // TC, big5
}}}

However the list may not be complete, and we may encounter problems when running in a system with code page that does not exist in the list. So we may probably add the following code to function int encoding_to_windows_codepage(char const *ccharset) in wconv_codepage.ipp:

{{{
--- E:\Build1\boost_1_55_0\libs\locale\src\encoding\wconv_codepage.ipp	2014-04-02 16:34:52.000000000 +0800
+++ E:\Build2\boost_1_55_0\libs\locale\src\encoding\wconv_codepage.ipp	2014-04-02 17:31:37.000000000 +0800
@@ -206,12 +206,18 @@
                 return ptr->codepage;
             }
             else {
                 return -1;
             }
         }
+        if(ptr==end && charset.size()>7 && charset.substr(0,7)==""windows"") {
+            int cp = atoi(charset.substr(7).c_str());
+            if(IsValidCodePage(cp)) {
+                return cp;
+            }
+        }
         return -1;
         
     }
 
     template<typename CharType>
     bool validate_utf16(CharType const *str,unsigned len)
}}}

This piece of code directly parses and validates the encoding string. The concern is that the call to IsValidCodePage may decrease the performance(not tested).",Bugs,assigned,To Be Determined,locale,Boost 1.55.0,Problem,,"locale,code page,Korean,Traditional Chinese,exception",