Boost C++ Libraries: Ticket #7606: u32regex causes bus error

attachment set

a.sanders@… — Tue, 30 Oct 2012 11:46:24 GMT

attachment → regex.cpp

anonymous — Tue, 30 Oct 2012 11:47:08 GMT

Wiki formatting made a mess of the quoted program so have attached it.

attachment set

a.sanders@… — Thu, 08 Nov 2012 10:04:05 GMT

attachment → regex-crash.cpp

Causes Bus Error when run

a.sanders@… — Thu, 08 Nov 2012 10:06:26 GMT

Just noticed I attached the wrong version of the file before (a version that didn't cause a Bus Error.) So I have now attached the version that does cause the Bus Error. At least you now have both versions.

Apologies for any confusion.

John Maddock — Wed, 28 Nov 2012 13:32:51 GMT

I'm unable to reproduce either on Win32 or ubuntu Linux with current SVN Trunk and ICU 49. Are you able to debug locally?

Wed, 28 Nov 2012 13:58:54 GMT

Probably. What are you after? A stack trace? Anything else?

Wed, 28 Nov 2012 14:20:52 GMT

Built a debug version of libboost_regex. I'll attach a stack trace from gdb. If you'd like me to investigate further please give me some pointers as to what to look for!

attachment set

Wed, 28 Nov 2012 14:24:15 GMT

attachment → regex gdb trace.txt

gdb stack trace

John Maddock — Wed, 28 Nov 2012 18:22:41 GMT

Thanks for the stack trace, unfortunately that makes even less sense now :-(

Can you set a breakpoint (basic_regex.cpp:399) inside:

   template <class InputIterator>
   basic_regex(InputIterator arg_first, InputIterator arg_last, flag_type f = regex_constants::normal)
   {
      typedef typename traits::string_type seq_type;
      seq_type a(arg_first, arg_last);
      if(a.size())
         assign(static_cast<const charT*>(&*a.begin()), static_cast<const charT*>(&*a.begin() + a.size()), f);
      else
         assign(static_cast<const charT*>(0), static_cast<const charT*>(0), f);
   }

What are the contents of "a" after construction?

Any chance that your code is compiled using a compiler code page that results in the input string not being valid ASCII/UTF8?

Thanks, John.

Fri, 30 Nov 2012 16:14:34 GMT

Here you are. Not sure that this looks helpful. At the "if" statement following the "a" construction took the true branch.

I'm not setting any compiler code page anywhere.

Breakpoint 2 at 0x15738: file /export/home/ashley/src/boost_1_51_0/boost/regex/v4/basic_regex.hpp, line 399.
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /export/home/ashley/tmp/regex
[Thread debugging using libthread_db enabled]
[New Thread 1 (LWP 1)]
[Switching to Thread 1 (LWP 1)]
Breakpoint 2, boost::basic_regex<int, boost::icu_regex_traits>::basic_regex<boost::u8_to_u32_iterator<char const*, int> > (
    this=0xffbffab0, arg_first=..., arg_last=..., f=0) at /export/home/ashley/src/boost_1_51_0/boost/regex/v4/basic_regex.hpp:400
400        {
(gdb) list
395           assign(p, f);
396        }
397
398        template <class InputIterator>
399        basic_regex(InputIterator arg_first, InputIterator arg_last, flag_type f = regex_constants::normal)
400        {
401           typedef typename traits::string_type seq_type;
402           seq_type a(arg_first, arg_last);
403           if(a.size())
404              assign(static_cast<const charT*>(&*a.begin()), static_cast<const charT*>(&*a.begin() + a.size()), f);
(gdb) n
402           seq_type a(arg_first, arg_last);
(gdb) n
403           if(a.size())
(gdb) print a
$1 = {<std::_Vector_base<int, std::allocator<int> >> = {
    _M_impl = {<std::allocator<int>> = {<__gnu_cxx::new_allocator<int>> = {<No data fields>}, <No data fields>}, _M_start = 0x27cd0,
      _M_finish = 0x27d00, _M_end_of_storage = 0x27d00}}, <No data fields>}

Fri, 30 Nov 2012 16:19:08 GMT

What I meant to say was the "if(a.size())" evaluated true.

anonymous — Fri, 30 Nov 2012 16:39:29 GMT

You're right, doesn't really help :-(

Try:

#include <boost/regex/icu.hpp>
template <class C>
void printout(const C& c)
{
   for(unsigned i = 0; i < c.size(); ++i)
      std::cout << std::hex << (int)c[i] << " ";
   std::cout << std::endl;
}
int main()
{
   using namespace boost;
   typedef u32regex::traits_type::string_type st;
   typedef boost::u8_to_u32_iterator<std::string::const_iterator, UChar32> conv_type;
   const std::string p = "[pq]\\.\\.[xy]";
   st t(conv_type(p.begin(), p.begin(), p.end()), conv_type(p.end(), p.begin(), p.end()));
   printout(p);
   printout(t);
   return 0;
}

Which should output:

5b 70 71 5d 5c 2e 5c 2e 5b 78 79 5d
5b 70 71 5d 5c 2e 5c 2e 5b 78 79 5d

Thanks! John.

Fri, 30 Nov 2012 16:48:40 GMT

It does indeed output

5b 70 71 5d 5c 2e 5c 2e 5b 78 79 5d
5b 70 71 5d 5c 2e 5c 2e 5b 78 79 5d

anonymous — Fri, 30 Nov 2012 17:30:52 GMT

Which leaves me stumped again.... what does Valgrind say?

anonymous — Fri, 30 Nov 2012 17:56:29 GMT

Some supplementary questions before we get too involved:

1) Is

regex e("[pq]\\.\\.[xy]");

OK?

2) Is:

wregex we(L"[pq]\\.\\.[xy]");

OK?

3) Are their multiple versions of ICU installed on this system? Any chance there's a mismatch between the headers included and libraries loaded, or between the version used when Boost was built, and the one used by the test program?

4) Do the regex tests run OK? To run, cd into libs/regex/test and do a "bjam toolset=sun". Assuming ICU is installed in the usual location, you should see a message at the start to say it's being used/tested.

Thanks again! John.

Wed, 12 Dec 2012 11:32:47 GMT

Apologies for the delay in doing the stuff you asked for. Life and work get in the way...

Replying to anonymous:

Some supplementary questions before we get too involved:

1) Is
regex e("[pq]\\.\\.[xy]");

Compiles and runs okay.

2) Is:

wregex we(L"[pq]\\.\\.[xy]");

Also compiles and runs okay.

3) Are their multiple versions of ICU installed on this system? Any chance there's a mismatch between the headers included and libraries loaded, or between the version used when Boost was built, and the one used by the test program?

There are multiple version of the .so files, but as far as I can tell there is only one set of header files. I don't think this should be a problem.

4) Do the regex tests run OK? To run, cd into libs/regex/test and do a "bjam toolset=sun". Assuming ICU is installed in the usual location, you should see a message at the start to say it's being used/tested.

I'm using gcc to compile so I did "bjam toolset=gcc". I'll attach the output separately. There were errors but they are a bit hard to spot from the warnings and errors spat out by the compiler. I'll attach two files. The first the output from running bjam the first time (rather a large file) and the second file is from running bjam again -- less output which hopefully makes it easier to spot the bus error from one of the tests.

Ashley.

attachment set

Wed, 12 Dec 2012 11:36:24 GMT

attachment → regex-gcc-test.txt.bz2

Output from first run of bjam toolset=gcc

attachment set

Wed, 12 Dec 2012 11:37:06 GMT

attachment → regex-gcc-test-2.txt

Output from second run of bjam toolset=gcc

John Maddock — Thu, 20 Dec 2012 16:50:53 GMT

My turn to apologize for the delay - I blame Christmas! :)

Thanks for running the tests, they show the same issue as you reported in your test case, I don't understand why it would work for wregex but not u32regex though :(

There must be some memory corruption/overrun going on, but it's going to be hard to diagnose by email! Is Valgrind available for that platform? If so it's output might help a lot, otherwise I'll have to write a special instrumented version for you to test with I guess.

Thanks, John.