Opened 6 years ago

Closed 5 years ago

#12864 closed Support Requests (obsolete)

AddressSanitizer: heap-use-after-free in boost::chrono::time_point

Reported by: jack@… Owned by: viboes
Milestone: To Be Determined Component: thread
Version: Boost 1.62.0 Severity: Problem
Keywords: Cc:

Description

When I run Zcash's Boost test suite under ASan locally (Ubuntu 16.0.4), the test suite starts up fine and can encounter ASan bugs in our code. But when it runs on our CI server (Amazon EC2 c4.8xlarge), ASan aborts as soon as the test suite starts, with the following failure:

==7928==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070000d6f00 at pc 0x7fd224ac6376 bp 0x7fd219366930 sp 0x7fd219366928
READ of size 8 at 0x6070000d6f00 thread T43
    #0 0x7fd224ac6375 in boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >::time_since_epoch() const /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/chrono/time_point.hpp:196
    #1 0x7fd224ac6375 in operator< <boost::chrono::system_clock, boost::chrono::duration<long int, boost::ratio<1l, 1000000000l> >, boost::chrono::duration<long int, boost::ratio<1l, 1000000000l> > > /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/chrono/time_point.hpp:323
    #2 0x7fd224ac6375 in wait_until<boost::chrono::duration<long int, boost::ratio<1l, 1000000000l> > > /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/condition_variable_fwd.hpp:211
    #3 0x7fd224ac6375 in CScheduler::serviceQueue() /home/admin/bbs/zcashASan/build/src/scheduler.cpp:58
    #4 0x7fd2245f7afb in boost::_mfi::mf0<void, CScheduler>::operator()(CScheduler*) const /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/mem_fn_template.hpp:49
    #5 0x7fd2245f7afb in operator()<boost::_mfi::mf0<void, CScheduler>, boost::_bi::list0> /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/bind.hpp:259
    #6 0x7fd2245f7afb in boost::_bi::bind_t<void, boost::_mfi::mf0<void, CScheduler>, boost::_bi::list1<boost::_bi::value<CScheduler*> > >::operator()() /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/bind/bind.hpp:1294
    #7 0x7fd2245f7afb in boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, CScheduler>, boost::_bi::list1<boost::_bi::value<CScheduler*> > > >::run() /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/detail/thread.hpp:116
    #8 0x7fd224c4b5f9 in thread_proxy (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0xa055f9)
    #9 0x7fd223c060a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x80a3)
    #10 0x7fd222afb62c in clone (/lib/x86_64-linux-gnu/libc.so.6+0xe862c)

0x6070000d6f00 is located 32 bytes inside of 72-byte region [0x6070000d6ee0,0x6070000d6f28)
freed by thread T44 here:
    #0 0x7fd2243e4fe7 in operator delete(void*) (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0x19efe7)
    #1 0x7fd224ac6672 in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::deallocate(std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >*, unsigned long) /usr/include/c++/4.9/ext/new_allocator.h:110
    #2 0x7fd224ac6672 in std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > > >::deallocate(std::allocator<std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >&, std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >*, unsigned long) /usr/include/c++/4.9/bits/alloc_traits.h:383
    #3 0x7fd224ac6672 in std::_Rb_tree<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> >, std::_Select1st<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::_M_put_node(std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >*) /usr/include/c++/4.9/bits/stl_tree.h:389
    #4 0x7fd224ac6672 in std::_Rb_tree<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> >, std::_Select1st<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::_M_destroy_node(std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >*) /usr/include/c++/4.9/bits/stl_tree.h:438
    #5 0x7fd224ac6672 in std::_Rb_tree<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> >, std::_Select1st<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::_M_erase_aux(std::_Rb_tree_const_iterator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >) /usr/include/c++/4.9/bits/stl_tree.h:1867
    #6 0x7fd224ac6672 in std::_Rb_tree<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> >, std::_Select1st<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::erase[abi:cxx11](std::_Rb_tree_iterator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >) /usr/include/c++/4.9/bits/stl_tree.h:868
    #7 0x7fd224ac6672 in std::multimap<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, boost::function<void ()>, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::erase[abi:cxx11](std::_Rb_tree_iterator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >) /usr/include/c++/4.9/bits/stl_multimap.h:638
    #8 0x7fd224ac6672 in CScheduler::serviceQueue() /home/admin/bbs/zcashASan/build/src/scheduler.cpp:68

previously allocated by thread T0 here:
    #0 0x7fd2243e4b6f in operator new(unsigned long) (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0x19eb6f)
    #1 0x7fd224acafc0 in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::allocate(unsigned long, void const*) /usr/include/c++/4.9/ext/new_allocator.h:104
    #2 0x7fd224acafc0 in std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > > >::allocate(std::allocator<std::_Rb_tree_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >&, unsigned long) /usr/include/c++/4.9/bits/alloc_traits.h:357
    #3 0x7fd224acafc0 in std::_Rb_tree<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> >, std::_Select1st<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::_M_get_node() /usr/include/c++/4.9/bits/stl_tree.h:385
    #4 0x7fd224acafc0 in _M_create_node<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long int, boost::ratio<1l, 1000000000l> > >, boost::function<void()> > > /usr/include/c++/4.9/bits/stl_tree.h:417
    #5 0x7fd224acafc0 in std::_Rb_tree_iterator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > std::_Rb_tree<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> >, std::_Select1st<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > >, std::less<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > >, std::allocator<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > const, boost::function<void ()> > > >::_M_insert_<std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, boost::function<void ()> > >(std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::pair<boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >, boost::function<void ()> >&&) /usr/include/c++/4.9/bits/stl_tree.h:1143
    #6 0x7fd225887daf (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0x1641daf)

Thread T43 created by T0 here:
    #0 0x7fd2243b372a in __interceptor_pthread_create (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0x16d72a)
    #1 0x7fd224c4a989 in boost::thread::start_thread_noexcept() (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0xa04989)
    #2 0x602000059f6f (+0x59f6f)

Thread T44 created by T0 here:
    #0 0x7fd2243b372a in __interceptor_pthread_create (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0x16d72a)
    #1 0x7fd224c4a989 in boost::thread::start_thread_noexcept() (/home/admin/bbs/zcashASan/build/src/test/test_bitcoin+0xa04989)
    #2 0x60200005a00f (+0x5a00f)

SUMMARY: AddressSanitizer: heap-use-after-free /home/admin/bbs/zcashASan/build/depends/x86_64-unknown-linux-gnu/share/../include/boost/chrono/time_point.hpp:196 boost::chrono::time_point<boost::chrono::system_clock, boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > >::time_since_epoch() const
Shadow bytes around the buggy address:
  0x0c0e80012d90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e80012da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e80012db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e80012dc0: 00 00 00 00 00 00 00 00 00 fa fa fa fa fa 00 00
  0x0c0e80012dd0: 00 00 00 00 00 00 00 fa fa fa fa fa fd fd fd fd
=>0x0c0e80012de0:[fd]fd fd fd fd fa fa fa fa fa fd fd fd fd fd fd
  0x0c0e80012df0: fd fd fd fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c0e80012e00: 00 fa fa fa fa fa 00 00 00 00 00 00 00 00 00 fa
  0x0c0e80012e10: fa fa fa fa 00 00 00 00 00 00 00 00 00 fa fa fa
  0x0c0e80012e20: fa fa 00 00 00 00 00 00 00 00 00 00 fa fa fa fa
  0x0c0e80012e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Contiguous container OOB:fc
  ASan internal:           fe
==7928==ABORTING

Change History (5)

comment:1 by anonymous, 6 years ago

Any updates here?

comment:2 by viboes, 5 years ago

Status: newassigned

I will need either the sources ( a link maybe) or an example that reproduce the issue.

I don't know who is allocating and reallocating the concerned memory.

comment:3 by viboes, 5 years ago

Component: chronothread

Moved to Boost.Thread as I don't think Boost.Chrono is concerned.

comment:4 by viboes, 5 years ago

Type: BugsSupport Requests

Moved to support request until it is clear who is allocating and deallocating the memory?

comment:5 by viboes, 5 years ago

Resolution: obsolete
Status: assignedclosed

Closed as there is no follow up.

Note: See TracTickets for help on using tickets.