Boost C++ Libraries: Ticket #12499: Memory allocation fails

attachment set

stephan.menzel@… — Tue, 04 Oct 2016 14:38:14 GMT

attachment → interprocess_bug.zip

Test Case project

anonymous — Tue, 04 Oct 2016 14:41:05 GMT

Of course, three typedefs missing:

typedef std::pair<const boost::uint64_t, ShmemStringPtr>  MessageValueType;
typedef boost::interprocess::allocator<MessageValueType, boost::interprocess::managed_shared_memory::segment_manager>  ShmemAllocator;
typedef boost::interprocess::map<boost::uint64_t, ShmemStringPtr, std::less<boost::uint64_t>, ShmemAllocator> MessageMap;

Ion Gaztañaga — Fri, 07 Oct 2016 23:14:09 GMT

Unfortunately, I could not reproduce the error. I suggest some changes to ease debugging:

Can you launch the client code as a thread in the server.cpp main, so that we can debug a single process? It still crashes?

Could you please send the generated Visual C++ project?

Are you using the latest Visual C++ 2015 Update? Just to make sure we are debuggning with the same toolset.

Can you add also the stack trace? I'd like to see the surrounding environment when the error triggers.

[Edit] when trying to active stack frame runtime check (/RTCs) Visual 2015 says it's not compatible with /O2 or /Ox optimization levels. So what does RelWithDebInfo use?

stephan.menzel@… — Mon, 10 Oct 2016 08:02:14 GMT

Hello Ion,

thanks for looking into this. The stacktrace is as follows:

>	client.exe!__report_gsfailure(unsigned __int64 stack_cookie) Line 199	C
 	client.exe!boost::container::map<unsigned __int64,boost::interprocess::offset_ptr<boost::container::basic_string<char,std::char_traits<char>,boost::interprocess::allocator<char,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index> > >,__int64,unsigned __int64,0>,std::less<unsigned __int64>,boost::interprocess::allocator<std::pair<unsigned __int64 const ,boost::interprocess::offset_ptr<boost::container::basic_string<char,std::char_traits<char>,boost::interprocess::allocator<char,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index> > >,__int64,unsigned __int64,0> >,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index> >,boost::container::tree_opt<0,1> >::operator[](const unsigned __int64 & x) Line 542	C++
 	client.exe!msgmap_insert(boost::container::map<unsigned __int64,boost::interprocess::offset_ptr<boost::container::basic_string<char,std::char_traits<char>,boost::interprocess::allocator<char,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index> > >,__int64,unsigned __int64,0>,std::less<unsigned __int64>,boost::interprocess::allocator<std::pair<unsigned __int64 const ,boost::interprocess::offset_ptr<boost::container::basic_string<char,std::char_traits<char>,boost::interprocess::allocator<char,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index> > >,__int64,unsigned __int64,0> >,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index> >,boost::container::tree_opt<0,1> > * n_map, boost::shared_ptr<CommandInterface> n_interface, const unsigned __int64 n_key, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & n_value) Line 61	C++
 	client.exe!ShmClient::send_command(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & n_command) Line 40	C++
 	client.exe!main(int argc, char * * argv) Line 11	C++

Looking at the throw location might offer a clue:

#elif defined _M_IX86 || defined _M_X64
    __declspec(noreturn) void __cdecl __report_gsfailure(GSFAILURE_PARAMETER)
    {
        if (IsProcessorFeaturePresent(PF_FASTFAIL_AVAILABLE))
        {
            __fastfail(FAST_FAIL_STACK_COOKIE_CHECK_FAILURE);
        }
        volatile UINT_PTR cookie[2];

This leads me to believe it might be a 64/32 bit build mishap along with those defines.

I did notice changes in the way boost builds and strangely it kept saying I build for 32 bit. Dependency walker showed a 64 bit lib though and I could link.

I build boost this way:

1) Open Visual Studio Developer shell.
2) Within, start the vcvars64.bat script to make it work on 64 bit arch.
3) Create b2.exe by starting bootstrap.bat
4) Run it using those parameters:
b2.exe --address-model=64 address-model=64 link=shared,static runtime-link=shared threading=multi variant=debug,release stage

I always used to build this way but maybe the changes in the build system caused this?

In any case, I will attach the created solution.

Thanks, Stephan

attachment set

stephan.menzel@… — Mon, 10 Oct 2016 08:05:30 GMT

attachment → interprocess_bug_including_solution.7z

Same test case but with generated MSVC14 solution included (CMake 3.6.1)

attachment set

stephan.menzel@… — Mon, 10 Oct 2016 11:06:21 GMT

attachment → interprocess_bug_including_solution.2.7z

Updated project with multi threaded single process test

stephan.menzel@… — Mon, 10 Oct 2016 11:11:02 GMT

Ahh, yes. I have also created a single process version as you requested. The error is the same. I have updated the attached project with that version. The executable is called mt.exe

About your RelWithDebInfo question, it doesn't allow those runtime checks for me either in non-debug modes. I guess that's why it's not Debug ;-) I just tried enabling those checks in Debug Config and see if this yields anything but I couldn't see any weird stuff. So as before, it runs fine in Debug config.

Ion Gaztañaga — Fri, 14 Oct 2016 16:01:14 GMT

There is something strange with the optimizer. If I just use internal functions of operator[] everything goes OK. One workaround I've found is to modify line :

BOOST_MOVE_CONVERSION_AWARE_CATCH( operator[] , key_type, mapped_type&, this->priv_subscript)

with

BOOST_MOVE_CONVERSION_AWARE_CATCH( operator[] , key_type, BOOST_CONTAINER_FORCEINLINE mapped_type&, this->priv_subscript)

With that change the optimizer seems to work ok in the test. Could you check that change makes your code OK again?

It seems like a MSVC bug, as in 32 bit versions there is no problem and other sanitizers don't seeem to catch any problem.

stephan.menzel@… — Sat, 15 Oct 2016 11:17:51 GMT

Hello Ion,

I can confirm that change does the trick. Thanks for the fix. My test case works again and I'm going to try the upgrade again.

I had a feeling it would be something like this. The error was just too weird. There are however quite a few other occurrences of similar code lines. I dare not try to change these as well as I believe I'm only using the map anyway. Just wanted to point out that there might be other container types affected.

Do you think it is a good idea to try to bring that to Microsofts attention? They might want to fix that on their end.

Cheers, Stephan

Ion Gaztañaga — Sat, 15 Oct 2016 19:31:57 GMT

Hi Stephan,

I think it's a good idea to report it to MS. It might be a problem with Interprocess, but even in that case I think we'd need help from compiler writers to diagnose it. Let me know if the upgrade goes fine.

status changed; resolution set

Ion Gaztañaga — Sat, 12 Nov 2016 23:39:18 GMT

status new → closed
resolution → fixed

The following commit in Boost.Move:

https://github.com/boostorg/move/commit/de55af3cbb935660a68940fc968d2509bc9f0c92

added the forceinline attribute to the function, so closing this ticket. Thanks for the report.