32-bit offset_ptr crashes on 64-bit platform

[Andrey Semashev]

There seem to be problems with handling offset_ptrs with 32-bit offsets on a 64-bit platform. The attached code sample tries to create a file mapping with managed_external_buffer using 32-bit offset_ptrs. The program crashes sometimes when trying to initialize the mapping (see the attached valgrind log for one of such crashes). The offset_ptr is declared as follows:

typedef boost::interprocess::offset_ptr< void, std::ptrdiff_t, std::int32_t > void_pointer;

My theory is that there are incorrect offset conversions somewhere in Boost.Interprocess which sometimes result in incorrect pointers and a subsequent crash. This may not happen on every run of the program because the mapping address of the file region can change between runs.

One of the problems with 32-bit offset_ptrs is that its implementation performs implicit casts of the offset to std::size_t (see calls to offset_ptr_to_raw_pointer and offset_ptr_to_offset), which makes incorrect result if the offset type is unsigned and smaller than std::size_t. I didn't find any restrictions on the offset type in the docs or the code, and by default the offset type is unsigned. IMO, the code should be corrected to perform sign extension of the offset or restrict the offset type to be signed integers. This should be documented as well.

However, even if my program uses a signed offset type, it still crashes, so there have to be other issues elsewhere.

I tested this on Kubuntu 14.04 x86_64.

[Andrey Semashev]

A test case to reproduce the crash

[Andrey Semashev]

Valgrind output

[Ion Gaztañaga]

Using signed integers is problematic because overflow is undefined behaviour, there were problems with some compilers. And yes, offset_ptr was not thought to be used with integers smaller than size_t (or more strictly, smaller than uintptr_t). The option to customize the OffsetType was to allow interoperability between 32 and 64 bit applications using 64 bit offset types.

Many times, in a 64 bit OS, a 32 bit offset type can't hold the distance between a stack variable and a shared memory address, as the OS can place them further than 4GB.

This is not documented and it should be explained.

[Andrey Semashev]

Replying to igaztanaga:

Using signed integers is problematic because overflow is undefined behaviour, there were problems with some compilers.

Then I think the sign extension should be coded explicitly. Since offset_ptr is supposed to point to locations before itself, the offset is logically a signed value (i.e. analogous to ptrdiff_t rather than size_t).

And yes, offset_ptr was not thought to be used with integers smaller than size_t (or more strictly, smaller than uintptr_t). The option to customize the OffsetType was to allow interoperability between 32 and 64 bit applications using 64 bit offset types.

Many times, in a 64 bit OS, a 32 bit offset type can't hold the distance between a stack variable and a shared memory address, as the OS can place them further than 4GB.

This is not documented and it should be explained.

If it's absolutely not possible to use offset_ptr with offsets smaller than ptrdiff_t then there should be a static assert prohibiting that.

But I wonder if it's necessary to store offset_ptrs on the stack. Is it possible to operate with offset_ptr in-place and only store normal pointers on the stack?