Boost C++ Libraries: Ticket #8305: Allow interprocess to work with big files on x32 POSIX systems

attachment set

Antony Polukhin — Mon, 18 Mar 2013 15:13:28 GMT

attachment → os_file_functions.hpp.patch

Antony Polukhin — Mon, 18 Mar 2013 15:23:51 GMT

#7295 will be partially fixed by this

Ion Gaztañaga — Mon, 18 Mar 2013 22:27:29 GMT

I have some questions about the patch. If I understand LFS correctly:

If _LARGEFILE64_SOURCE is defined, then we have new functions and types like open64/off64_t...

If _FILE_OFFSET_BITS=64 then "open"/"off_t"... are 64 bit ready.

Using O_LARGEFILE does not seem to guarantee compatibility as _FILE_OFFSET_BITS can be 32 and off_t would be 32 bits. We could open files bigger than 2GB but no seek or map them.

Maybe the solution is to use xxx64 functions and types if _LARGEFILE64_SOURCE is defined and _FILE_OFFSET_BITS is not defined or it's equal to 32. Or am I missing something?

Antony Polukhin — Tue, 19 Mar 2013 07:15:29 GMT

Replying to igaztanaga:

I have some questions about the patch. If I understand LFS correctly:

If _LARGEFILE64_SOURCE is defined, then we have new functions and types like open64/off64_t...

If _FILE_OFFSET_BITS=64 then "open"/"off_t"... are 64 bit ready.

Using O_LARGEFILE does not seem to guarantee compatibility as _FILE_OFFSET_BITS can be 32 and off_t would be 32 bits. We could open files bigger than 2GB but no seek or map them.

Maybe the solution is to use xxx64 functions and types if _LARGEFILE64_SOURCE is defined and _FILE_OFFSET_BITS is not defined or it's equal to 32. Or am I missing something?

After two more rereads of LSF I've finally got it right: To work with large files on x32 user only needs to define _FILE_OFFSET_BITS=64, so patch is incorrect.

But maybe forcing x32 users to define _FILE_OFFSET_BITS=64 will be a good idea:

#if !defined(__x86_64__) && !defined(__ppc64__) && \
 (!defined(_FILE_OFFSET_BITS) || _FILE_OFFSET_BITS==32)
#error "To avoid errors while using big files define _FILE_OFFSET_BITS macro to 64 for your project."
#endif

, otherwise this may lead to hard detectable errors.

t.glaser@… — Tue, 11 Aug 2015 09:01:22 GMT

x32 does define x86_64 though, you need to check for ILP32 which is defined on x32 but not on i386 or amd64.