id summary reporter owner description type status milestone component version severity resolution keywords cc 4374 Shared Memory Bus Error Matt Cupp Ion Gaztañaga "Found on StackOverflow at http://stackoverflow.com/questions/3103255/boostinterprocess-shared-memory-bus-error I'm using CentOS 5.4 x86_64 and Boost 1.42.0 on a cluster that uses Open-MPI 1.3.3. I'm writing a shared library that uses shared memory to store large amounts of data for multiple processes to use. There's also a loader application that will read in the data from the files and load them into the shared memory. When I run the loader application, it determines the amount of memory that it needs to store the data exactly then adds 25% for overhead. For just about every file, it'll be over 2 gigs worth of data. When I make the memory request using Boost's Interprocess library, it says it has successfully reserved the requested amount of memory. But when I use start to use it, I get a ""Bus error"". From what I can tell, the bus error is a result of accessing memory outside the range that is available for the memory segment. So I started looking into how the shared memory is on Linux and what to check to make sure my system is correctly configured to allow that large amount of shared memory. 1. I looked at the ""files"" at /proc/sys/kernel/shm*: * shmall - 4294967296 (4 Gb) * shmmax - 68719476736 (68 Gb) * shmmni - 4096 2. I called the ipcs -lm command: ------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 67108864 max total shared memory (kbytes) = 17179869184 min seg size (bytes) = 1 From what I can tell, those settings indicate that I should be able to allocate enough shared memory for my purposes. So I created a stripped down program that created large amounts of data in shared memory and compiled it with the line: g++ ShmTest.cpp -lboost_system -lrt Then ran it and attached output (edited to make it smaller). If you run the program with the a parameter (any will work, just need to increase the argc), it preallocate the vector but will still result in a bus error at the same array index. I checked the size of the ""files"" at /dev/shm using the ls -ash /dev/shm command: total 2.0G 0 . 0 .. 2.0G ShmTest And just like with my original application it the size of the allocated shared memory is capped at 2 gigs. Given that it ""successfully"" allocated 2352000000 bytes of memory, in gigabytes (using 1024*1024*1024) it should be 2.19 Gb. On Linux the shared memory mechanisms it uses (tmpfs) by default limits it to half the system RAM. So on the my cluster it’s 2 Gb because we have 4 Gb system RAM. So when it tried to allocate the shared memory segment, it allocated up to the max size left on the /dev/shm. But issue came when the Boost library didn’t indicate an error or even the report correct amount of free memory when it couldn’t allocate the requested amount of memory. It was just happy to apparently chug along until it reached the end of the segment and then errored. The long term solution is to update the /etc/fstab file to make the change permanently, but a command line call can be run to increase the size of the available shared memory on each node until reboot. mount -o remount,size=XXX /dev/shm Where the XXX is the amount of memory to make available (for example size=4G). Shouldn't Boost have a way to detect that it doesn't actually have the full amount available? Instead of blindly letting the user allocate memory and then error?" Bugs closed Boost 1.44.0 interprocess Boost 1.42.0 Problem invalid shm ipc tmpfs