Boost C++ Libraries: Ticket #4960: boost::pool_allocator for vector of vectors exhausts memory

Steven Watanabe — Fri, 17 Dec 2010 17:51:13 GMT

I can't reproduce the problem with gcc 4.4.1, or with MSVC 10. I'd guess that it has something to do with the standard library implementation of vector, though.

Can you try applying the patch to singleton_pool which I'm about to attach, run the following variation and save the output? It should produce code that you can compile which demonstrates the problem. Check that it actually triggers the problem, and attach it.

#include <boost/pool/pool_alloc.hpp>
#include <vector>
#include <iostream>
typedef std::vector<int, boost::pool_allocator<int> > EventVector;
typedef std::vector<EventVector> IndexVector;
int main()
{
  std::cout << "#include <boost/pool/pool_alloc.hpp>\n";
  std::cout << "int main() {\n";
  std::cout << "void* tmp;\n";
  {
    IndexVector iv;
    int limit = 30;
    for (int i = 0; i < limit; ++i)
      iv.push_back(EventVector());
  }
  std::cout << "}\n";
  return 0;
}

attachment set

Steven Watanabe — Fri, 17 Dec 2010 17:52:19 GMT

attachment → singleton_pool.hpp.patch

Patch adding instrumentation to singleton_pool

attachment set

mattiasg — Tue, 21 Dec 2010 13:40:15 GMT

attachment → poolalloctest2_rhel5

Output of the modified test program with the patched singleton_pool.hpp

mattiasg — Tue, 21 Dec 2010 13:44:08 GMT

On a Suse 11 machine with gcc 4.3.4 I cannot reproduce the problem, but on a RHEL5 machine with gcc 4.1.2 a run with your patched singleton_pool.hpp produces the output which I've attached to this issue. To avoid crashing the server I ran the test on I had to abort the program around the time it used half (~16Gb) of the available memory.

Steven Watanabe — Sun, 02 Jan 2011 03:19:57 GMT

Sorry for the delay. The file you attached is an executable. I don't think it's the output.

attachment set

mattiasg — Tue, 04 Jan 2011 13:32:35 GMT

attachment → rhel5_out2

Proper output file

mattiasg — Tue, 04 Jan 2011 13:33:38 GMT

Ahhh, crap. Attached the output file now, sorry.

cc set

John Maddock — Fri, 07 Jan 2011 11:27:04 GMT

cc boostpool@… added

John Maddock — Mon, 14 Feb 2011 18:44:04 GMT

I can reproduce this with gcc-4.0.4 as well. Strange bug!

John Maddock — Tue, 15 Feb 2011 18:34:04 GMT

Update:

This is a result of two bugs:

1) std::vector invokes undefined behaviour by calling Allocator::allocate(0). This has been fixed in more recent GCC versions. 2) Boost.Pool, when it tries to allocate 0 chunks thinks allocation has failed (a bug), so allocates a whole new block, *and* ups the size of the next block to allocate by a factor of 2. So just a few allocation requests for 0 chunks blows up the heap.

This will be fixed in the sandbox version of Boost.Pool soon.

John Maddock — Tue, 15 Feb 2011 18:39:10 GMT

(In [68920]) Add test case for bug #4960. Add conditional instrumentation code to headers so we can see what's being allocated when. Fix issue #4960 by allowing pool to allocate 0 chunks. Refs #4960.

status changed; resolution set

John Maddock — Tue, 02 Aug 2011 17:04:12 GMT

status new → closed
resolution → fixed

(In [73495]) Merge updated Pool lib from trunk. Fixes #1252. Fixes #2696. Fixes #4960. Fixes #5526. Fixes #5568. Fixes #5700.