perfomance: memory cleanup for pool takes too long

[nobody]

hi,

In my project I use boost::pool because I have to 
instantiate millions of object at the beginning of the 
application, and only get rid of the at termination time.

I noticed that when processing large data
(which may require 1Gb memory for the application while 
it is running)
the application may finish its processing approx after 10 
seconds.
the cleanup part (which consists only of calling the pool 
release methods on allocated objects)
on the other hand, takes many minutes
(I usually give up after 5 minutes)

debugging the code i noticed that the while loop in
the nextof function 
(of SIMPLE SEGREGATED STORAGE.HPP)

takes many iterations.
this leads me to the belief that the reason the cleanup 
takes too long is because trying to cleanup 1GB memory 
of million of objects at once causes 
pool to retain all its chunks in the hope of more 
allocations in the future
which in turns causes it to waste all computation on 
running to the end of (endless) lists in the nextof 
function

I will appreciate a suggestion on how to fix this
until a more permanent solution is applied to the pool 
library

thanks
aviad
aviadr@pob.huji.ac.il

[nobody]

Logged In: NO 


If the objects are just allocated once at the beginning of the app, and then destroyed at the very end, it's a very simple memory management scenario.

Why not just create a very simple allocator that just steps along a boost::array, allocating bytes from it? The allocator would never need to reuse data so the task is a lot simpler.

Ideal would be for boost::poool to handle that, but this should get you out of trouble.

-- Matthew