Boost C++ Libraries: Ticket #9019: Bug: mapped_region shouldn't create a new file mapping but reuse the one provided in file_mapping

Ion Gaztañaga — Tue, 20 Aug 2013 17:30:38 GMT

Calling CreateFileMapping in the file_mapping class is not a trivial task as the size of the file can change and the file mapping would need to be recreated on the fly. Also, it wouldn't allow the same file using copy on write as the file mapping must be opened with an special flag for that purpose, whereas another view would like to write directly to the file. Do you have any performance numbers about the overhead of not reusing the file mapping? All portable shared memory interfaces I know create new file mappings when mapping the memory so I guess the performance impact is not very high.

Ion Gaztañaga — Tue, 20 Aug 2013 17:36:29 GMT

Just as additional information, I've found:

http://www.labri.fr/perso/betrema/winnt/manamemo.html

"To use a memory-mapped file, you start by creating a memory-mapped file object. The act of creating an MMF object has very little impact on system resources. It does not affect your process's address space, and no virtual memory is allocated for the object (other than for the internal resources that are necessary in representing the object)."

Maybe it has no big impact in system resources but it could be much slower than reusing the object.

michal.fronczyk@… — Tue, 20 Aug 2013 21:55:41 GMT

Thanks for such a quick response. I think the problem is not about the time needed to create file mapping, but about the lack of ability to reuse already fetched pages of the file in different mapped regions. The performance problem we observed occurred in a multithreaded program. Using a profiler we discovered that the problem lies in the code where many threads were creating read-only mapped_regions of the same file using a shared instance of the file_mapping class. Different threads mapped different regions of the file, but there was a very high probability that a thread will shortly need the data region mapped previously by one of the other threads. I'm assuming that the destructor of the mapped_region class releases the mapping, so all the pages fetched during using that region are released when the region is destroyed and need to be fetched again from the disk when another region maps the same part of the file. Fetching the same parts of the file many times hurts the performance in our case.

We changed the code to create one shared read-only mapped_region that maps the whole file and we use pointer arithmetic to read different parts of the file in different threads. The performance gain we observed was significant, but I don't have the numbers now. However, this solution isn't ideal because we can't do that in the 32bit version of the app since the file is big (several GBs). We map the whole file in the 64bit version only. BTW, do you see any problems with mapping a 20GB file as a whole to the 64bit address space?

My thinking is that keeping the file_mapping instance and reusing it in multiple mapped_regions, that map small portions of the file, will allow us to reuse the fetched file pages in different mapped_regions as needed and the performance will be more or less the same as in the case of one shared mapped_region that maps the whole file. Please let me know if I'm wrong.

Ion Gaztañaga — Wed, 21 Aug 2013 09:35:42 GMT

I think all file mappings share the same cached pages as coherency must be maintained:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366763(v=vs.85).aspx

"With one important exception, file views derived from any file mapping object that is backed by the same file are coherent or identical at a specific time. Coherency is guaranteed for views within a process and for views that are mapped by different processes."

(The exception is network files)

I don't think the issue is related to destroying the file mapping (in fact the file mapping is destroyed just after mapping the memory with MapViewOfFile), but because the mapped_region is unmapped (UnmapViewOfFile). The OS knows that no other thread is mapping those pages so it could think it's a good opportunity to release them (although if there is enough file cache, it should be still there). I'm not sure how this can be fixed as it's quite OS dependent.

Mapping the whole file shouldn't be a problem, at least in windows, as physical memory committed when the page is read (although I guess Windows will prefetch some pages).

michal.fronczyk@… — Fri, 23 Aug 2013 13:10:54 GMT

Thanks for explanation. You're probably right - this can't be fixed in an OS independent way.

status changed; resolution set

Ion Gaztañaga — Fri, 13 Sep 2013 20:29:38 GMT

status new → closed
resolution → fixed

I'm closing this as it does not seem there is a solution for it. Please reopen it if you have more information to improve the situation. Thanks for the report.