Boost C++ Libraries: Ticket #13442: pmr::synchronized_pool_resource is affected by thread priority inversion https://svn.boost.org/trac10/ticket/13442 <p> In our setup, we have several high priority threads that occasionally need to allocate memory using <code>synchronized_pool_resource::allocate</code>. Such memory is later released by a lower priority thread using <code>synchronized_pool_resource::deallocate</code>. </p> <p> It might sometimes happen that several <code>allocate</code> calls happen simultaneously with <code>deallocate</code>. Because of the spinlock implementation used by dlmalloc, the high priority threads are in a busy loop trying to acquire the lock, but the kernel might have no resources left to awake the "cleaner" thread that is currently holding the lock. </p> <p> Attached is a minimum example that Linux <code>SCHED_FIFO</code> scheduler. Running the program, on my system I can see: </p> <blockquote> <p> 1) The program prints 'failed' several times, meaning that some calls to <code>allocate</code> blocked for more than a second. </p> </blockquote> <blockquote> <p> 2) The total execution time is approximately 120 times slower than a version that does not use any thread priority. </p> </blockquote> <p> Inspecting the program with <code>perf</code> one can see that most of the time is spent inside <code>boost_cont_sync_lock</code>, in particular in <code>sched_yield</code>. </p> <p> A possible solution could be to switch to a pure pthread implementation by defining <code>USE_SPIN_LOCKS=0</code> before including <code>dlmalloc_2_8_6.c</code> </p> <p> System: <code>Linux SPC-LT48 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3 (2017-12-03) x86_64 GNU/Linux</code> </p> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/13442 Trac 1.4.3 enniobarbaro@… Wed, 07 Feb 2018 09:42:03 GMT attachment set https://svn.boost.org/trac10/ticket/13442 https://svn.boost.org/trac10/ticket/13442 <ul> <li><strong>attachment</strong> → <span class="trac-field-new">main.cpp</span> </li> </ul> Ticket Ion Gaztañaga Fri, 09 Feb 2018 17:17:32 GMT <link>https://svn.boost.org/trac10/ticket/13442#comment:1 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/13442#comment:1</guid> <description> <p> I can't see any sensible fix, synchronized_pool_resourc is not thought to be prepared to handle priority inversion and several platforms don't support something like PTHREAD_PRIO_INHERIT and the default pthread implementation might not support priority inheritance protocol. I don't know if also setting this flag can affect performance for the common case. </p> <p> Maybe you should build the library using USE_SPIN_LOCKS=0 and this should be at least documented. </p> </description> <category>Ticket</category> </item> </channel> </rss>