attachment set

Thu, 25 Apr 2013 15:18:29 GMT

attachment → vxworks.hpp.diff

Added documentation to the OS configuration how to deal with the priority inversion scenario

anonymous — Fri, 26 Apr 2013 08:34:36 GMT

I'm not against applying this patch, but I wonder if anyone will find this information there? Is this basically an issue when using Boost.Threads? If so the thread lib docs might be a better place, or is it more general than that?

Fri, 26 Apr 2013 10:09:48 GMT

Replying to anonymous:

Is this basically an issue when using Boost.Threads? If so the thread lib docs might be a better place, or is it more general than that?

No, this is not boost.threads specific! It may even be not considered boost specific. It is a general problem when using semaphores with an real time OS. I try to explain:

POSIX defines that when using semaphores the OS should use a suitable default, except the creator of the semaphore explicitely uses a semaphore attribute different from the default. For detailed information you might take a look at this link:

http://pubs.opengroup.org/onlinepubs/7908799/xsh/pthread.h.html

Here especially see functions pthread_mutex_init(), pthread_mutexattr_init() and pthread_mutexattr_setprotocol().

Now, the default for VxWorks Real Time Processes is PTHREAD_PRIO_NONE. This may be OK for certain applications, for others it will be not: If low- and high-priority tasks will race for ownership of a resource (in fact this is not limited to a mutex!) this might lead to a deadlock with the high priority task waiting for the resource ownership, owned by the low-priority task, which on the other hand is unable to release the resource because it is not being scheduled, i.e. it does not get the cpu to do its work.

The solution for this dilema is called "priority inheritance protocol" and includes a temporary increase of the priority of the low-priority task until it releases the resource. So, if instead the mutex uses PTHREAD_PRIO_INHERIT this will pose an additional protocol overhead burden on the caller (the scheduler might reschedule a low priority task to a high priority), but it will not break functionality.

Now, the problems with potentially all boost libraries are the following:

Several libraries of boost do use semaphores. And this might change with every new release of boost. Maybe one library stops using semaphores, maybe another starts doing it.
As they are libraries by definition you cannot know the way the user intends to use them. Maybe for the specific application PTHREAD_PRIO_NONE would be sufficient, maybe PTHREAD_PRIO_INHERIT would be required. The library has no way to find out. Thus it must fall back to a version which might not give optimized performance but that is secure and does not deadlock. This means is has to use PTHREAD_PRIO_INHERIT as a protocol.
I think the alternative of giving the library user the option to chose the semaphore protocol during instanciation of a library template is not an option. This will blead out very implementation and OS-specific details to the user - surely a definitive no-go for a boost library.

So, two options how to solve this do remain:

Change every instanciation of semaphores inside boost. This can be done and is not very difficult. But it easily breaks when libraries change their usage of semaphores.
Change the default the underlying OS uses so that it always uses PTHREAD_PRIO_INHERIT. How to do this is what I added in the configuration remarks for VxWorks, in the hope that people using VxWorks will configure their boost version and take a look at the file. The problem here is that it requires you to change the OS, not boost. Wind River delivers VxWorks also as source code, so this is not a problem in itself, but it needs to be done explicitely by every user.

So, to cut a long story short:

I would suggest to commit this comments to vxworks.hpp, it doesn't hurt and it could be useful if someone stumbles upon it.
If you tell me where this should belong I volunteer to add a small chapter "VxWorks specialities" to the boost documentation, containing these hints.
I suppose this problem does not only apply to VxWorks, but to other real time OS as well, depending on what defaults they use for POSIX-semaphores. E.g. QNX could potentially suffer from the very same effects, but I have no acces to this OS here, so this is just suspicion. But if so, this should be mentioned in the documentation, too, if possible with a similar solution.
And as we are in this topic already, one last remark concerning POSIX-attributes, here regarding threads: For a real time OS it would be beneficial to be able to chose the attributes used for a thread (specifically its priority). Unfortunately, boost.threads does not give you this option. You only have the option to change the priority of a thread that is already running (non-portable via get_native_handle()). But this might very well be too late for a real time OS and this fact made me set aside boost.threads, it is not very usable for a RTOS in its current implementation. I know this has been discussed in length here already and this limitations have their reason (i.e. portability), so I didn't want to restart this over again. But in fact this is a kind of nuisance, because it means setting aside join() and other very useful functionality as well, which is a pitty. Actually I'm thinking about whether it would be possible to create a portable interface for very OS-specific details, like thread attributes, mutex attributes, etc. If this could be done somehow it would greatly enhance functionality.

John Maddock — Fri, 26 Apr 2013 10:36:18 GMT

(In [84053]) Apply patch from #8494. Refs #8494.

John Maddock — Fri, 26 Apr 2013 10:47:53 GMT

I've committed the patch, also added a stub Wiki entry for this here: https://svn.boost.org/trac/boost/wiki/Guidelines/VxWorks and sent you an invitation to sign up for the Trac system which will let you edit the Wiki. Can I get you to add the above information there?

Many thanks, John Maddock.

Peter Brockamp — Mon, 29 Apr 2013 12:08:19 GMT

Replying to johnmaddock:

... Can I get you to add the above information there?

I added the documentation from the file and put on some additional information how to set up everything. Please have a look at it whether this is OK! There's one link inside to wikipedia, I hope it's OK to link to outward sites? If not I will remove this again.

Best regards

Peter

status changed; resolution set

John Maddock — Thu, 13 Jun 2013 16:24:08 GMT

status new → closed
resolution → fixed

Thanks!

Boost C++ Libraries: Ticket #8494: Added comments how to solve the priority inversion problem

attachment set

status changed; resolution set