Boost C++ Libraries: Ticket #5392: Message queue temporary dir named "Select LastBootUpTime from Win32_OperatingSystem" (on win XP)

Ion Gaztañaga — Thu, 31 Mar 2011 08:01:32 GMT

I need more context to track the bug. The second process is a process of the same user? Which error code do you get in functions between between lines 1525 and 1558?

attachment set

Ion Gaztañaga — Thu, 31 Mar 2011 08:22:56 GMT

attachment → wmi.patch

Patch for WMI to make blocking call

wsx22@… — Thu, 31 Mar 2011 13:09:26 GMT

Replying to igaztanaga:

I need more context to track the bug. The second process is a process of the same user?

Yes

Which error code do you get in functions between between lines 1525 and 1558?

I will try to debug that information later, I'm new in c++ and winapi

anonymous — Thu, 31 Mar 2011 23:19:53 GMT

I've debugged get_wmi_class_attribute function

with orginal code from 1_46_1

during new boost::interprocess::message_queue(boost::interprocess::create_only, ... ) instruction there are 3 invocations of get_wmi_class_attribute function, all of it returns correct string

first execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)
long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x00000000
pWbemServices->ExecQuery(..)
returns 0
pEnumObject->Reset()
returns 0
pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
goes into while body
pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0
strValue = vwchar.value.pbstrVal;
ok, returns correct value
return bRet;
bRet = true

second and third execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)
long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119
pWbemServices->ExecQuery(..)
returns 0
pEnumObject->Reset()
returns 0
pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
program goes into while body
pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0
strValue = vwchar.value.pbstrVal;
ok, returns correct value
return bRet;
bRet = true

during boost::interprocess::message_queue mq(boost::interprocess::open_only, ... ) instruction there are 2 invocations of get_wmi_class_attribute function, all of it returns wrong string

first and second execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)
long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 (RPC_E_TOO_LATE_IG)
pWbemServices->ExecQuery(..)
returns 0
pEnumObject->Reset()
returns 0
pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0x80041003
program not goes into while body
return bRet;
bRet = false

with orginal code from 1_46_1 witch wmi.patch

during new boost::interprocess::message_queue(boost::interprocess::create_only, ... ) instruction there are 3 invocations of get_wmi_class_attribute function, all of it returns correct string

first execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)
long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x00000000
pWbemServices->ExecQuery(..)
returns 0
pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
goes into while body
pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0
strValue = vwchar.value.pbstrVal;
ok, returns correct value
return bRet;
bRet = true

second and third execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)
long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 (RPC_E_TOO_LATE_IG)
pWbemServices->ExecQuery(..)
returns 0
pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
goes into while body
pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0
strValue = vwchar.value.pbstrVal;
ok, returns correct value
return bRet;
bRet = true

during boost::interprocess::message_queue mq(boost::interprocess::open_only, ... ) instruction there are 2 invocations of get_wmi_class_attribute function, all of it returns wrong string

first and second execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)
long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 (RPC_E_TOO_LATE_IG)
pWbemServices->ExecQuery(..)
returns 0x80041003
return bRet;
bRet = false

implications:

CoInitializeSecurity returns 0x80010119

if it was executed before

pEnumObject->Next(..)or pWbemServices->ExecQuery(..) fails if

if it was executed before from another process

this bug is related to:

https://svn.boost.org/trac/boost/ticket/4895 and https://svn.boost.org/trac/boost/ticket/4907

get_wmi_class_attribute function executes multiple times per one boost::interprocess::message_queue instruction, maybe its result should be cached somewhare

Ion Gaztañaga — Fri, 01 Apr 2011 07:56:01 GMT

0x80041003 is WBEM_E_ACCESS_DENIED which seems to be a permission issue. All executions are from the same process? Two processes from two different users?

Also, MSDN documentation says pEnumObject->Next can't return 0x80041003.

wsx22@… — Fri, 01 Apr 2011 09:54:39 GMT

creating message queue by executing
new boost::interprocess::message_queue(boost::interprocess::create_only, ... )
instruction, is done by one process.

trying to open existing message queue by executing
boost::interprocess::message_queue mq(boost::interprocess::open_only, ... )
instruction, is done by another process.

Both proceses are running on same windows user account (with administrator rights)

Also, MSDN documentation says pEnumObject->Next can't return 0x80041003.

mine pEnumObject->Next returns this code when pWbemServices->ExecQuery(...) is executed with WBEM_FLAG_RETURN_IMMEDIATELY_IG flag

when pWbemServices->ExecQuery(...) is executed with WBEM_FLAG_RETURN_WHEN_COMPLETE_IG | WBEM_FLAG_FORWARD_ONLY_IG flags, 0x80041003 is returned from pWbemServices->ExecQuery(...)

status changed; resolution set

Ion Gaztañaga — Fri, 01 Apr 2011 17:09:29 GMT

status new → closed
resolution → fixed

WMI is provoking a lot of problems for Boost Interprocess users. I've decided to remove bootstamp use in windows to obtain kernel persistence in Windows. This means that windows shared memory/queues will survive to reboots, but this behaviour is allowed by POSIX. Using bootstamps to detect reboots is doing more harm than good.

WMI is removed in Boost 1.47

wsx22@… — Sat, 02 Apr 2011 00:53:06 GMT

@igaztanaga

I have seen Your last revision 70838 at SVN. I was trying to test it but when I compiled my project with last revision (70838) of interprocess library, I've got an error

c:\c\boost_1_46_1\boost\interprocess\detal\move.hpp(17) : fatal error C1083: Cannot open include file: 'boost/move/move.hpp': No such file or directory

Last revision of interprocess library witch i can compile is 70055, I'm using Ms Visual Studio 2008.

(ps. I'm working on my patch for #5392 issue)

anonymous — Sat, 02 Apr 2011 07:50:56 GMT

Interprocess is now dependent on recently accepted Boost.Move. Download it from SVN.

attachment set

wsx22@… — Mon, 04 Apr 2011 18:14:48 GMT

attachment → boost_ticket5392.patch.zip

proposed patch, see comment 9

wsx22@… — Mon, 04 Apr 2011 18:16:21 GMT

I have written patch witch workaround WMI issue on win XP, I think message queue can be kernel persistance again. My project compiled with this patch (boost 1_46_1 svn rev. 70821 + patch) is running on my win xp for 2 days now without any problems.

Kai k: — Mon, 02 May 2011 12:23:45 GMT

Replying to igaztanaga:

WMI is removed in Boost 1.47

I have the same problem and I had a look at the road map to see when milestone 1.47 is going to be finished. Unfortunately the milestone is 2 days late! Is there any planning to release 1.47 in the near future?

maik.mory@… — Fri, 20 May 2011 17:48:18 GMT

Bug observed on Windows 7 SP1 x64 with Visual Studio 2010. boost_ticket5392.patch did the job.

Elaborate Report

I used "Process Explorer -> View -> Lower Pane View -> Handles" to inspect the name of the shared memory segment.

with boost_1_64_1 unpatched:
Process A: C:\ProgramData\boost_interprocess\20110520003539.574067\foo
Process B: C:\ProgramData\boost_interprocess\Select LastBootUpTime from Win32_OperatingSystem\foo

with boost_1_64_1 wmi.patch:
Process A: C:\ProgramData\boost_interprocess\20110520003539.574067\foo
Process B: C:\ProgramData\boost_interprocess

with boost_1_64_1 boost_ticket5392.patch:
Process A: C:\ProgramData\boost_interprocess\704368109915cc\foo
Process B: C:\ProgramData\boost_interprocess\704368109915cc\foo

Ion Gaztañaga — Sat, 21 May 2011 06:36:29 GMT

ShutdownTime is not correctly written if machine is correctly powered off. Apart from that, using WMI for Vista and 7 still has problems for some users. I'm afraid we'll need to wait for a better solution.

Kai K. — Wed, 20 Jul 2011 14:21:56 GMT

After updating to 1.47.0 this bug still exists! Please reopen this ticket or create a new one for managed_shared_memory.

I use

bipc::managed_shared_memory(bipc::create_only, SHARED_MEM_NAME, nSegmentSize);

to create shared memory and

bipc::managed_shared_memory segment(bipc::open_only, SHARED_MEM_NAME);

to acquire it in another thread. My paths:

...\boost_interprocess\Select LastBootUpTime from Win32_OperatingSystem\SHARED_MEM_NAME
...\boost_interprocess\20110720031715.524718

First directory is created while creating the shared memory. The second one is created by open it in the other process! The object file is missing in the second directory, therefore the shared memory can not be accessed!

Kai K. — Thu, 21 Jul 2011 08:58:58 GMT

Okay I had a deeper look at my problem.

inline bool get_wmi_class_attribute( std::wstring& strValue, const wchar_t *wmi_class, const wchar_t *wmi_class_var)
{
  ...
  while( 0 == pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned ) )
  {
    com_releaser<IWbemClassObject_IG> IWbemClassObject_releaser(pClassObject);
    if ( 0 == pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 ) ){
      bRet = true;
      strValue = vwchar.value.pbstrVal;
      VariantClear(&vwchar );
      break;
    }
  }
}

The call of pEnumObject->Next(...) fails with WBEM_E_ACCESS_DENIED for the first process. I'm going to check the rights of this process. Currently I have no clue why the access rights are different for the two processes.

Kai K. — Thu, 21 Jul 2011 11:33:07 GMT

I checked the creation of threads in the first process and a change of the used API (_beginthread > _beginthreadex) did not help. Therefore I search the net for issues corresponding to pEnumObject->Next(...) and the result WBEM_E_ACCESS_DENIED. I found the KB948829 article which recommends the usage of CoSetProxyBlanket. I integrated the call of CoSetProxyBlanket in get_wmi_class_attribute and now my shared memory works again. See the attached patch win32_api.hpp.#5392.patch.

attachment set

Kai K. — Thu, 21 Jul 2011 11:34:04 GMT

attachment → win32_api.hpp.#5392.patch

fixes WBEM_ACCESS_DENIED problem

anonymous — Fri, 22 Jul 2011 16:47:45 GMT

I got the same problem with boost 1.47.0 too, please re-open this ticket. Thanks.

status changed; resolution deleted

anonymous — Sat, 23 Jul 2011 06:22:50 GMT

status closed → reopened
resolution fixed

ccasciano@… — Wed, 17 Aug 2011 19:08:53 GMT

Will there be a fix for this in 1.48?

ccasciano@… — Fri, 19 Aug 2011 14:19:29 GMT

Will there be a fix for this in 1.48?

anonymous — Wed, 24 Aug 2011 16:41:55 GMT

I observed this bug on Windows 7 64 bit and Windows XP SP3 32 bit. Commenting out the

#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME

boost/interprocess/detail/tmp_dir_helpers.hpp

solved the problem for me in Boost 1.47 but its probably evil

It basically bypasses the whole check the bootstamp and create a folder named after it stuff which seems to be the root of the problem anyway. Shared memory then survives reboots though but that is no problem for my use case.

anonymous — Sun, 25 Sep 2011 02:20:12 GMT

I got the very same problem with 1.47 as well. However, it seems that it has been addressed in the trunk version, since that one works for me without problems, so I suppose it will be fixed in 1.48.

anonymous — Wed, 26 Oct 2011 08:01:41 GMT

Still happening in 1.47 on my side too. Does anyone know when a new release will be perform(1.48)? Thanks

anonymous — Tue, 08 Nov 2011 22:46:19 GMT

I have just looked at the Boost.Interprocess temp folder management on Windows, here:

http://svn.boost.org/svn/boost/trunk/boost/interprocess/detail/tmp_dir_helpers.hpp

(function get_tmp_base_dir). The bootstamp has been disabled, which sidesteps the problem of having to find a reliable way to formulate a directory name that is unique for each boot.

Because of this, however, a program running with Boost.Interprocess 1.48 will see all its shared memory objects (and probably other Boost.Interprocess constructs) deleted as soon as a program with Boost.Interprocess <=1.47 runs on this machine.

So I am suggesting that the "boost_interprocess" created by get_tmp_base_dir() directory (placed inside the Windows shared documents folder) gets renamed so that the folder cleanup done by the create_tmp_and_clean_old*() functions does not wipe 1.48 Boost.Interprocess shared memory (and other) objects. I don't really have a good suggestion for the new name... boost_interprocess_2, I guess; not sure how this aspect of the component is versioned.

I know this because I have recently patched my version of Boost 1.47 with the changes I saw coming in 1.48, and I now realize that applications using an earlier version of Boost (1.42, in my case) are cleaning up the boost_interprocess folder. Hope it's not too late for this change to be done.

Incidentally, an entry point to let the programmer override this directory name would be useful (so that one can have a real "namespace" for his or her use of Boost.Interprocess objects), but this can be discussed separately.

anonymous — Sat, 24 Dec 2011 10:27:18 GMT

I added the patch in trunk, but WMI is still disabled by default. I would be grateful if someone could uncomment

#define BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME

#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME

#include <boost/interprocess/detail/win32_api.hpp>

in boost/interprocess/detail/tmp_dir_helpers.hpp and check if shared memory works with WMI. I have no enough COM knowledge to properly get the bootstamp, so I need some help.

anonymous — Sun, 25 Mar 2012 20:35:47 GMT

This makes message_queue almost entirely unusable in some cases, causing Bitcoin-Qt to either freeze (in a nice infinite loop) or refuse to be able to create the message_queue. See patch at https://github.com/bitcoin/bitcoin/pull/986/files for a temporary workaround that wfm, though probably not in all cases.

Ion Gaztañaga — Sun, 25 Mar 2012 21:12:23 GMT

What makes message queue unusable, the original problem or curret Boost 1.49 code uncommenting the defines indicated in the previous comment?

anonymous — Mon, 26 Mar 2012 14:28:07 GMT

The problem is, that currently on Windows, there is no path for the mq files created under \boost_interprocess\ but instead all queue files reside in the root.

anonymous — Mon, 02 Apr 2012 19:22:05 GMT

Is there a chance to get these enabled as default with the next release:

#define BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME
#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME
#include <boost/interprocess/detail/win32_api.hpp>

And to include the fixed version for get_last_bootup_time() from: https://github.com/bitcoin/bitcoin/pull/986/files#L0R26

We really need a clean and working solution, as we want to use cross-platform code :).

Ion Gaztañaga — Mon, 02 Apr 2012 21:06:30 GMT

Yes, there is a chance if someone tests the trunk code enabling those defines and assures that solves the problem. And the last_bootup_time patch is not a solution, it's an optimization to cache the value that is not thread-safe, so it's broken.

If just enabling those defines in trunk code does not solve the problem, then the problem is still there.

anonymous — Mon, 02 Apr 2012 22:14:46 GMT

I'm going to try this tomorrow, thanks for the quick response!

anonymous — Tue, 03 Apr 2012 09:19:22 GMT

I DLed the current trunk version and made a file comparison, am I right, that there are no further changes in tmp_dir_helpers.hpp and win32_api.hpp in comparison to the final 1.49 release?

anonymous — Wed, 11 Apr 2012 21:51:21 GMT

@igaztanaga: After enabling the 3 lines from comment:28 it seems to work. No need to change get_last_bootup_time()!

anonymous — Sat, 09 Jun 2012 23:36:38 GMT

Uncommenting the define/includes in boost/interprocess/detail/tmp_dir_helpers.hpp works fine on Win7 where it never worked before on Boost 1.49. It also fixes a bug where, if a system crashed while a message_queue is open, trying to open will get in an infinite loop with "socket select error 10022"

anonymous — Wed, 13 Jun 2012 19:58:29 GMT

Sorry, the socket select error is unrelated, but it does cause an infinite loop and use 100% CPU.

status changed; resolution set

Ion Gaztañaga — Sun, 10 Feb 2013 21:14:44 GMT

status reopened → closed
resolution → fixed

This bug seems to be definitively solved in Boost 1.53.