Message queue temporary dir named "Select LastBootUpTime from Win32_OperatingSystem" (on win XP)

[wsx22@…]

There is still problem with message queue temp directory related to bug ​https://svn.boost.org/trac/boost/ticket/4010

I'm using boost interprocess message queue on Windows XP SP3 32bit, and boost 1_46_1

In first application i'm executing function

mq = new boost::interprocess::message_queue(
		boost::interprocess::create_only 
		,mqName->c_str()
		,10
		,sizeof(MqMessageStruct)
	);

then boost creates temp directory

C:\Documents and Settings\All Users\Application Data\boost_interprocess\20110327091835.364787\

this is OK. In second application I try to create message_queue object by execute constructor

boost::interprocess::message_queue mq(
		boost::interprocess::open_only
		,mqName->c_str()
	);

boost creates temp directory named:

C:\Documents and Settings\All Users\Application Data\boost_interprocess\Select LastBootUpTime from Win32_OperatingSystem\

and throws exception. Existing message queue can't be open and can't work. In boost 1_44 this code was working good (except issue described in ticket 4010)

I've watching source code and I think that problem is somewhere in file boost_1_46_1\boost\interprocess\detail\win32_api.hpp in function inline bool get_wmi_class_attribute( std::wstring& strValue, const wchar_t *wmi_class, const wchar_t *wmi_class_var) between lines 1525 and 1558. Maybe pEnumObject Object in line 1547 is empty.

Another issue is that in file \boost_1_46_1\boost\interprocess\detail\tmp_dir_helpers.hpp in function inline void get_bootstamp(std::string &s, bool add = false) (line 38) there is no check for result returned from winapi::get_last_bootup_time(bootstamp); (line 41)

[Ion Gaztañaga]

I need more context to track the bug. The second process is a process of the same user? Which error code do you get in functions between between lines 1525 and 1558?

[Ion Gaztañaga]

Patch for WMI to make blocking call

[wsx22@…]

Replying to igaztanaga:

I need more context to track the bug. The second process is a process of the same user?

Yes

Which error code do you get in functions between between lines 1525 and 1558?

I will try to debug that information later, I'm new in c++ and winapi

[anonymous]

I've debugged get_wmi_class_attribute function

with orginal code from 1_46_1

during new boost::interprocess::message_queue(boost::interprocess::create_only, ... ) instruction there are 3 invocations of get_wmi_class_attribute function, all of it returns correct string

first execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)

long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x00000000

pWbemServices->ExecQuery(..)
returns 0

pEnumObject->Reset()
returns 0

pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
goes into while body 

pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0

strValue = vwchar.value.pbstrVal;
ok, returns correct value

return bRet;
bRet = true

second and third execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)

long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 

pWbemServices->ExecQuery(..)
returns 0

pEnumObject->Reset()
returns 0

pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
program goes into while body 

pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0

strValue = vwchar.value.pbstrVal;
ok, returns correct value

return bRet;
bRet = true

during boost::interprocess::message_queue mq(boost::interprocess::open_only, ... ) instruction there are 2 invocations of get_wmi_class_attribute function, all of it returns wrong string

first and second execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)

long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 (RPC_E_TOO_LATE_IG)

pWbemServices->ExecQuery(..)
returns 0

pEnumObject->Reset()
returns 0

pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0x80041003
program not goes into while body 

return bRet;
bRet = false

with orginal code from 1_46_1 witch wmi.patch

during new boost::interprocess::message_queue(boost::interprocess::create_only, ... ) instruction there are 3 invocations of get_wmi_class_attribute function, all of it returns correct string

first execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)

long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x00000000

pWbemServices->ExecQuery(..)
returns 0

pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
goes into while body 

pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0

strValue = vwchar.value.pbstrVal;
ok, returns correct value

return bRet;
bRet = true

second and third execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)

long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 (RPC_E_TOO_LATE_IG)

pWbemServices->ExecQuery(..)
returns 0

pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned )
returns 0
goes into while body 

pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 )
returns 0

strValue = vwchar.value.pbstrVal;
ok, returns correct value

return bRet;
bRet = true

during boost::interprocess::message_queue mq(boost::interprocess::open_only, ... ) instruction there are 2 invocations of get_wmi_class_attribute function, all of it returns wrong string

first and second execution looks like this:

long co_init_ret = CoInitialize(0);
co_init_ret = 0x00000001 (S_FALSE_IG)

long sec_init_ret = CoInitializeSecurity
sec_init_ret = 0x80010119 (RPC_E_TOO_LATE_IG)

pWbemServices->ExecQuery(..)
returns 0x80041003

return bRet;
bRet = false

implications:

• CoInitializeSecurity returns 0x80010119

if it was executed before

• pEnumObject->Next(..)or pWbemServices->ExecQuery(..) fails if

if it was executed before from another process

• this bug is related to:

​https://svn.boost.org/trac/boost/ticket/4895 and ​https://svn.boost.org/trac/boost/ticket/4907

• get_wmi_class_attribute function executes multiple times per one boost::interprocess::message_queue instruction, maybe its result should be cached somewhare

[Ion Gaztañaga]

0x80041003 is WBEM_E_ACCESS_DENIED which seems to be a permission issue. All executions are from the same process? Two processes from two different users?

Also, MSDN documentation says pEnumObject->Next can't return 0x80041003.

[wsx22@…]

creating message queue by executing
new boost::interprocess::message_queue(boost::interprocess::create_only, ... )
instruction, is done by one process.

trying to open existing message queue by executing
boost::interprocess::message_queue mq(boost::interprocess::open_only, ... )
instruction, is done by another process.

Both proceses are running on same windows user account (with administrator rights)

Also, MSDN documentation says pEnumObject->Next can't return 0x80041003.

mine pEnumObject->Next returns this code when pWbemServices->ExecQuery(...) is executed with WBEM_FLAG_RETURN_IMMEDIATELY_IG flag

when pWbemServices->ExecQuery(...) is executed with WBEM_FLAG_RETURN_WHEN_COMPLETE_IG | WBEM_FLAG_FORWARD_ONLY_IG flags, 0x80041003 is returned from pWbemServices->ExecQuery(...)

[Ion Gaztañaga]

WMI is provoking a lot of problems for Boost Interprocess users. I've decided to remove bootstamp use in windows to obtain kernel persistence in Windows. This means that windows shared memory/queues will survive to reboots, but this behaviour is allowed by POSIX. Using bootstamps to detect reboots is doing more harm than good.

WMI is removed in Boost 1.47

[wsx22@…]

@igaztanaga

I have seen Your last revision 70838 at SVN. I was trying to test it but when I compiled my project with last revision (70838) of interprocess library, I've got an error

c:\c\boost_1_46_1\boost\interprocess\detal\move.hpp(17) : fatal error C1083: Cannot open include file: 'boost/move/move.hpp': No such file or directory

Last revision of interprocess library witch i can compile is 70055, I'm using Ms Visual Studio 2008.

(ps. I'm working on my patch for #5392 issue)

[anonymous]

Interprocess is now dependent on recently accepted Boost.Move. Download it from SVN.

[wsx22@…]

proposed patch, see comment 9

[wsx22@…]

I have written patch witch workaround WMI issue on win XP, I think message queue can be kernel persistance again. My project compiled with this patch (boost 1_46_1 svn rev. 70821 + patch) is running on my win xp for 2 days now without any problems.

[Kai k:]

Replying to igaztanaga:

WMI is removed in Boost 1.47

I have the same problem and I had a look at the road map to see when milestone 1.47 is going to be finished. Unfortunately the milestone is 2 days late! Is there any planning to release 1.47 in the near future?

[maik.mory@…]

Bug observed on Windows 7 SP1 x64 with Visual Studio 2010. boost_ticket5392.patch did the job.

Elaborate Report

I used "Process Explorer -> View -> Lower Pane View -> Handles" to inspect the name of the shared memory segment.

with boost_1_64_1 unpatched:
Process A: C:\ProgramData\boost_interprocess\20110520003539.574067\foo
Process B: C:\ProgramData\boost_interprocess\Select LastBootUpTime from Win32_OperatingSystem\foo

with boost_1_64_1 wmi.patch:
Process A: C:\ProgramData\boost_interprocess\20110520003539.574067\foo
Process B: C:\ProgramData\boost_interprocess

with boost_1_64_1 boost_ticket5392.patch:
Process A: C:\ProgramData\boost_interprocess\704368109915cc\foo
Process B: C:\ProgramData\boost_interprocess\704368109915cc\foo

[Ion Gaztañaga]

ShutdownTime is not correctly written if machine is correctly powered off. Apart from that, using WMI for Vista and 7 still has problems for some users. I'm afraid we'll need to wait for a better solution.

[Kai K.]

After updating to 1.47.0 this bug still exists! Please reopen this ticket or create a new one for managed_shared_memory.

I use

bipc::managed_shared_memory(bipc::create_only, SHARED_MEM_NAME, nSegmentSize);

to create shared memory and

bipc::managed_shared_memory segment(bipc::open_only, SHARED_MEM_NAME);

to acquire it in another thread. My paths:

...\boost_interprocess\Select LastBootUpTime from Win32_OperatingSystem\SHARED_MEM_NAME
...\boost_interprocess\20110720031715.524718

First directory is created while creating the shared memory. The second one is created by open it in the other process! The object file is missing in the second directory, therefore the shared memory can not be accessed!

[Kai K.]

Okay I had a deeper look at my problem.

inline bool get_wmi_class_attribute( std::wstring& strValue, const wchar_t *wmi_class, const wchar_t *wmi_class_var)
{
  ...
  while( 0 == pEnumObject->Next( WBEM_INFINITE_IG, uCount, &pClassObject, &uReturned ) )
  {
    com_releaser<IWbemClassObject_IG> IWbemClassObject_releaser(pClassObject);
    if ( 0 == pClassObject->Get( L"LastBootUpTime", 0, &vwchar, 0, 0 ) ){
      bRet = true;
      strValue = vwchar.value.pbstrVal;
      VariantClear(&vwchar );
      break;
    }
  }
}

The call of pEnumObject->Next(...) fails with WBEM_E_ACCESS_DENIED for the first process. I'm going to check the rights of this process. Currently I have no clue why the access rights are different for the two processes.

[Kai K.]

I checked the creation of threads in the first process and a change of the used API (_beginthread > _beginthreadex) did not help. Therefore I search the net for issues corresponding to pEnumObject->Next(...) and the result WBEM_E_ACCESS_DENIED. I found the ​KB948829 article which recommends the usage of CoSetProxyBlanket. I integrated the call of CoSetProxyBlanket in get_wmi_class_attribute and now my shared memory works again. See the attached patch win32_api.hpp.#5392.patch.

[Kai K.]

fixes WBEM_ACCESS_DENIED problem

[anonymous]

I got the same problem with boost 1.47.0 too, please re-open this ticket. Thanks.

[ccasciano@…]

Will there be a fix for this in 1.48?

[ccasciano@…]

Will there be a fix for this in 1.48?

[anonymous]

I observed this bug on Windows 7 64 bit and Windows XP SP3 32 bit. Commenting out the

#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME

in

boost/interprocess/detail/tmp_dir_helpers.hpp

solved the problem for me in Boost 1.47 but its probably evil

It basically bypasses the whole check the bootstamp and create a folder named after it stuff which seems to be the root of the problem anyway. Shared memory then survives reboots though but that is no problem for my use case.

[anonymous]

I got the very same problem with 1.47 as well. However, it seems that it has been addressed in the trunk version, since that one works for me without problems, so I suppose it will be fixed in 1.48.

[anonymous]

Still happening in 1.47 on my side too. Does anyone know when a new release will be perform(1.48)? Thanks

[anonymous]

I have just looked at the Boost.Interprocess temp folder management on Windows, here:

​http://svn.boost.org/svn/boost/trunk/boost/interprocess/detail/tmp_dir_helpers.hpp

(function get_tmp_base_dir). The bootstamp has been disabled, which sidesteps the problem of having to find a reliable way to formulate a directory name that is unique for each boot.

Because of this, however, a program running with Boost.Interprocess 1.48 will see all its shared memory objects (and probably other Boost.Interprocess constructs) deleted as soon as a program with Boost.Interprocess <=1.47 runs on this machine.

So I am suggesting that the "boost_interprocess" created by get_tmp_base_dir() directory (placed inside the Windows shared documents folder) gets renamed so that the folder cleanup done by the create_tmp_and_clean_old*() functions does not wipe 1.48 Boost.Interprocess shared memory (and other) objects. I don't really have a good suggestion for the new name... boost_interprocess_2, I guess; not sure how this aspect of the component is versioned.

I know this because I have recently patched my version of Boost 1.47 with the changes I saw coming in 1.48, and I now realize that applications using an earlier version of Boost (1.42, in my case) are cleaning up the boost_interprocess folder. Hope it's not too late for this change to be done.

Incidentally, an entry point to let the programmer override this directory name would be useful (so that one can have a real "namespace" for his or her use of Boost.Interprocess objects), but this can be discussed separately.

[anonymous]

I added the patch in trunk, but WMI is still disabled by default. I would be grateful if someone could uncomment

#define BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME

#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME

#include <boost/interprocess/detail/win32_api.hpp>

in boost/interprocess/detail/tmp_dir_helpers.hpp and check if shared memory works with WMI. I have no enough COM knowledge to properly get the bootstamp, so I need some help.

[anonymous]

This makes message_queue almost entirely unusable in some cases, causing Bitcoin-Qt to either freeze (in a nice infinite loop) or refuse to be able to create the message_queue. See patch at ​https://github.com/bitcoin/bitcoin/pull/986/files for a temporary workaround that wfm, though probably not in all cases.

[Ion Gaztañaga]

What makes message queue unusable, the original problem or curret Boost 1.49 code uncommenting the defines indicated in the previous comment?

[anonymous]

The problem is, that currently on Windows, there is no path for the mq files created under \boost_interprocess\ but instead all queue files reside in the root.

[anonymous]

Is there a chance to get these enabled as default with the next release:

#define BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME
#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME
#include <boost/interprocess/detail/win32_api.hpp>

And to include the fixed version for get_last_bootup_time() from: ​https://github.com/bitcoin/bitcoin/pull/986/files#L0R26

We really need a clean and working solution, as we want to use cross-platform code :).

[Ion Gaztañaga]

Yes, there is a chance if someone tests the trunk code enabling those defines and assures that solves the problem. And the last_bootup_time patch is not a solution, it's an optimization to cache the value that is not thread-safe, so it's broken.

If just enabling those defines in trunk code does not solve the problem, then the problem is still there.

[anonymous]

I'm going to try this tomorrow, thanks for the quick response!

[anonymous]

I DLed the current trunk version and made a file comparison, am I right, that there are no further changes in tmp_dir_helpers.hpp and win32_api.hpp in comparison to the final 1.49 release?

[anonymous]

@igaztanaga: After enabling the 3 lines from comment:28 it seems to work. No need to change get_last_bootup_time()!

[anonymous]

Uncommenting the define/includes in boost/interprocess/detail/tmp_dir_helpers.hpp works fine on Win7 where it never worked before on Boost 1.49. It also fixes a bug where, if a system crashed while a message_queue is open, trying to open will get in an infinite loop with "socket select error 10022"

[anonymous]

Sorry, the socket select error is unrelated, but it does cause an infinite loop and use 100% CPU.

[Ion Gaztañaga]

This bug seems to be definitively solved in Boost 1.53.