Boost C++ Libraries: Ticket #4874: multi_array compile errors using Visual C++ 2010 in debug mode

Marshall Clow — Wed, 19 Jan 2011 19:37:23 GMT

Does this problem still exist? (My copy of VS 2010 is on order)

johan.seland@… — Fri, 25 Feb 2011 21:14:56 GMT

Yes.

This problem still exist when compiling in debug mode.

The final patch in [1] does not solve the problem, and triggers warnings of the type:

c:\program files (x86)\microsoft visual studio 10.0\vc\include\xutility(478): warning C4700: uninitialized local variable '_Cat' used

Paul.Choudhury@… — Fri, 29 Apr 2011 15:55:42 GMT

I am in the process of porting my application to VS2010 and Boost v1.46.1. My apps usage of boost::multi_array<double,3> is triggering the error described here. When can I expect a patch that works? I really need multi_array to work in both release and debug modes.

anonymous — Sun, 01 May 2011 18:17:41 GMT

I confirm this bug in 1.46.1 version. My code did compile perfectly in VS2008 and does not in VS2010 debug mode (32 bits)

version changed

anonymous — Tue, 10 May 2011 14:32:03 GMT

version Boost 1.45.0 → Boost 1.47.0

I can confirm this bug in boost version 1.47.

r.undheim@… — Tue, 07 Jun 2011 13:14:28 GMT

You could try to path code with problem (in view.hpp) to something like:

  multi_array_view& operator=(const multi_array_view& other) {
    if (&other != this) {
      // make sure the dimensions agree
      BOOST_ASSERT(other.num_dimensions() == this->num_dimensions());
      BOOST_ASSERT(std::equal(other.shape(),
                              other.shape()+this->num_dimensions(),
                              this->shape()));
#if _MSC_VER >= 1600
      auto
        iterThis = begin();
      auto
        iterOther = other.begin();
      for (; iterThis != end(); ++iterThis, ++iterOther)
        *iterThis = *iterOther;
#else
      // iterator-based copy
      std::copy(other.begin(),other.end(),begin());
#endif
    }
    return *this;
  }

Just to get the code through compiler, and wait for a proper fix later.

anonymous — Mon, 01 Aug 2011 18:32:32 GMT

This bug still exists and is preventing our builds in debug mode. Is there a chance it will be fixed? As always, thanks for what you do! :)

Thu, 08 Sep 2011 10:10:44 GMT

The same behavior is with all boost versions started at least from 1.33.1 up to the latest one (1.47.0). Is anybody dealing with it? It doesn't look like that is going to be fixed. Why then boost reports that it compatible with VS2010 ???

anonymous@… — Thu, 15 Sep 2011 09:41:50 GMT

I was having the same problem. During compilation I got these warnings as well:

1>c:\program files\microsoft visual studio 10.0\vc\include\xutility(478):
warning C4700: uninitialized local variable '_Cat' used

When I ran the application it crashed at the std::copy() in view.hpp but also in subarray.hpp. I had to apply the change suggested by r.undheim to both files and now it runs.

Thanks!

Thu, 15 Sep 2011 10:49:56 GMT

There is a comment from Microsoft support:

http://connect.microsoft.com/VisualStudio/feedback/details/687655/compilation-issue-in-debug-mode-with-boost-library

(IDL refers to _ITERATOR_DEBUG_LEVEL, which is 0 in release mode and 2 in debug mode.)

-- The problem is that multi_array iterators report themselves as input iterators, which can't be written to (e.g. as the destination of a copy()). Under IDL=0, copy() doesn't actually care how its destination iterator is marked with an iterator category tag, it just tries to write to the destination iterator, and that works. Under IDL > 0, copy() wants to know whether its source and destination iterators are both random-access, in which case it can perform a range check and unwrap the destination iterator for increased performance. It does this by overloading foo(input_iterator_tag, output_iterator_tag) and foo(random_access_iterator_tag, random_access_iterator_tag) and calling foo(category(source), category(destination)). If both source and destination are random-access, both overloads are viable (i.e. callable) and the second overload is preferred, so it's actually called. Otherwise, source is required to be at least an input iterator (maybe stronger) and destination is required to be at least an output iterator (maybe stronger) so the first overload must be viable and the second overload will be non-viable, so the first overload wins by default.

However, if the destination is not marked as an output iterator or stronger, then the first overload is non-viable as well as the second, so overload resolution fails and boom.

Stephan T. Lavavej Visual C++ Libraries Developer

So based on that comment I only can add the only solution would be the one provided by r.undheim.

And again all version of boost do not compatible with Visual Studio 2010 (till 1.47.0 at least).

hegsie@… — Fri, 16 Sep 2011 13:23:53 GMT

Does anyone know of a timeline for the fix for this issue? Is it a serious change or just a patch? Cheers

istanbul.tr@… — Thu, 13 Oct 2011 03:14:49 GMT

Here is another workaround (and a probably more general fix):

Open boost/multi_array/iterator.hpp
Replace first occurrence of boost::random_access_traversal_tag (Line 57 in version 1.47.0) with std::random_access_iterator_tag

Search second (line 75), replace that whole typedef with

  typedef typename array_iterator::iterator_facade_ facade_type;

hth

anonymous — Mon, 07 Nov 2011 16:55:55 GMT

Can anybody confirm that istanbul.tr's fix works and is standards complaint? Hopefully there will be an official fix soon as this issue essentially prevents multi_array from being used in any code that needs to be compatible with VS2010 debug builds.

anonymous — Wed, 09 Nov 2011 20:45:17 GMT

Not sure whether it is 100% standard compatible but it works well in the VS2010 debug mode.

Ronald Garcia — Sat, 07 Jan 2012 19:46:53 GMT

istanbul.tr's fix is most definitely not a safe fix. Your code may crash at runtime. see https://svn.boost.org/trac/boost/ticket/4539

Ronald Garcia — Tue, 10 Jan 2012 03:31:28 GMT

I have checked a possible fix into the trunk of boost. I would appreciate feedback on whether the changes fix the problem.

Ronald Garcia — Wed, 11 Jan 2012 02:52:37 GMT

After looking into this some more, VC++ 2010 may be incorrect in its implementation of copy. Due to some ambiguity in the C++ standard, it appears that any iterator that satisfies input_iterator or less can be an output_iterator without having an iterator_category that is convertible to output_iterator_tag. It turns out that random_access_iterator_tag and friends are not convertible to output_iterator_tag, and the standard simply says that an iterator must have as its tag its most specific tag. output_iterator_tag gets used in the standard for things like back_insert_iterator, which are not even input_iterators. Observe that the Boost Concept checking library does no iterator_category checking whatsoever for the OutputIterator Concept.

Bill Buklis — Wed, 08 Feb 2012 21:23:45 GMT

So far the version in the trunk works for me. It compiles clean and seems to work as intended. Can this version be put in to the 1.49 beta? Currently 1.49 is using the old broken version. I'd hate for this issue to continue in yet another boost release.

Ronald Garcia — Thu, 09 Feb 2012 23:55:50 GMT

Before I commit this change to a release I would like to be sure that the current trunk changes are not covering what might actually be an error in VC++ 2010. It may well be that VC is incorrect in requiring that an iterator have output_iterator_tag as its category. The standard seems ambiguous about this.

anonymous — Mon, 13 Feb 2012 20:18:55 GMT

I have asked about this issue on comp.std.c++, and from what I have heard so far, my concerns about Visual C++'s current behavior and my potential "fix" for multi_array are justified. The C++ standard is ambiguous around these issues. The most relevant text that I can find in my copy of a standard draft, (24.3.3) says: For every iterator of type Iterator, iterator_traits<Iterator>::iterator_category shall be defined to be the most specific category tag that describes the iterator’s behavior.

It appears that the intent of the standard suggests:

1) That the iterator category specifically BE an instance of one of the iterator tags, not simply CONVERTIBLE TO one. This makes my potential fix of providing an input_output_iterator_tag illegal.

2) On this basis, it appears that output_iterator_tag plays the role of a placeholder, and not a normative requirement. Looking further at the standard, the classes that have output_iterator_tag as their category are classes like back_inserter and ostream_iterator, that do not satisfy the requirements of ANY OTHER iterator category, but since they are required by the above text to have an iterator tag, it simply must be output_iterator_tag, because otherwise they would violate the standard by not having a tag or having a non-applicable tag. As such, it seems that output_iterator_tag just fills that space and is otherwise non-normative.

By this interpretation, Visual C++'s debug-mode implementation is incorrect because checking for output_iterator_tag is not a requirement of an output iterator, it's just something that is true when an iterator is ONLY an output iterator and thus not qualified to have any other tag.

cc set

Bill Buklis — Wed, 22 Feb 2012 17:52:49 GMT

cc Bill Buklis added

You're probably right as I know that VS2005 and VS2008 were overzealous and would break otherwise good code with there excessive debug checking. I remember having to add extra overloads to function objects just to satisfy the debug checking.

Has this issue been raised on Microsoft's connect as a potential bug? Hopefully they'll fix it for the next VS version.

On the other hand, even if it may be technically incorrect, I would still like it to work. Right now it's practically unusable without this "fix". Perhaps it can be put in as a conditional compile workaround just for VS2010 (or at least VS2010 debug).

Ronald Garcia — Thu, 23 Feb 2012 14:38:27 GMT

This issue has not been raised with Microsoft. It would be great if someone could contact the proper authorities on this matter. Also, I'm quite busy right now, and 1.49 is all but out the door, so a fix will likely have to wait for 1.50. If you could propose a patch against the current release branch based on the code in the trunk, that would be helpful.

anonymous — Tue, 20 Mar 2012 06:38:25 GMT

Additional comments regarding this issue from the mailing list:

http://lists.boost.org/Archives/boost/2012/01/189715.php

baertman@… — Wed, 11 Apr 2012 10:54:45 GMT

So what's the best workaround around this issue at the moment?

attachment set

Bill Buklis — Wed, 11 Apr 2012 19:40:02 GMT

attachment → base.hpp

MSVC version check for output_iterator_tag (merge with trunk version)

Bill Buklis — Wed, 11 Apr 2012 19:47:55 GMT

I'm using the updated version from the trunk. This works fine. I attached a slightly modified version of the base.hpp file. You will need the rest of the files from trunk in addition to this file. The only modification is to check for the MSVC version. This way if you're not using MSVC it should act as normal.

This is the updated section:

struct mutable_iterator_tag
 : boost::random_access_traversal_tag, std::input_iterator_tag
{
#if BOOST_WORKAROUND(BOOST_MSVC, <= 1700)
	operator std::output_iterator_tag() const
	{
		return std::output_iterator_tag();
	}
#endif
};

The only thing I added was the BOOST_WORKAROUND if check.

Note: I made the output_iterator_tag modification/addition valid for all Visual Studio versions up to and including the next version (VC11) as judging from the mailing list thread mentioned previously this bug may still be in there. I have not yet tried VC11, so I do not know if they have fixed it. Regardless, it shouldn't hurt anything.

Thoughts? Comments?

Bill Buklis — Wed, 11 Apr 2012 21:01:22 GMT

I also updated iterator.hpp to fix a problem with iterator derefencing. See ticket:6554. You may wish to get this file as well. Hopefully both of these patches can be added to the general release.

anonymous — Thu, 12 Apr 2012 08:31:27 GMT

I'm still getting

error C2665: 'std::_Copy_impl' : none of the 2 overloads could convert all the argument types

with the trunk version.

Bill Buklis — Thu, 12 Apr 2012 17:33:47 GMT

First let's make sure that you have the right version of the multi_array headers (located in the boost/multi_array folder). There are only three files that are different from the regular release:

base.hpp
concept_checks.hpp
iterator.hpp

In base.hpp:

On line 84 where it defines array_iterator you should see an additional template parameter for IteratorCategory that is not in the general release. There are other differences as well, but this is a tell-tale sign.

In concept_checks.hpp:

At line 134 there is an additional line in the MutableMultiArrayConcept struct definition that is not in the general release. This is the only difference.

      function_requires< boost::OutputIterator<iterator,value_type> >();

In iterator.hpp:

Near the beginning just after the definition of operator_arrow_proxy is a declaration for array_iterator. Just like in base.hpp there is an additional template parameter for IteratorCategory that is not in the general release.

Without these updated files simply calling resize will generate the _Copy_impl error. With these files, I do not get the error.

anonymous — Fri, 13 Apr 2012 06:28:02 GMT

worked for me, thx

status changed; resolution set

Ronald Garcia — Thu, 17 May 2012 14:52:53 GMT

status new → closed
resolution → fixed

(In [78496]) Pushing fixes over from the trunk to the release branch. Fixes #5664 Fixes #4874

Ronald Garcia — Thu, 17 May 2012 15:00:41 GMT

I updated the release and trunk branches with a fix targeted only to VS 2010. However, the documentation for MultiArray now notes that a MultiArray iterator must model output_iterator. However, this does not require an iterator to have output_iterator_tag. The standard is somewhat ambiguous on this point because the iterator categories muddle traversal and reference behavior in unfortunate ways.

rakesh — Mon, 04 Jun 2012 08:28:08 GMT

Hi Garcia,

Can we expect the fix with the 1.50 release ?

Thanks

Ronald Garcia — Tue, 05 Jun 2012 18:02:46 GMT

Replying to rakesh:

Hi Garcia,

Can we expect the fix with the 1.50 release ?

Thanks

Hi Rakesh,

The answer is a definite Yes! Thanks to everyone for their contributions to this ticket, which ultimately led to the fix.

Bill Buklis — Tue, 05 Jun 2012 19:20:17 GMT

That's excellent news.

Any possibility of also getting ticket:6554 into 1.50? I attached a proposed fix for this one.

I noticed that ticket:6811 has a proposed fix as well. I haven't personally come across this issue, but it's still interesting.

anonymous — Tue, 05 Jun 2012 22:56:41 GMT

Hello Bill,

This is probably not the appropriate place to talk about other tickets, but the answer is no: 1.50 is currently in Beta, so only show-stopper fixes are going to be incorporated.

status changed; resolution deleted

a.elovikov@… — Thu, 06 Sep 2012 10:28:27 GMT

status closed → reopened
resolution fixed

Hi all. I see the same error when using Intel Compiler because it uses the same headers as MSVC does.

$ icl.exe /nologo -Fo"libs/multi_array/test/../../../bin.v2/libs/multi_array/test/assert.test/intel-win/debug/threading-multi/assert.obj" -TP /Z7 /Od /Ob0 /W3 /GR /MDd /Zc:forScope /Zc:wchar_t /Qwn5 /Qwd985 -Qoption,c,--arg_dep_lookup -D_SECURE_SCL=0 -Qoption,cpp,--arg_dep_lookup -Qpchi- /wd4675 /EHs -c -DBOOST_ALL_NO_LIB=1 -DBOOST_TEST_NO_AUTO_LINK=1 "-Ilibs/multi_array/test/../../.." "libs/multi_array/test/assert.cpp"
assert.cpp
libs/multi_array/test/../../../boost/test/utils/basic_cstring/basic_cstring.hpp(60): warning #66: enumeration value is out of "int" range
      enum npos_type { npos = static_cast<size_type>(-1) };
                              ^
c:/Program files (x86)/Microsoft Visual Studio 10.0/VC/include/xutility(2215): error: no instance of overloaded function "std::_Copy_impl" matches the argument list
            argument types are: (boost::detail::multi_array::array_iterator<int, const int *, boost::mpl::size_t<2Ui64>, boost::detail::multi_array::const_sub_array<int, 1Ui64, const int *>, boost::random_access_traversal_tag>, boost::detail::multi_array::array_iterator<int, const int *, boost::mpl::size_t<2Ui64>, boost::detail::multi_array::const_sub_array<int, 1Ui64, const
int *>, boost::random_access_traversal_tag>, boost::detail::multi_array::array_iterator<int, int *,
                      boost::mpl::size_t<2Ui64>, boost::detail::multi_array::sub_array<int, 1Ui64>, boost::random_access_traversal_tag>, boost::detail::iterator_category_with_traversal<std::input_iterator_tag, boost::random_access_traversal_tag>, boost::detail::iterator_category_with_traversal<std::input_iterator_tag, boost::random_access_traversal_tag>)
        return (_Copy_impl(_First, _Last,
                ^
          detected during:
            instantiation of "_OutIt std::_Copy_impl(_InIt, _InIt, _OutIt, std::tr1::false_type) [with _InIt=boost::detail::multi_array::array_iterator<int, const int *, boost::mpl::size_t<2Ui64>, boost::detail::multi_array::const_sub_array<int, 1Ui64, const int *>, boost::random_access_traversal_tag>, _OutIt=boost::detail::multi_array::array_iterator<int, int *, boost::mpl::size_t<2Ui64>, boost::detail::multi_array::sub_array<int, 1Ui64>, boost::random_access_traversal_tag>]" at line 2227
            instantiation of "_OutIt std::copy(_InIt, _InIt, _OutIt) [with _InIt=boost::detail::multi_array::array_iterator<int, const int *, boost::mpl::size_t<2Ui64>, boost::detail::multi_array::const_sub_array<int, 1Ui64, const int *>, boost::random_access_traversal_tag>, _OutIt=boost::detail::multi_array::array_iterator<int, int *, boost::mpl::size_t<2Ui64>, boost::detail::multi_array::sub_array<int, 1Ui64>, boost::random_access_traversal_tag>]" at line 490 of
                      "libs/multi_array/test/../../../boost/multi_array/multi_array_ref.hpp"
            instantiation of "boost::multi_array_ref<T, NumDims> &boost::multi_array_ref<T, NumDims>::operator=(const ConstMultiArray &) [with T=int, NumDims=2Ui64, ConstMultiArray=boost::multi_array<int, 2Ui64, std::allocator<int>>]" at line 377 of "libs/multi_array/test/../../../boost/multi_array.hpp"
            instantiation of "boost::multi_array<T, NumDims, Allocator> &boost::multi_array<T, NumDims, Allocator>::operator=(const boost::multi_array<T, NumDims, Allocator> &) [with T=int, NumDims=2Ui64, Allocator=std::allocator<int>]" at line 52 of "libs/multi_array/test/assert.cpp"
compilation aborted for libs/multi_array/test/assert.cpp (code 2)

But BOOST_MSVC is defined only in boost/config/compiler/visualc.hpp and left undefined in case of icl:

$ icl.exe /nologo -Fo"libs/multi_array/test/../../../bin.v2/libs/multi_array/test/assert.test/intel-win/debug/threading-multi/assert.obj" -TP /Z7 /Od /Ob0 /W3 /GR /MDd /Zc:forScope /Zc:wchar_t /Qwn5 /Qwd985 -Qoption,c,--arg_dep_lookup -D_SECURE_SCL=0 -Qoption,cpp,--arg_dep_lookup -Qpchi- /wd4675 /EHs -c -DBOOST_ALL_NO_LIB=1 -DBOOST_TEST_NO_AUTO_LINK=1 "-Ilibs/multi_array/test/../../.." "libs/multi_array/test/assert.cpp" -E -QdD | grep -e "BOOST_MSVC"
assert.cpp
#define BOOST_MSVC6_MEMBER_TEMPLATES
#define BOOST_MSVC_WORKAROUND_GUARD 1
#define BOOST_MSVC_FULL_VER_WORKAROUND_GUARD 1
#define BOOST_MSVC_TYPENAME typename
#define BOOST_MPL_HAS_XXX_TRAIT_NAMED_DEF(trait,name,default_) template< typename T, typename fallback_ = boost::mpl::bool_<default_> > struct trait { struct gcc_3_2_wknd { template< typename U > static boost::mpl::aux::yes_tag test( boost::mpl::aux::type_wrapper<U> const volatile* , boost::mpl::aux::type_wrapper<BOOST_MSVC_TYPENAME U::name>* = 0 ); static boost::mpl::aux::no_tag test(...); }; typedef boost::mpl::aux::type_wrapper<T> t_; BOOST_STATIC_CONSTANT(bool, value = sizeof(gcc_3_2_wknd::test(static_cast<t_*>(0))) == sizeof(boost::mpl::aux::yes_tag) ); typedef boost::mpl::bool_<value> type; };

I suppose that the correct define to check is _MSC_VER which is defined when using Intel Compiler. The following seems to fix it for me:

$ diff -Naur boost/multi_array/base.hpp{.orig,}
--- boost/multi_array/base.hpp.orig     2012-09-06 17:18:17.497270700 +0700
+++ boost/multi_array/base.hpp  2012-09-06 17:21:22.735770700 +0700
@@ -256,7 +256,7 @@
 // MSVC 2010 is broken in debug mode: it requires
 // that an Output Iterator have output_iterator_tag in its iterator_category if
 // that iterator is not bidirectional_iterator or random_access_iterator.
-#if BOOST_WORKAROUND(BOOST_MSVC, >= 1600)
+#if BOOST_WORKAROUND(_MSC_VER, >= 1600)
 struct mutable_iterator_tag
  : boost::random_access_traversal_tag, std::input_iterator_tag
 {
@@ -314,7 +314,7 @@
   //
   // iterator support
   //
-#if BOOST_WORKAROUND(BOOST_MSVC, >= 1600)
+#if BOOST_WORKAROUND(_MSC_VER, >= 1600)
   // Deal with VC 2010 output_iterator_tag requirement
   typedef array_iterator<T,T*,mpl::size_t<NumDims>,reference,
                          mutable_iterator_tag> iterator;

herohuyongtao@… — Mon, 28 Jan 2013 07:07:46 GMT

But this problem still exists in VS2012 Debug x64 mode even with these updates. Please help to fix. Thanks.

Replying to bill_buklis:

First let's make sure that you have the right version of the multi_array headers (located in the boost/multi_array folder). There are only three files that are different from the regular release:

base.hpp
concept_checks.hpp
iterator.hpp
In base.hpp:

On line 84 where it defines array_iterator you should see an additional template parameter for IteratorCategory that is not in the general release. There are other differences as well, but this is a tell-tale sign.

In concept_checks.hpp:

At line 134 there is an additional line in the MutableMultiArrayConcept struct definition that is not in the general release. This is the only difference.
      function_requires< boost::OutputIterator<iterator,value_type> >();
In iterator.hpp:

Near the beginning just after the definition of operator_arrow_proxy is a declaration for array_iterator. Just like in base.hpp there is an additional template parameter for IteratorCategory that is not in the general release.

Without these updated files simply calling resize will generate the _Copy_impl error. With these files, I do not get the error.

cc changed

nilsgladitz@… — Thu, 13 Nov 2014 11:33:14 GMT

cc nilsgladitz@… added

anonymous — Tue, 02 Dec 2014 11:27:24 GMT

My fix for Intel compiler (x64 debug mode) - to pick up the workaround in the BOOST implementation:

#define BOOST_MSVC _MSC_VER

anonymous — Wed, 28 Oct 2015 11:11:37 GMT

I'm getting this same issue on MSVC12 with boost 1.58 on 64 Bit debug. It's causing a bit of a headache. Is this bug still being looked into?

keywords, version changed

anonymous — Wed, 28 Oct 2015 11:14:36 GMT

keywords 2013 added
version Boost 1.47.0 → Boost 1.58.0

ikku100 — Thu, 11 Aug 2016 10:00:10 GMT

This problem still exists for Intel compiler 16, boost 1.60, in Debug mode for visual studio, both for 32 and 64 bit.

kristian.bjerre.schmidt@… — Tue, 15 Aug 2017 12:59:46 GMT

I ran into this upgrading an old codebase to use Boost 1.64 and compile on windows in addition to linux.

DISCLAIMER: I don't know what I'm doing. I'm quite new to c++, and I just fiddled around until something worked.

Based on the answer from istanbul.tr, I made the following modifications: In multi_array/iterator.hpp line 69, I replaced boost::random_access_traversal_tag with std::random_access_iterator_tag

In multi_array/base.hpp line 227, I added std::random_access_iterator_tag to the base classes of mutable_iterator_tag. This gives me a bunch of warnings that say that mutable_iterator_tag's base class std::input_iterator_tag is already a base class of std::random_access_iterator_tag. But now it works!

Like I mentioned, I have no idea what I'm doing or why this exact configuration works. Any comments are appreciated, and hopefully this gives someone smarter than me a hint as to what needs to be done to fix this.