Boost C++ Libraries: Ticket #9578: Adapters (map_keys, map_values) cause undefined behavior (segv, etc) when applied to R-Values, especially in the context of BOOST_FOREACH https://svn.boost.org/trac10/ticket/9578 <p> When applied to r-values (for example, maps returned by value from a function), adapters do not make a copy of the r-value but just capture a reference to it. This makes the returned range_iterator dangerous, if it is used after the r-value (the temporary) is destroyed. </p> <p> In particular, the following use causes undefined behavior on all compilers I'd tested on: g++ (4.1.2, 4.4.3, 4.6.3) and Visual Studio (15 [VC9], 16 [VC10]). </p> <div class="wiki-code"><div class="code"><pre><span class="n">BOOST_FOREACH</span><span class="p">(</span><span class="k">const</span> <span class="n">string</span> <span class="o">&amp;</span><span class="n">s</span><span class="p">,</span> <span class="n">return_map_by_value</span><span class="p">()</span> <span class="o">|</span> <span class="n">map_keys</span><span class="p">)</span> <span class="p">...</span> </pre></div></div><p> Note that the following works fine (since BOOST_FOREACH takes care of differentiating between l-values and r-values and does the right thing to r-values): </p> <div class="wiki-code"><div class="code"><pre><span class="n">BOOST_FOREACH</span><span class="p">(</span><span class="k">const</span> <span class="n">MapType</span><span class="o">::</span><span class="n">value_type</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="n">return_map_by_value</span><span class="p">())</span> <span class="p">...</span> </pre></div></div><p> It seems that boost::range::adapters need to do a similar thing to what BOOST_FOREACH is does to identify r-values and make a copy (which the compiler is likely to optimize away). </p> <p> Attached is a test sample and the resulting behavior. The output is from code compiled with g++ 4.6.3 with no optimizations. </p> <pre class="wiki">$ g++ -I ../include/ container_test.cpp $ ./a.out Test #1 (local variable) Generating map key = '12345' key = 'QWERT' key = 'igor!' Test #2 (value_type) Generating map key = '12345' key = 'QWERT' key = 'igor!' Test #3 (temp ref) Generating map key = '12345' key = 'QWERT' key = 'igor!' Test #4 (range_iterator) Generating map key = '12345' key = 'QWERT' key = 'igor!' Test #5 (bug) Generating map key = '12345APãr`ãr¨ârãr1 ãrÿÿÿÿABCDE1ãrÿÿÿÿQWERTAâr½™ÿÀâr ârHãr¸ãr1àárÿÿÿÿqwertAärH½™ÿPärärHãr¸ãrAårÐãârxârAÀãrÐãr¨ârãrAÀärX™ÿårÐärHãr¸ãrAäârxârA€ärär¨ârãrA€årؽ™ÿÐårårHãr¸ãrA°ârPåârxârA@årPår¨ârãr </pre> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/9578 Trac 1.4.3 Igor Lubashev <ilubashe@…> Wed, 15 Jan 2014 20:37:51 GMT attachment set https://svn.boost.org/trac10/ticket/9578 https://svn.boost.org/trac10/ticket/9578 <ul> <li><strong>attachment</strong> → <span class="trac-field-new">container_test.cpp</span> </li> </ul> <p> Test to demonstrate undefined behavior of boost::range::adapter::map_keys when applied to an r-value and used with BOOST_FOREACH </p> Ticket Igor Lubashev <ilubashe@…> Thu, 30 Jan 2014 16:59:13 GMT cc set https://svn.boost.org/trac10/ticket/9578#comment:1 https://svn.boost.org/trac10/ticket/9578#comment:1 <ul> <li><strong>cc</strong> <span class="trac-author">ilubashe@…</span> added </li> </ul> Ticket Neil Groves Fri, 21 Feb 2014 20:33:14 GMT status changed https://svn.boost.org/trac10/ticket/9578#comment:2 https://svn.boost.org/trac10/ticket/9578#comment:2 <ul> <li><strong>status</strong> <span class="trac-field-old">new</span> → <span class="trac-field-new">assigned</span> </li> </ul> Ticket Neil Groves Mon, 03 Mar 2014 01:47:54 GMT status changed; resolution set https://svn.boost.org/trac10/ticket/9578#comment:3 https://svn.boost.org/trac10/ticket/9578#comment:3 <ul> <li><strong>status</strong> <span class="trac-field-old">assigned</span> → <span class="trac-field-new">closed</span> </li> <li><strong>resolution</strong> → <span class="trac-field-new">wontfix</span> </li> </ul> <p> See: <a class="ext-link" href="https://svn.boost.org/trac/boost/ticket/5475"><span class="icon">​</span>https://svn.boost.org/trac/boost/ticket/5475</a> </p> <p> BOOST_FOREACH is considered to be deprecated and in maintenance mode. There are no simple solutions for C++03 that don't cause unacceptable performance overhead and enormous complexity. </p> Ticket ilubashe@… Mon, 03 Mar 2014 04:49:48 GMT status, severity changed; resolution deleted https://svn.boost.org/trac10/ticket/9578#comment:4 https://svn.boost.org/trac10/ticket/9578#comment:4 <ul> <li><strong>status</strong> <span class="trac-field-old">closed</span> → <span class="trac-field-new">reopened</span> </li> <li><strong>resolution</strong> <span class="trac-field-deleted">wontfix</span> </li> <li><strong>severity</strong> <span class="trac-field-old">Showstopper</span> → <span class="trac-field-new">Problem</span> </li> </ul> <p> This bug has little to do with BOOST_FOREACH. This is a boost::range problems. You'll get the same problem with C++11 "for" loops. </p> <p> for(const string &amp;s : return_map_by_value() | map_keys) </p> Ticket Neil Groves Mon, 03 Mar 2014 14:33:05 GMT <link>https://svn.boost.org/trac10/ticket/9578#comment:5 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9578#comment:5</guid> <description> <p> Yes you would, but I have no solution for this. It seems to me to be a lifetime issue that is not fixable without enormous detrimental impact on other code. I closed the ticket not due to unwillingness to alter the code, but due to having no good solution. If we alter the semantics to require copying this has huge impact on some ranges. If we automatically move this would either require large performance overhead due to bloated adapters and would alter the semantic to have other surprising features. </p> <p> I'll see if you or anyone else have any suggestions for what you actually would like done. AFAICT it simply is a general C++ issue. You have to be careful with lifetimes. Of course if there is a solution I have overlooked that can make things better with little impact on other cases I would jump at it immediately. </p> </description> <category>Ticket</category> </item> <item> <author>ilubashe@…</author> <pubDate>Mon, 03 Mar 2014 14:59:12 GMT</pubDate> <title/> <link>https://svn.boost.org/trac10/ticket/9578#comment:6 </link> <guid isPermaLink="false">https://svn.boost.org/trac10/ticket/9578#comment:6</guid> <description> <p> I called out BOOST_FOREACH in filing the bug, because BOOST_FOREACH has the solution to a very similar problem it had to deal with. </p> <p> I would certainly not propose range adapter to copy values wholesale. The only values that need copying are r-values. In fact, I would be very surprised if there is ever a case when you do NOT want to copy an r-value in a range adapter. The l-values should certainly never be copied. </p> <p> BOOST_FOREACH has a way to tell which values are l-values and which are r-values. </p> <p> There is also a related bug: <a class="assigned ticket" href="https://svn.boost.org/trac10/ticket/7630" title="#7630: Bugs: Range adaptors do not play nicely with range-based for loops (assigned)">#7630</a> </p> </description> <category>Ticket</category> </item> <item> <dc:creator>Neil Groves</dc:creator> <pubDate>Mon, 03 Mar 2014 15:07:55 GMT</pubDate> <title>status changed; resolution set https://svn.boost.org/trac10/ticket/9578#comment:7 https://svn.boost.org/trac10/ticket/9578#comment:7 <ul> <li><strong>status</strong> <span class="trac-field-old">reopened</span> → <span class="trac-field-new">closed</span> </li> <li><strong>resolution</strong> → <span class="trac-field-new">duplicate</span> </li> </ul> <p> I'm marking this as a duplicate of 7630 now since that seems to be the more general issue. I've been worrying about this lifetime issue for sometime, but haven't a good solution. I spent a little time looking at the BOOST_FOREACH implementations and I'll take a look at some of the suggestions made for ticket 7630. It might be that this can now be made to work well with the C++11 support becoming much more mature and the rvalue support that much more reliable. </p> <p> I hope I don't cause any offence by collapsing this into one defect. Please feel free to make comments on the <a class="assigned ticket" href="https://svn.boost.org/trac10/ticket/7630" title="#7630: Bugs: Range adaptors do not play nicely with range-based for loops (assigned)">#7630</a> ticket. </p> <p> Since there is even an example solution I should be able to find a way to make this work. I'll obviously have to take care to ensure that the fix doesn't introduce any unexpected nasty surprises. </p> <p> Thanks for helping. </p> Ticket