Boost C++ Libraries: Ticket #8001: stl_input_iterator generators garbage collection problems https://svn.boost.org/trac10/ticket/8001 <p> I am running into some unexpected errors when using stl_input_iterator. I have a fairly simple converter registered to convert an iterable list of values (custom enum) to a unordered_set. The code looks like this: </p> <div class="wiki-code"><div class="code"><pre><span class="k">struct</span> <span class="n">advicetype_set_from_python</span> <span class="p">{</span> <span class="n">advicetype_set_from_python</span><span class="p">()</span> <span class="p">{</span> <span class="n">converter</span><span class="o">::</span><span class="n">registry</span><span class="o">::</span><span class="n">push_back</span><span class="p">(</span><span class="o">&amp;</span><span class="n">convertible</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">construct</span><span class="p">,</span> <span class="n">type_id</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="n">AdviceType</span><span class="o">&gt;&gt;</span><span class="p">());</span> <span class="p">}</span> <span class="k">static</span> <span class="kt">void</span><span class="o">*</span> <span class="n">convertible</span><span class="p">(</span><span class="n">PyObject</span> <span class="o">*</span><span class="n">obj</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="n">obj</span><span class="o">==</span><span class="n">Py_None</span><span class="p">)</span> <span class="k">return</span> <span class="n">obj</span><span class="p">;</span> <span class="n">PyObject</span><span class="o">*</span> <span class="n">iter</span> <span class="o">=</span> <span class="n">PyObject_GetIter</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">iter</span><span class="o">==</span><span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span> <span class="n">throw_error_already_set</span><span class="p">();</span> <span class="k">return</span> <span class="mi">0</span><span class="p">;</span> <span class="p">}</span> <span class="n">Py_DECREF</span><span class="p">(</span><span class="n">iter</span><span class="p">);</span> <span class="k">return</span> <span class="n">obj</span><span class="p">;</span> <span class="p">}</span> <span class="k">static</span> <span class="kt">void</span> <span class="n">construct</span><span class="p">(</span><span class="n">PyObject</span><span class="o">*</span> <span class="n">obj</span><span class="p">,</span> <span class="n">converter</span><span class="o">::</span><span class="n">rvalue_from_python_stage1_data</span> <span class="o">*</span><span class="n">data</span><span class="p">)</span> <span class="p">{</span> <span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="n">AdviceType</span><span class="o">&gt;</span> <span class="n">result</span><span class="p">;</span> <span class="k">if</span> <span class="p">(</span><span class="n">obj</span><span class="o">!=</span><span class="n">Py_None</span><span class="p">)</span> <span class="p">{</span> <span class="n">object</span> <span class="n">o</span> <span class="o">=</span> <span class="n">object</span><span class="p">(</span><span class="n">handle</span><span class="o">&lt;&gt;</span><span class="p">(</span><span class="n">obj</span><span class="p">));</span> <span class="n">stl_input_iterator</span><span class="o">&lt;</span><span class="n">AdviceType</span><span class="o">&gt;</span> <span class="n">begin</span><span class="p">(</span><span class="n">o</span><span class="p">),</span> <span class="n">end</span><span class="p">;</span> <span class="n">result</span><span class="p">.</span><span class="n">insert</span><span class="p">(</span><span class="n">begin</span><span class="p">,</span> <span class="n">end</span><span class="p">);</span> <span class="p">}</span> <span class="kt">void</span> <span class="o">*</span><span class="n">storage</span> <span class="o">=</span> <span class="p">((</span><span class="n">converter</span><span class="o">::</span><span class="n">rvalue_from_python_storage</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="n">AdviceType</span><span class="o">&gt;&gt;*</span><span class="p">)</span><span class="n">data</span><span class="p">)</span><span class="o">-&gt;</span><span class="n">storage</span><span class="p">.</span><span class="n">bytes</span><span class="p">;</span> <span class="k">new</span> <span class="p">(</span><span class="n">storage</span><span class="p">)</span> <span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="n">AdviceType</span><span class="o">&gt;</span><span class="p">(</span><span class="n">result</span><span class="p">);</span> <span class="n">data</span><span class="o">-&gt;</span><span class="n">convertible</span> <span class="o">=</span> <span class="n">storage</span><span class="p">;</span> <span class="p">}</span> <span class="p">};</span> </pre></div></div><p> When using this code we were seeing segfaults due to what appears to be invalid reference counters. Since I suspected a problem in this convertor I wrote a simple test which triggers the converter a number of times: </p> <div class="wiki-code"><div class="code"><pre><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5000</span><span class="p">):</span> <span class="n">my_func</span><span class="p">({</span><span class="n">AdviceType</span><span class="o">.</span><span class="n">one</span><span class="p">,</span> <span class="n">AdviceType</span><span class="o">.</span><span class="n">two</span><span class="p">})</span> </pre></div></div><p> Which had an interesting result: the tests immediately aborted with a "Fatal Python error: GC object already tracked" error. Commenting out the contents of the if-statement in construct() made the problem go away, which makes we think the bug is in the stl_input_iterator used there. To test this I rewrote my code to use a standard !PyObject_GetIter / !PyIter_Next loop and that indeed made all problems go away. </p> en-us Boost C++ Libraries /htdocs/site/boost.png https://svn.boost.org/trac10/ticket/8001 Trac 1.4.3 Ralf W. Grosse-Kunstleve Thu, 07 Feb 2013 17:34:30 GMT status changed; resolution set https://svn.boost.org/trac10/ticket/8001#comment:1 https://svn.boost.org/trac10/ticket/8001#comment:1 <ul> <li><strong>status</strong> <span class="trac-field-old">new</span> → <span class="trac-field-new">closed</span> </li> <li><strong>resolution</strong> → <span class="trac-field-new">invalid</span> </li> </ul> <p> It is highly unlikely that the problem is in Boost.Python. Please use valgrind or some similar tool to debug the problem, which is most likely in your code. </p> Ticket