transform_width may cause buffer overruns

[anonymous]

When using boost::archive::iterators::transform_width iterator for e.g. base64 encoding, and number of input bits does not fit into an integer number of output bits, the iterator dereferences an element beyond the last available.

Consider this piece of code:

typedef std::vector<unsigned char> bytes;

bytes b;
b.push_back(1);

typedef
    base64_from_binary<
        transform_width<
             bytes::const_iterator,
             6,
             sizeof(bytes::value_type) * 8, char> > base64;

std::stringstream o;
std::copy(base64(b.begin()), base64(b.end()), std::ostream_iterator<char>(o));

Provided vector::const_iterator implementation does checks against dereferencing an invalid iterator (e.g. vc8 stl), the above code will assert, because of an attempt to read the second (non-existant) element from the vector

[Robert Ramey]

The only way I could figure out to avoid this is not to do it.

That is when I have this situation I can't use the end() iterator. The serialization library avoids doing this so it never has a problem. That is it counts the iterations so it never comes up. I tried but couldn't find a better way.

Robet Ramey

[kostic]

Yes, I realise this. However, there's a simple solution, albeit it changes the interface. The solution is for transform_width to take a pair of iterators (begin and end), not one, so that it knows exactly when to stop and not go over the end.

I have seen a lot of people using transform_width for base64 encoding with std::copy, which with current implementation is what one should not do. Well, it should either not compile at all or should work as expected (in the most intuitive way)

Thanks for a quick response by the way

[Robert Ramey]

lol the quick response is just luck. I'm now trying to address all my trac tickets.

I crafted the interface "dataflow iterators" so that I could compose the iterators at compile time in an "natural" way. I was and am exceedingly pleased with the result - except for the transform with iterator. I couldn't figure out a way - I don't think there is one, to determine how many characters there is in the output, so I had to count the input ones (or is it vice-versa). There is an alternative in range iterators, but I didn't like the syntax and the requirement that one specifies the end "too" soon.

I would be very suprised to find anyone using "dataflow iterators" at all. I love this concept and would like to see it used more for tuples - (hmm maybe that's what fusion is/does), but it doesn't seem that anyone else warms up to it. oh well.

I'll look into making std::copy fail at compile time. If you're interested, feeel free to send me a suggested patch. In the mean time, please don't reopen this ticket. I'm trying to get them cleared and I can't shovel against the tide.

Robert Ramey

Robert Ramey

[anonymous]

Replying to ramey:

I'll look into making std::copy fail at compile time. If you're interested, feeel free to send me a suggested patch. In the mean time, please don't reopen this ticket. I'm trying to get them cleared and I can't shovel against the tide.

Well, you could put a static assert in "equal" method to stop it being ever used with this iterator. This makes sense as comparing these iterators is inherently unsafe (as it allows comparing with end)

[kostic]

Replying to anonymous:

Replying to ramey:

I'll look into making std::copy fail at compile time. If you're interested, feeel free to send me a suggested patch. In the mean time, please don't reopen this ticket. I'm trying to get them cleared and I can't shovel against the tide.

Well, you could put a static assert in "equal" method to stop it being ever used with this iterator. This makes sense as comparing these iterators is inherently unsafe (as it allows comparing with end)

Also, as to people using dataflow iterators, this page promotes the exact usage you say should be avoided: http://www.boost.org/libs/serialization/doc/dataflow.html

Konstantin Begun

[smfr at smfr.org]

Hi Robert In this mail: <​http://lists.boost.org/boost-users/2008/09/40207.php> you said you'd address this problem. Are we likely to see a fix in Boost 1.37? It's still blocking me from relying on XML serialization.