Can't use iterator_range as template argument in tokenizer

[roman300178@…]

I tried to use boost tokenizer directly with char* (instead of std::string) In order to do so, I needed convert the char* to iterator_range. I managed to do so by calling "as_literal" method. Afterwards I used iterator range as tokenizer argument and took tokenizer's iterator for token parsing. However I got compiler error that "assign" method on iterator_range object isn't defined. I think this compilation problem could easily fixed by adding to iterator_range new "assign" method:

iterator_range& assign(IteratorT beg, IteratorT end)

{

m_Begin = beg; m_End = end; #ifndef NDEBUG singular = false; #endif return *this;

}

thanks, Roman

[Steven Watanabe]

This would probably be a bad idea, since the semantics of assign for containers are significantly different from what they would be for iterator_range.

[anonymous]

Steven, could you please elaborate on this? semantics of assign for example for string container does have such signature and since I want to use iterator_range as a container for boost::tokenizer I am forced (by tokenizer implementation) to have such assign method thanks, Roman

[Steven Watanabe]

Assign for containers makes a copy of all the elements. Clearly, iterator_range is not going to do this. I think it's a bad idea for iterator_range to try to mimic the mutable interface of containers, when the actual behavior is not consistent. While I appreciate that you want to use iterator_range with tokenizer, I think adding assign to iterator_range is probably the wrong solution. I think it would be better if tokenizer could be changed so that it doesn't rely on assign.

[Roman Bober <roman300178@…>]

I see,and of course my assign method suggestion above doesn't make copy of elements, but rather "repoints" to new location by given iterators. However, I understand your point, that notion of assign for containers is different from iterator_range concept. Solution I think lies somewhere in the middle, since tokenizer takes as template argument Container and iterator_range isn't truly a container, thus there must be some way to interact between. So change to the tokenizer could be: it would take iterator_range as it's template argument as well?

[Neil Groves]

Reassigned since this is I am maintaining Boost.Range

[Neil Groves]

The documented requirements for the Tokenizer argument are greater than those provided by the Range Concept. In particular the Tokenizer algorithm needs to be able to utilize container functions to resize / allocate etc. This is far outside of the scope of the Range Concept by design.

Since using a container appears an easy and optimal solution I am closing this issue.