Splitting a string into an iterator

If you don't need to consume the whole string, that's because you are looking for something specific, right? Then just look for that, with re or .find() instead of splitting. That way you can find the part of the string you are interested in, and split that.

Not directly splitting strings as such, but the re module has re.finditer() (and corresponding finditer() method on any compiled regular expression).

@Zero asked for an example:

>>> import re
>>> s = "The quick    brown\nfox"
>>> for m in re.finditer('\S+', s):
...     print(m.span(), m.group(0))
... 
(0, 3) The
(4, 9) quick
(13, 18) brown
(19, 22) fox

Like s.Lott, I don't quite know what you want. Here is code that may help:

s = "This is a string."
for character in s:
    print character
for word in s.split(' '):
    print word

There are also s.index() and s.find() for finding the next character.

---

Later: Okay, something like this.

>>> def tokenizer(s, c):
...     i = 0
...     while True:
...         try:
...             j = s.index(c, i)
...         except ValueError:
...             yield s[i:]
...             return
...         yield s[i:j]
...         i = j + 1
... 
>>> for w in tokenizer(s, ' '):
...     print w
... 
This
is
a
string.