28

I have a list of custom-class objects (sample is below).

Using: list(itertools.chain.from_iterable(myBigList)) I wanted to "merge" all of the stations sublists into one big list. So I thought I need to make my custom class an iterable.

Here is a sample of my custom class. 

class direction(object) :
    def __init__(self, id) :
        self.id = id              
        self.__stations = list()

    def __iter__(self):
        self.__i = 0                #  iterable current item 
        return iter(self.__stations)

    def __next__(self):
        if self.__i<len(self.__stations)-1:
            self.__i += 1         
            return self.__stations[self.__i]
        else:
            raise StopIteration

I implemented __iter__ and __next__ but it doesn't seems to work. They're not even called.

Any idea what could I've done wrong?

Note: Using Python 3.3

rayryeng
  • 102,964
  • 22
  • 184
  • 193
Matthieu Riegler
  • 31,918
  • 20
  • 95
  • 134

3 Answers3

36

__iter__ is what gets called when you try to iterate over a class instance:

>>> class Foo(object):
...     def __iter__(self):
...         return (x for x in range(4))
...
>>> list(Foo())
[0, 1, 2, 3]

__next__ is what gets called on the object which is returned from __iter__ (on python2.x, it's next, not __next__ -- I generally alias them both so that the code will work with either...):

class Bar(object):
    def __init__(self):
        self.idx = 0
        self.data = range(4)
    def __iter__(self):
        return self
    def __next__(self):
        self.idx += 1
        try:
            return self.data[self.idx-1]
        except IndexError:
            self.idx = 0
            raise StopIteration  # Done iterating.
    next = __next__  # python2.x compatibility.

In the comments, it was asked how you would construct and object that could be iterated multiple times. In this case, I'd recommend taking the same approach that Python takes and split the iterator from the data container:

class BarIterator(object):
    def __init__(self, data_sequence):
        self.idx = 0
        self.data = data_sequence
    def __iter__(self):
        return self
    def __next__(self):
        self.idx += 1
        try:
            return self.data[self.idx-1]
        except IndexError:
            self.idx = 0
            raise StopIteration  # Done iterating.


class Bar(object):
    def __init__(self, data_sequence):
        self.data_sequence = data_sequence
    def __iter__(self):
        return BarIterator(self.data_sequence)
martineau
  • 119,623
  • 25
  • 170
  • 301
mgilson
  • 300,191
  • 65
  • 633
  • 696
  • Well I'm really confused, because I was my code around that was wrong. But your answer did make it a bit clearer to me how iterables work ! – Matthieu Riegler Feb 09 '14 at 22:36
  • Why not just return the iterator of `self.data`? While fine for OP's purposes, this is not thread safe. You can call `next()` on the iterator before returning it if you want to skip the first element. – Mad Physicist Feb 08 '17 at 19:24
  • @MadPhysicist -- That's a fair point. It's been 2 years since I answered this, but I _think_ that the reason that I did it this way was to demonstrate to OP how to use the iterator protocol with `__iter__` and `__next__`. – mgilson Feb 08 '17 at 19:29
  • @mgilson. Makes sense. Valuable answer either way. – Mad Physicist Feb 08 '17 at 19:29
  • What should I do to allow multiple loops over this object? Should I reset self.idx to 0 on the call to __iter()__? – uuu777 Oct 15 '20 at 18:01
  • 1
    In that case, I'd take a page out of what the stdlib does for buildin containers -- e.g. `list._iter__` returns a `listiterator` instance. The `listiterator` is still only once iterable. It's really just an index and a pointer back to the original data. In this way, a caller can call `iter(some_list)` as many times as they want and get independent iterators each time. – mgilson Oct 20 '20 at 16:08
  • > What should I do to allow multiple loops over this object? Should I reset self.idx to 0 on the call to iter() From [Iterator Types](https://docs.python.org/3/library/stdtypes.html#iterator-types): > Once an iterator’s __next__() method raises StopIteration, it must continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken. – Mike Kaganski Feb 18 '21 at 15:32
  • @MikeKaganski -- updated the answer to hopefully address your question. – mgilson Feb 24 '21 at 06:13
  • I likely was unclear. I didn't ask a question, I rather answered a question by @zzz777 above (quoting their question, and then quoting an excerpt from the documentation, to show that the repeated iteration using the same iterator is against the standard). – Mike Kaganski Feb 24 '21 at 13:08
7

simply implementing __iter__ should be enough. 

class direction(object) :
    def __init__(self, id) :
        self.id = id              
        self.__stations = list()

    def __iter__(self):
        #return iter(self.__stations[1:]) #uncomment this if you wanted to skip the first element.
        return iter(self.__stations)


a = direction(1)
a._direction__stations= range(5)

b = direction(1)
b._direction__stations = range(10)

import itertools
print list(itertools.chain.from_iterable([a,b]))
print list(itertools.chain.from_iterable([range(5),range(10)]))

output: 

[0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

See here for why it's _direction__stations

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with classname_spam, where classname is the current class name with leading underscore(s) stripped.

M4rtini
  • 13,186
  • 4
  • 35
  • 42
  • You need to use `iter(self.__stations[1:])`, as per OP's code they are skipping the first item. – Ashwini Chaudhary Feb 09 '14 at 22:27
  • I'll at it as an option with uncommenting code. since he did not explicitly say that was what he wanted. I'll assume that more natural implementation of giving all elements. And that his original implementation of `next`had a little bug in it. – M4rtini Feb 09 '14 at 22:34
  • 3
    Or, `x = iter(self.__stations); next(x); return x` if you don't want to make an unnecessary copy ;-) – mgilson Feb 09 '14 at 23:01
3

You can subclass list as well:

class Direction(list):
    def __init__(self, seq=[], id_=None):
        list.__init__(self,seq)
        self.id = id_ if id_ else id(self)

    def __iter__(self):
        it=list.__iter__(self) 
        next(it)                       # skip the first...
        return it  

d=Direction(range(10))
print(d)       # all the data, no iteration
# [0, 1, 2, 3, 4]

print (', '.join(str(e) for e in d))     # 'for e in d' is an iterator
# 1, 2, 3, 4

ie, skips the first.

Works for nested lists as well:

>>> d1=Direction([range(5), range(10,15), range(20,25)])
>>> d1
[range(0, 5), range(10, 15), range(20, 25)]
print(list(itertools.chain.from_iterable(d1)))
[10, 11, 12, 13, 14, 20, 21, 22, 23, 24]
dawg
  • 98,345
  • 23
  • 131
  • 206