2

In Python, it's more memory-efficient to use xrange() instead of range when iterating.

The trouble I'm having is that I want to iterate over a large list -- such that I need to use xrange() and after that I want to check an arbitrary element.

With range(), it's easy: x = range(...) + [arbitrary element].

But with xrange(), there doesn't seem to be a cleaner solution than this:

for i in xrange(...):
    if foo(i):
        ...
if foo(arbitrary element):
        ...

Any suggestions for cleaner solutions? Is there a way to "append" an arbitrary element to a generator?

Newb
  • 2,810
  • 3
  • 21
  • 35
  • 1
    Can you clarify what kind of iteration you're doing? If you're iterating over a list itself using `xrange(len(my_list))` you shouldn't be using `xrange` at all, and can do this using `for i, elem in enumerate(my_list)` then check the value of i. – Two-Bit Alchemist Sep 01 '16 at 18:22
  • @Two-BitAlchemist I am certainly not iterating over `len(my_list)`. I don't know why you would assume that. Without loss of generality, let's assume that I am iterating over a range sufficiently large to warrant using `xrange()` instead of `range()`, e.g. `xrange(99999999)`. – Newb Sep 01 '16 at 18:31
  • 1
    "I don't know why you would assume that." `range(len(x))` is a very common iteration anti-pattern in Python and appears on StackOverflow questions hundreds of times a day. Just wanted to clarify. – Two-Bit Alchemist Sep 01 '16 at 19:23
  • @Two-BitAlchemist fair enough. Sorry for the aggressive response. – Newb Sep 01 '16 at 20:02

2 Answers2

9

itertools.chain lets you make a combined iterator from multiple iterables without concatenating them (so no expensive temporaries):

from itertools import chain

# Must wrap arbitrary element in one-element tuple (or list)
for i in chain(xrange(...), (arbitrary_element,)):
    if foo(i):
        ...
ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
7

I would recommend keeping the arbitrary_element check out of the loop, but if you want to make it part of the loop, you can use itertools.chain:

for i in itertools.chain(xrange(...), [arbitrary_element]):
    ...
user2357112
  • 260,549
  • 28
  • 431
  • 505
  • Damn I was late :D This is the true answer. – Rockybilly Sep 01 '16 at 18:22
  • 1
    I disagree with the recommendation -- duplicating the contents of the loop is surely worse than chaining the ranges together. –  Sep 01 '16 at 18:23
  • @Hurkyl: You don't have to duplicate the loop body if you make it a function. – user2357112 Sep 01 '16 at 18:24
  • @user2357112: I don't disagree, but that's not what you (appear to have) recommended. (also, factoring the body out would obfuscate the code if it's sufficiently simple) –  Sep 01 '16 at 18:25
  • Putting it in the loop is plausible in this example given that `chain` is such a small operation with no extra memory used. – Rockybilly Sep 01 '16 at 18:26