How can I create an indexable map(), or a decorated list()?

Question

I have a long list of file paths, like:

images = ['path/to/1.png', 'path/to/2.png']

I know that I can wrap this list in a map iterator, which provides sequential access to items in list mapped through a function, like:

image_map = map(cv2.imread, images)

Then I can lazily load those image files as I iterate over the list:

next(image_map)
=> pixels

But I want random access to the original list, mapped through my map function:

image_map[400]
=> pixels

I don't want to convert it to a list, because that would load all of my images into memory, and they don't fit into memory:

# Bad:
list(image_map)[400]

Another way to think about this might be a decorator on list.__getitem__.

I know that I can sub-class a list, but I'm really hoping for a cleaner way of doing this.

This is what subclassing is _for_ -- you need to modify the behavior of `list` slightly by adding a line to `__getitem__` -- I think you will struggle to find a better solution than inheritance — MoxieBall, Jul 24 '18 at 21:40
based on [this answer](https://stackoverflow.com/a/37418100/8061009) map does not actually support non-sequential-laziness, so you would need to bypass `map` as well. — Will, Jul 24 '18 at 21:44
FYI, “indexable map” is a misleading term here, since, as a noun, a *map* usually refers to a specific data structure, which corresponds to Python’s `dict`. `map` in Python is a *function*, and thus refers to a *verb*. If you want to use it as a noun, “fmap” (or, better, “functor”) would be the correct term, though it’s not typically used (nor widely understood) in Python. — Konrad Rudolph, Jul 24 '18 at 21:49
Thanks for the fast help @MoxieBall @Juergen! SO members are still the best. — colllin, Jul 24 '18 at 21:57

Juergen · Accepted Answer · 2018-07-24T21:53:48.347

Why don't you just create an accessor class?

class ImageList(object):
    def __init__(self, images):
        self.images = images

    def get_image(self, image_num):
        return cv2.imread(self.images[image_num])

You of course also can buffer read images.

You can also provide a __getitem__ method to have list-like access:

def __getitem__(self, key):
    return cv2.imread(self.images[key])

Usage:

images = ['path/to/1.png', 'path/to/2.png']

image_list = ImageList(images)

image = image_list.get_image(400)    # the same as 
image = image_list[400]              # this

By the way: Of course you could subclass list but in the Python community more explicite is favored. And it is simply more clear here to have a separate class instead of sub-classing list. It is also not the best style to over-use inheritance.

score 3 · Answer 2 · answered Jul 24 '18 at 21:50

One issue with using map if you can't convert it the result to a list is that you can only iterate through it once. If you can't fit the whole thing into memory at once and you need random access, then you're going to want to keep around as little information as possible until you really need it -- which sounds like when __getitem__ is called. Thus, if you're intent on being able to write image_map[n] to get the pixel data (instead of just calling cv2.imread(image_map[n]), you will have to create a list subclass whose __getitem__ calls cv2.imread.

e.g.

class cv2_list(list):
    def __getitem__(self, item):
        return cv2.imread(super().__getitem__(item))

How can I create an indexable map(), or a decorated list()?

2 Answers2