How to select a sliding window of elements from a list of lists?

Question

Say I have the following list of lists:

x = [[1,2,3],[4,5,6],[7,8,9,10]]

And I wish to select all 'windows' of size e.g. n=4, staggered by a distance of e.g. d=2:

    [[1,2,3],[4]]                # Starts at position `0`
        [[3],[4,5,6]]            # Starts at position `d`
              [[5,6],[7,8]]      # Starts at position `2d`
                    [[7,8,9,10]] # Starts at position `3d`

I.e. I wish to take intersecting 'slices' where the windows overlap with the sublists.

How would I go about this?

https://numpy.org/doc/stable/reference/generated/numpy.lib.stride_tricks.sliding_window_view.html#numpy.lib.stride_tricks.sliding_window_view — iacob, Mar 24 '21 at 22:38

score 4 · Accepted Answer · edited Feb 14 '21 at 11:44

If you precompute some indices, you can reconstruct any window with a virtual one-liner:

import itertools
import operator

def window(x, start, stop):
    first = indices[start][0]
    last = indices[stop-1][0]
    return [
        [x[i][j] for i, j in g] if k in (first, last) else x[k]
        for k, g in itertools.groupby(
            indices[start:stop],
            key=operator.itemgetter(0))
        ]

def flat_len(x):
    """Return length of flattened list."""
    return sum(len(sublist) for sublist in x)

n=4; d=2
x = [[1,2,3],[4,5,6],[7,8,9,10]]

indices = [(i, j) for i, sublist in enumerate(x) for j in range(len(sublist))]

for i in range(0,flat_len(x)-n+1,d):
    print(window(x,i,i+n,indices))

>>> [[1, 2, 3], [4]]
>>> [[3], [4, 5, 6]]
>>> [[5, 6], [7, 8]]

score 2 · Answer 2 · edited May 14 '19 at 10:57

I would go for nested for loops, although it's not pretty:

x = [[1,2,3],[4,5,6],[7,8,9,10]]

def window(x, n, offset):
    pos = 0
    res = []
    for l in x:
        # Skip `l` if offset is larger than its length 
        if len(l) + pos <= offset:
            pos += len(l)
            continue
        # Stop iterating when window is complete
        elif pos >= n + offset:
            break

        tmp = []
        for el in l:
            #if `el` is in window, append it to `tmp`
            if offset <= pos < n + offset:
                tmp.append(el)
            # Stop iterating when window is complete
            elif pos >= n + offset:
                break
            pos += 1
        res.append(tmp)
    return res

def flat_len(x):
    """Return length of flattened list."""
    return sum(len(sublist) for sublist in x)

n = 4
d = 2

for i in range(0, flat_len(x) - n + 1, d):
    print(window(x, n, i))

Paritosh Singh · Answer 3 · 2019-05-13T11:29:00.120

Another approach can be to actually use the flat list, get the right window, but fix it afterwards. I caved in and used a little numpy at the end, makes the fixing easier.

x = [[1,2,3],[4,5,6],[7,8,9,10]]
from itertools import chain
import numpy as np
n = 4
d = 2
def custom_slider(x, n, d):
    x_shape = [len(l) for l in x]
    x_cumsum_shape = np.cumsum(x_shape) #this will come in handy for fixing slices later
    x_flat = list(chain.from_iterable(x))
    result = []
    for i in range(0, len(x_flat) - n + 1, d):
        #essentially get slice points, using the current index i to start. ignore negative or zero slices
        split_pts = (x_cumsum_shape - i)[x_cumsum_shape - i > 0] 

        #[i: i + n] gives the correct slice. use split points to correctly mimic original arrays
        temp = [list(item) for item in np.split(x_flat[i: i + n], split_pts) if item.size]

        result.append(temp) #could also turn function into generator by yielding instead
    return result

custom_slider(x, n, d)

Output:

[[[1, 2, 3], [4]], [[3], [4, 5, 6]], [[5, 6], [7, 8]], [[7, 8, 9, 10]]]

How to select a sliding window of elements from a list of lists?

3 Answers3