What is the preferred way to concatenate sequences in Python 3?

Question

Right now, I'm doing:

import functools
import operator

def concatenate(sequences):
    return functools.reduce(operator.add, sequences)

print(concatenate([['spam', 'eggs'], ['ham']]))
# ['spam', 'eggs', 'ham']

Needing to import two separate modules to do this seems clunky.

An alternative could be:

def concatenate(sequences):
    concatenated_sequence = []
    for sequence in sequences:
        concatenated_sequence += sequence
    return concatenated_sequence

However, this is incorrect because you don't know that the sequences are lists.

You could do:

import copy

def concatenate(sequences):
    head, *tail = sequences
    concatenated_sequence = copy.copy(head)
    for sequence in sequences:
        concatenated_sequence += sequence
    return concatenated_sequence

But that seems horribly bug prone -- a direct call to copy? (I know head.copy() works for lists and tuples, but copy isn't part of the sequence ABC, so you can't rely on it... what if you get handed strings?). You have to copy to prevent mutation in case you get handed a MutableSequence. Moreover, this solution forces you to unpack the entire set of sequences first. Trying again:

import copy 

def concatenate(sequences):
    iterable = iter(sequences)
    head = next(iterable)
    concatenated_sequence = copy.copy(head)
    for sequence in iterable:
        concatenated_sequence += sequence
    return concatenated_sequence

But come on... this is python! So... what is the preferred way to do this?

Have you seen `itertools.chain()`? I'm not sure it handles all of your intended use cases, though. — Wooble, Jan 15 '13 at 16:33
Don't be afraid of standard library imports. It is very likely that some other module (even from the standard library) will grab `functools` and/or `operator` anyway. — Oleh Prypin, Jan 15 '13 at 16:45

Martijn Pieters · Accepted Answer · 2019-11-01T00:41:23.967

I'd use itertools.chain.from_iterable() instead:

import itertools

def chained(sequences):
    return itertools.chain.from_iterable(sequences):

or, since you tagged this with python-3.3 you could use the new yield from syntax (look ma, no imports!):

def chained(sequences):
    for seq in sequences:
        yield from seq

which both return iterators (use list() on them if you must materialize the full list). Most of the time you do not need to construct a whole new sequence from concatenated sequences, really, you just want to loop over them to process and/or search for something instead.

Note that for strings, you should use str.join() instead of any of the techniques described either in my answer or your question:

concatenated = ''.join(sequence_of_strings)

Combined, to handle sequences fast and correct, I'd use:

def chained(sequences):
    for seq in sequences:
        yield from seq

def concatenate(sequences):
    sequences = iter(sequences)
    first = next(sequences)
    if hasattr(first, 'join'):
        return first + ''.join(sequences)
    return first + type(first)(chained(sequences))

This works for tuples, lists and strings:

>>> concatenate(['abcd', 'efgh', 'ijkl'])
'abcdefghijkl'
>>> concatenate([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> concatenate([(1, 2, 3), (4, 5, 6), (7, 8, 9)])
(1, 2, 3, 4, 5, 6, 7, 8, 9)

and uses the faster ''.join() for a sequence of strings.

Whoa, that's some nice syntax! `yield from` is new in 3.3 right? — Kos, Jan 15 '13 at 16:36

score 2 · Answer 2 · answered Jul 20 '18 at 00:57

2

what is wrong with:

from itertools import chain
def chain_sequences(*sequences):
  return chain(*sequences)

answered Jul 20 '18 at 00:57

Samantha Atkins

658
4
12

I don't think it is a question, but actually an attempt to answer. More explanation would have made that clearer. – Laurenz Albe Jul 20 '18 at 07:45
It is in the answer section so it seems clear enough it was a proposed answer. – Samantha Atkins Jul 25 '18 at 00:00

Oleh Prypin · Answer 3 · 2013-01-15T16:41:05.460

1

Use itertools.chain.from_iterable.

import itertools

def concatenate(sequences):
    return list(itertools.chain.from_iterable(sequences))

The call to list is needed only if you need an actual new list, so skip it if you just iterate over this new sequence once.

edited Jan 15 '13 at 16:41

answered Jan 15 '13 at 16:34

Oleh Prypin

33,184
10
89
99

`concatenate(['a', 'b'])` will return `['a', 'b']` instead of `'ab'`, so this is not correct for my use case. How would you figure out the sequence type? – ToBeReplaced Jan 15 '13 at 16:40
@ToBeReplaced: string concatenation is best handled with `str.join()` in any case. – Martijn Pieters Jan 15 '13 at 16:42
Why the "from_iterable" piece when chain(*sequences) works? – Samantha Atkins Jul 25 '18 at 00:02

What is the preferred way to concatenate sequences in Python 3?

3 Answers3