10

This wiki.python.org page on algorithmic complexity of some data structures says the following for a collections.deque object:

A deque (double-ended queue) is represented internally as a doubly linked list. (Well, a list of arrays rather than objects, for greater efficiency.) Both ends are accessible, but even looking at the middle is slow, and adding to or removing from the middle is slower still.

Two questions:

1) Is adding to the middle of a deque even possible? I don't see any method to do so in the API.

2) Why would removing (or adding) be slower than lookup in the middle of a deque? It's a doubly-linked list, so add/remove should be a constant time operation once you've found the object you want to add.

Eli Rose
  • 6,788
  • 8
  • 35
  • 55

3 Answers3

6
  1. It's possible to delete items using the remove() method or the del keyword. It's not possible to insert items. (The only possible way to insert that wouldn't show up in the API documentation would be slice assignment, and that's invalid on a deque.)
  2. Because, as the description says, it's actually a doubly-linked list of arrays. So inserting or removing things could require elements to be moved from one array to another. (I haven't looked at the implementation, but I know deque uses a stride technique when looking for elements and I assume the size of the arrays used is the same as the stride length, which is 62.) You'd have to shift a lot of memory around in a regular list when deleting items, too, but at least it's all in one chunk and it can be moved efficiently.
kindall
  • 178,883
  • 35
  • 278
  • 309
  • What do you mean at the end when you say the memory for a `list` is "all one chunk"? Python lists are contiguous arrays *of pointers* not contiguous arrays *of memory*. That is, the pointers are contiguous, but whatever they point to can be wherever, and it's one of the main reasons for fractured memory in Python. Just think of arbitrarily assigning list item 2, say, to some large, pre-allocated object, like `my_list[1] = some_object` .. I'm probably just confused by your phrasing, but surely you're not saying that all of `my_lists`'s "memory" is shuffled around for the large `some_object`? – ely Sep 11 '15 at 19:13
  • No, I just mean that deleting or inserting items in a `list` requires moving only one chunk of memory. It's the pointers that are being moved, true. – kindall Sep 11 '15 at 19:15
  • Gotcha, thanks for clarifying. For anyone else who stumbles on these comments, I am basically referencing the explanation from Jake VanderPlas's article ["Why Python Is Slow"](https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/), specifically in section 3 on the object model. – ely Sep 11 '15 at 19:15
  • Hmm, I see how does using arrays instead of simple nodes make items easier to find? Don't you still have to check each item of an array you skip over? Unless they're sorted I suppose? – Eli Rose Sep 12 '15 at 05:29
  • @EliRose: "slow" may mean two distinct things here: (1) looking in the middle (`a[n//2]`) is linear time `O(n)` for `deque` and constant time `O(1)` for arrays i.e., `O(n)` is "slower" than `O(1)` (for large enough `n`) (2) deleting in the middle (`del a[n//2]`) is `O(n)` for both `deque` and arrays but arrays may perform better: asymptotic time complexity is the same but other properties such as cache locality (for the pointers) may make one algorithm/implementation faster than the other (by a constant factor). – jfs Sep 12 '15 at 11:29
  • Answer is outdated and I couldn't see any way to edit it into shape. – wim Jan 23 '17 at 19:37
6

It is possible to insert with python3.5. index(), insert(), and copy() are three new methods only available in python3.5, new in python 3.5, there is no way to insert into a deque in earlier versions:

The deque class now defines index(), insert(), and copy(), as well as supports + and * operators. This allows deques to be recognized as a MutableSequence and improves their substitutability for lists. (Contributed by Raymond Hettinger issue 23704.)

You can remove from the middle of a deque, either using del or deque.remove:

 deq = deque([1,2,3,4,5,6])
 del deq[4] 
 deq.remove(3)
Padraic Cunningham
  • 176,452
  • 29
  • 245
  • 321
2

1) Looks like deque.insert() was added in Python 3.5

2) From the source: https://github.com/python/cpython/blob/d16a1520c2b22c3cbde7547937ba283db3e88ff5/Modules/_collectionsmodule.c#L952-L958

insert(), remove(), and delitem() are implemented in terms of rotate() for simplicity and reasonable performance near the end points. If for some reason these methods become popular, it is not hard to re-implement this using direct data movement (similar to the code used in list slice assignments) and achieve a performance boost (by moving each pointer only once instead of twice).

The reason it's slower is because of how it was implemented.

Nathaniel
  • 770
  • 7
  • 14