Stack Overflow
Stack Overflow

Questions tagged [python-internals]

How does Python work underneath the hood? Use for questions relating to (for instance) the design decisions made and the internal data structures and algorithms used.

This tag is to be used for those posts relating to the internal working of code written in the programming language.

The scope of this tag includes the standard library [Python 2, Python 3].

732 questions
73
votes
1 answer

Where is the "from __future__ import braces" code?

I was wondering what is exactly the code that gets executed on the command: >>> from __future__ import braces SyntaxError: not a chance Since Python is open-sourced, I opened C:\Python27\Lib\__future__.py and looked. Surprisingly, I found nothing…
Elisha
  • 4,811
  • 4
  • 30
  • 46
72
votes
1 answer

Improving performance of very large dictionary in Python

I find that if I initialize an empty dictionary at the beginning, and then adding elements to the dictionary in a for loop (about 110,000 keys, the value for each key is a list, also increasing in the loop), the speed goes down as for loop goes. I…
szli
  • 36,893
  • 11
  • 32
  • 40
71
votes
1 answer

Why does the size of this Python String change on a failed int conversion

From the tweet here: import sys x = 'ñ' print(sys.getsizeof(x)) int(x) #throws an error print(sys.getsizeof(x)) We get 74, then 77 bytes for the two getsizeof calls. It looks like we are adding 3 bytes to the object, from the failed int call. Some…
jeremycg
  • 24,657
  • 5
  • 63
  • 74
69
votes
4 answers

Why do I get this many iterations when adding to and removing from a set while iterating over it?

Trying to understand the Python for-loop, I thought this would give the result {1} for one iteration, or just get stuck in an infinite loop, depending on if it does the iteration like in C or other languages. But actually it did neither. >>> s =…
noob overflow
  • 955
  • 5
  • 14
67
votes
5 answers

list comprehension filtering - "the set() trap"

A reasonably common operation is to filter one list based on another list. People quickly find that this: [x for x in list_1 if x in list_2] is slow for large inputs - it's O(n*m). Yuck. How do we speed this up? Use a set to make filtering…
roippi
  • 25,533
  • 4
  • 48
  • 73
67
votes
4 answers

How exactly is Python Bytecode Run in CPython?

I am trying to understand how Python works (because I use it all the time!). To my understanding, when you run something like python script.py, the script is converted to bytecode and then the interpreter/VM/CPython–really just a C Program–reads in…
mergesort
  • 5,087
  • 13
  • 38
  • 63
67
votes
5 answers

'order' of unordered Python sets

I understand that sets in Python are unordered, but I'm curious about the 'order' they're displayed in, as it seems to be consistent. They seem to be out-of-order in the same way every time: >>> set_1 = set([5, 2, 7, 2, 1, 88]) >>> set_2 = set([5,…
ivan
  • 6,032
  • 9
  • 42
  • 65
60
votes
1 answer

How does Python's cmp_to_key function work?

I came across this function here. I am baffled as to how this would be implemented -- how does the key function generated by cmp_to_key know what "position" a given element should be without checking how the given element compares with every other…
math4tots
  • 8,540
  • 14
  • 58
  • 95
59
votes
5 answers

Two variables in Python have same id, but not lists or tuples

Two variables in Python have the same id: a = 10 b = 10 a is b >>> True If I take two lists: a = [1, 2, 3] b = [1, 2, 3] a is b >>> False according to this link Senderle answered that immutable object references have the same id and mutable…
Ram Vallury
  • 617
  • 1
  • 6
  • 7
58
votes
2 answers

Why is string's startswith slower than in?

Surprisingly, I find startswith is slower than in: In [10]: s="ABCD"*10 In [11]: %timeit s.startswith("XYZ") 1000000 loops, best of 3: 307 ns per loop In [12]: %timeit "XYZ" in s 10000000 loops, best of 3: 81.7 ns per loop As we all know, the in…
WKPlus
  • 6,955
  • 2
  • 35
  • 53
57
votes
2 answers

Why is b.pop(0) over 200 times slower than del b[0] for bytearray?

Letting them compete three times (a million pops/dels each time): from timeit import timeit for _ in range(3): t1 = timeit('b.pop(0)', 'b = bytearray(1000000)') t2 = timeit('del b[0]', 'b = bytearray(1000000)') print(t1 / t2) Time…
Kelly Bundy
  • 23,480
  • 7
  • 29
  • 65
56
votes
2 answers

Set literal gives different result from set function call

Why does the set function call wipe out the dupes, but parsing a set literal does not? >>> x = Decimal('0') >>> y = complex(0,0) >>> set([0, x, y]) {0} >>> {0, x, y} {Decimal('0'), 0j} (Python 2.7.12. Possibly same root cause as for this similar…
wim
  • 338,267
  • 99
  • 616
  • 750
56
votes
5 answers

About the changing id of an immutable string

Something about the id of objects of type str (in python 2.7) puzzles me. The str type is immutable, so I would expect that once it is created, it will always have the same id. I believe I don't phrase myself so well, so instead I'll post an example…
Bach
  • 6,145
  • 7
  • 36
  • 61
56
votes
3 answers

How references to variables are resolved in Python

This message is a a bit long with many examples, but I hope it will help me and others to better grasp the full story of variables and attribute lookup in Python 2.7. I am using the terms of PEP 227 (http://www.python.org/dev/peps/pep-0227/) for…
user3022222
  • 857
  • 9
  • 9
55
votes
7 answers

Complexity of len() with regard to sets and lists

The complexity of len() with regards to sets and lists is equally O(1). How come it takes more time to process sets? ~$ python -m timeit "a=[1,2,3,4,5,6,7,8,9,10];len(a)" 10000000 loops, best of 3: 0.168 usec per loop ~$ python -m timeit…
Omid
  • 2,617
  • 4
  • 28
  • 43
1 2 3
…
48 49