4

In python, I have a global variable defined that gets read/incremented by different threads. Because of the GIL, will this ever cause problems without using any kind of locking mechanism?

Bobby Brown
  • 41
  • 1
  • Are you talking about an implementation using the "threading" module? In this case, you should use the locking mechanism provided: see http://docs.python.org/library/threading.html – avpx Jan 28 '10 at 19:18

2 Answers2

6

The GIL only requires that the interpreter completely executes a single bytecode instruction before another thread can take over. However, there is no reason to assume that an increment operation is a single instruction. For example:

>>> import dis
>>> dis.dis(compile("x=753","","exec"))
  1           0 LOAD_CONST               0 (753)
              3 STORE_NAME               0 (x)
              6 LOAD_CONST               1 (None)
              9 RETURN_VALUE
>>> dis.dis(compile("x+=1","","exec"))
  1           0 LOAD_NAME                0 (x)
              3 LOAD_CONST               0 (1)
              6 INPLACE_ADD
              7 STORE_NAME               0 (x)
             10 LOAD_CONST               1 (None)
             13 RETURN_VALUE

As you can see, even these simple operations are more than a single bytecode instruction. Therefore, whenever sharing data between threads, you must use a separate locking mechanism (eg, threading.lock) in order to maintain data consistency.

tzot
  • 92,761
  • 29
  • 141
  • 204
Daniel G
  • 67,224
  • 7
  • 42
  • 42
  • 2
    Ouch. I mean, 3 upvotes and nobody noticed the `dis.dis("x=753")` output? I corrected the code. – tzot Feb 01 '10 at 21:01
  • Here's an example where I should be paying more attention to what I'm writing - thank you :). Also, it displays the point even more clearly - after loading the value, there are *several* instructions before the value is stored again. – Daniel G Feb 01 '10 at 21:23
  • this is not even 100% true. The bytecode (almost any) can actually call more python code, via `__getitem__` or `__add__` or any sort of other scenario. Then you end up with threads swapping mid-way through a single bytecode. – fijal Aug 31 '12 at 16:33
3

Yes, multithreading without locking almost always causes problems, with or without a GIL.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • 2
    Note that the GIL isn't there to protect your application data; it's only there to protect some Python internal structures like the reference counts for Python objects. You still have to lock your structures. – S.Lott Jan 28 '10 at 19:42
  • 2
    Exactly. The GIL protects Python from itself, but it doesn't do anything to protect you. – Greg Hewgill Jan 29 '10 at 02:34
  • 1
    And of course the vast majority of Python execution engines don't even *have* a GIL. – Jörg W Mittag Jan 29 '10 at 04:39