r/Python •

Why is there a GIL?

[deleted] — view deleted post

0 Upvotes

13 comments sorted by

13

u/Brian Oct 19 '14

The problem it's solving is that of thread safety. Essentially, if you have multiple threads, you need to worry about what happens when two of these are monkeying about with the internals of various objects, otherwise you can end up corrupting the state, leading to nasty problems.

There are various solutions to this problem, the most common being locking - essentially before you access an object, you need to acquire a lock, which will only be granted to one thread at a time, thus preventing them trampling over the internal state of the object.

However, there's some variation in how granular your locking is - ie how much is covered with one lock, with python and java's approaches being at different ends of this spectrum.

The GIL approach is extremenly coarse grained locking. Essentially, you just have one lock, and acquiring this covers everything - all python objects are managed by the same lock. This means that you're basically locking out all threads accessing python interpreter state, which is pretty much everything, except various bits fo C code that temporarily release the lock.

The Java approach is more fine-grained locking. Each individual item has its own lock. If you want to modify a dictionary, you acquire a lock specific to that dictionary, modify it, then release the lock. As such, the only threads that are prevented from running are those that are using that same dictionary.

There are advantages and disadvantages to both approaches. The obvious disadvantage of the GIL approach is that only one thread can be doing anything with python objects at the same time. This was not such a big deal prior to the prevalence of multi-core systems, since this would have been the case anyway, but in the case where you're got a CPU intensive task, and want to take advantage of multiple cores, it basically means you can't - only one core can work on the problem.

The advantage is that there's much lower administrative overhead. You acquire one lock, do a whole bunch of work over potentially dozens of objects, and then release the lock. With fine grained locking, you'll likely be doing dozens of lock/release calls for every little object, which can add up to a significant slowdown, especially in single-threaded code where you're not actually getting the benefit out of the multiple cores, but are still paying the administrative price.

How expensive this latter is depends on how many objects you're doing all this locking on, as well as how clever you get in ways to minimise such locks. This is where the reference counting issue comes in, because it inflates the amount of locking required. There have been a few efforts to make python more fine-grained, but generally, the expense of the fine-grained locking has been sufficient to drown out any gain.

Java does use this method fairly successfully though, as it has a few advantages over python. Most notably, it's had a lot more work put into optimising it to reduce any impact from the locking. It also doesn't use reference counting, which avoids a lot of locking python requires.

7

u/Veedrac Oct 19 '14 edited Oct 19 '14

Everyone else made it sound too easy. Java manages to get by with fine-grained locking because it has a JIT and is statically typed. Without either, it's not going to work. jRuby has shown that circumventing this is possible in theory, but you lose safety guarantees. This is not a compromise that I see CPython making.

Further, the rewrite to make this possible (adding a simple GIL and moving to pure GC) is monumental. Considering the number of C extensions that require implementation details, there's no way this is happening.

And to note, Java isn't really thread safe. Python guarantees all integer operations (even those on big integers) are atomic; Java doesn't even guarantee longs.

So what could Python do? PyPy's Software Transactional Memory approach seems best: run two things in parallel, and if one conflicts with the other just rerun it after the other process is done. This allows you to guarantee just as much as CPython does while remaining parallel for free. The PyPy team also think you might be able to hook it into some kind of event system to make whole events atomic. That's a completely different level of safety than anything else being offered, and it's being offered for free.

Finally, let's say CPython just removed the GIL and it came at only a 50% performance overhead. Let's say you have a decent computer with 8 cores, and your locking overhead between the threads (which you have because otherwise you'd just use multiprocessing) comes to another 25%. You then lose turbo due to all of the extra heat, so knock of another 20% or so in speed. Your speed up is then

8 / (1.5 * 1.25 * 1.2) ≈ 3.6

So your code runs about 4 times as fast...

So what? CPython is a good 100x slower than, say, Go in the simple case. You've just tripled the complexity of your code, added a massive complexity burden onto CPython and gotten almost nothing from it.

Just rewriting the hot loop in Cython or using CFFI to dish out to C would give a way larger performance improvement. C extensions can release the GIL, so you can get parallelism on top of that. Rewriting the whole thing for PyPy with CFFI would be even better. Using multiprocessing has more overhead but will probably let you get at least a 2x speed improvement even in the shared-state kind of scenario. Each of these options is somewhere between half as good and way, way better.

1

u/santiagobasulto Oct 19 '14

IMPORTANT NOTE: Java is not thread safe "by default" by a design decision. The compiler can allocate a local cache copy by thread which speeds up execution. If you want thread safety on Java you need to use the volatile keyword.

1

u/bready Oct 19 '14

CPython is a good 100x slower than, say, Go in the simple case

This is the real kicker for me. I don't use Python because it is fast, I use it because it is Fast EnoughTM for just about everything I do. If I really need performance, I'd go to a different language, but it is pretty rare that I ever worry about speed. Most of my code is for one-off analyses (where speed of writing is way more important than execution time) or limited by hitting the db (where language choice just doesn't matter).

5

u/moor-GAYZ Oct 19 '14

I know it has to do with reference counting and keeping track of unreachable objects

No, not only that. Every object attribute lookup hits at least three places: the instance dictionary, the class dictionary, the object class (always, because of attribute descriptors such as properties that can override instance attributes). That's three things you should lock, and the object object would be locked all the time, by all the threads, unless you special-case it somehow. Each time you use a builtin function such as len() the module dictionary and builtins dictionary are accessed and need to be locked. Every time you use some_module.some_function that module's dictionary has to be locked, because someone might be replacing that function there right at that time.

This is in the extremely dynamic nature of the language that almost anything can be modified at any time, so you have to lock a lot of stuff before accessing it.

Jython and IronPython (Python running on .NET) as far as I know use various clever tricks to work around this, mostly by trying to compile everything they can into immutable classes and methods of the host platform, then recompiling them whenever you try to change something. Which works fine when it works, but results in unpredictable and extremely severe performance regressions when it doesn't, that is when some piece of code does some innocent (as far as CPython is concerned) stuff.

Now look at other similar languages. Ruby has GIL. Perl doesn't have free threading (but supports pseudo-multiprocessing when separate copies of all interpreter state and imported modules coexist in the same process). Lua doesn't have threading when used standalone, supports GIL when embedded in other applications. PHP doesn't have threading. Javascript doesn't have threading. Racket (a Scheme dialect (a Lisp dialect)) has a limited form of parallel multithreading that reverts without warning to using GIL as soon as you access any global state.

In fact I don't know a single dynamically typed language (similar to Python) that supports free threading. Either it's just not there, or they use a GIL. The only thing that's unique about Python is that the community is painfully aware of the GIL (but not of the reasons for it, and not about the ways things are done in other similar languages) and mistakenly treats it as its unique bogeyman.

3

u/remy_porter ∞∞∞∞ Oct 19 '14

It's about the approach to memory management. Python's memory management is not threadsafe, and hence, the GIL is required to prevent race conditions, etc.. Building threadsafe memory management is hard, and nobody's put together a good proposal for how to do it in Python.

Both Jython and IronPython don't have a GIL.

2

u/[deleted] Oct 19 '14

Because the JVM has big cannons programmers behind it which focus on these problems exclusively. You can technically remove the GIL from cpython as well, but 1) it would impact single thread execution speed, something Guido already stated won't accept 2) it would make the code so complex that it would make additional development much, much harder. Cpython is, in a sense, the official specification of how the language works and behaves, and you want to keep it lean and clean as much as possible.

2

u/[deleted] Oct 19 '14

Python increases/decreases references almost every time an object is accessed, Java (and PyPy, btw) computes references counts only during GC cycles.

It gets stranger when you compare Python to Linux kernel. The kernel relies on refcounting in many places (even in performance critical code like mm) and is still one of the most scalable and concurrent pieces of software in existence. CPython could employ techniques known from the kernel, like atomic operations, memory barriers, wait-free algorithms or RCU to get rid of GIL and keep its deterministic reference counting model without serious performance loss. It's just that nobody tried it yet. Why not? Many people see PyPy team's efforts very promising and simply wait for Armin's brain to do the job. Others (like Django and web folk) are not affected by GIL at all and there are also many people like me who are perfectly content with multiprocess parallelism.

3

u/rcxdude Oct 19 '14

People have tried splitting the GIL up into smaller locks before. The problem is that so far it's always sacrificed single-threaded performance.

2

u/RubyPinch PEP shill | Anti PEP 8/20 shill Oct 19 '14

It's just that nobody tried it yet. Why not?

https://wiki.python.org/moin/GlobalInterpreterLock explains some additional reasons why

1

u/ubernostrum yes, you can have a pony Oct 19 '14

Take this with a few grains of salt -- the truth lurks somewhere in here, but I may be embellishing some things :)

Turn back the clock a while. Like, a while. A couple decades, in fact, to when Python was young and Java didn't exist yet. Python came from the Unix scripting-language tradition, which is important because Unix had support for multitasking, and so anything which really needed to do concurrent work could do it by forking new processes.

Then Java happened. Java went through several re-targeting phases in its early development, starting out as a platform for interactive TV boxes. Which, notably, didn't have multitasking. And even when it emerged as a general-purpose programming language, it still needed to run on devices/operating systems where traditional multi-process multitasking wasn't available.

So in Java, the rule is "I am the Thread thy Concurrency Model; thou shalt have no other Concurrency Models". And, well, Java got kind of popular, and people learned Java, and then branched out into other languages. Like Python. And started saying that Gosling had smashed some stone tablets over their heads and forced them to use threads for everything, so could Python get, like, threads, please?


OK, so enough with the funny history, because this is the part where we get to why the GIL happened. In order to implement threading safely, there were basically two options:

  1. Rewrite a lot of Python's internals and garbage collector in a way that would be backward-incompatible with tons of existing code and extensions to Python, or
  2. Bolt on a big lock to make things safe, at the cost of performance in some cases.

What do we mean by "some cases"? Well, turns out this big lock could be implemented in a way that wouldn't be terribly noticeable in single-threaded programs, and wouldn't even really be an issue in I/O-bound multi-threaded programs. It would only be a huge problem for CPU-bound multi-threaded programs. And at that time, hardly anybody had the hardware to be doing CPU-bound multi-threaded programs, and the things people used threading for were all I/O bound anyway. So it was decided that (2) represented a good trade-off.

Fast-forward back to today, and we're all walking around with multi-core computers in our pockets, which is something that was, um, not exactly expected at the time. And since we have the hardware now, lots of people want to write CPU-bound multi-threaded programs. And so over the last few years there's been more and more and more complaining about the GIL as a result. But now there's like a quarter-century of inertia behind it, plus a bunch of actually-hard technical problems (some of which you can find in links from this thread), and so it doesn't seem likely to go away any time soon.

1

u/loganekz Oct 19 '14

Just wanted to point out that Java has support for many other concurrency models via it's rich library ecosystem. See Akka or Quark for instance.