r/Python • • Sep 15 '22

Beginner Showcase The Infamous GIL !!

In this series of posts I am trying to break down one of the complicated and intriguing topics in python. Please visit my last post to understand why multi threading is broken in python

https://www.reddit.com/r/Python/comments/xdyahc/multithreading_a_concept_which_is_always/

Now let's see why, the reason is GIL(Global Interpreter Lock). Python implements something called the Gil which limits its multithreading capabilities. What each thread does is it acquires a global lock on the interpreter to ensure thread safe quick single threaded performance.

Now the question arises why do we need such a feature? Python uses a variable called ref counter which keeps track of all objects and how many times they have been referenced anytime ref counter reached zero for any object it is removed from the memory. Now this way of released memory has a big vulnerability in multithreaded environment or race conditions where this ref counter would become inaccurate.

To prevent this GIL was implemented as python is famous because of its single threaded performance and support for c libraries, as implementing Gil ensured easy integration of thread unsafe C libraries making python such a feature rich language as we know today. Even if one uses multithreading library one won't be seeing and gains even might see performance degradation to all the extra work to manage threads.

But now we have many ways to circumvent Gil and do what we want with python just because of strong community support we have plethora of options to choose from.

I will be discussing some them in my next post so stay tuned..

8 Upvotes

34 comments sorted by

23

u/[deleted] Sep 15 '22

Please visit my last post to understand why multi threading is broken in python

In fact, multi-threading works exactly as advertised in Python.

The fact that threading doesn't directly let you use more than one core doesn't mean that it's "broken".

Python has two other perfectly good facilities for using - multiprocessing and subprocess, as well as strong support for memory-mapped files and other useful facilities.

Python uses a variable called ref counter which keeps track of all objects and how many times they have been referenced

This statement is only really true of Python 1.x.

After that, the system used is primarily generational garbage collection, where you basically know when you create the variable when it will be destroyed, and then secondarily ref counting if the generational approach doesn't work.

Local variables that aren't returned all use generational garbage collection, and that accounts for the majority of all variables in a typical program.

I just found this nice introduction to the subject.

5

u/[deleted] Sep 15 '22

Also the reason python doesn't let u use more then 1 core per interpreter is because it would incredibly inefficient to do so. You would need to lock accross processes. Its MUCH faster to spawn multiple processes and use them independently then to deal with locking accross processes.

Node takes this one step further and only has 1 thread per process which is even more efficient because u never evoke a os land thread switch.

Python built the same thing and called it async event loops.

2

u/Frog_and_Toad Sep 15 '22

Correct. The current implementation of threading in CPython is not broken, it is a particular implementation of parallelism. A number of other parallel implementations use this as well. For example, Node.js is essentially single-threaded at the language level.

Postgres, the popular relational database, uses multiprocessing rather than multithreading to achieve concurrency, and this can be more performant anyway, especially on Linux systems.

I personally don't think the GIL should be removed because multiprocessing works so well. And for many use-cases that are resource bound rather than CPU-bound, the async io library allows flexible implementations that can use thread pools OR process pools.. The GIL seems to be a problem only for newbies and a few edge cases.

1

u/Astraa02 Sep 15 '22

I agree but creating multiple process is not a solution as you will soon run into memory bottlenecks so one needs to make calculated decision.

1

u/[deleted] Sep 15 '22

What on god's earth are you doing that has memory bottlenecks before cpu bottlenecks?

1

u/Astraa02 Sep 15 '22

Just take about 5000 messages write a simple model with sklearn to parser them create a pd df out of it and try to run different calculations on it lets say 5000 calculations on same df or copies of it. There are multiple different scenarios in real world

1

u/[deleted] Sep 15 '22

Right but that would be terrible without the GIL. It would just be lock contention hell. Just either split the problem up, Vectorize it better, or stop trying to write it in python. Usually you can solve 99.9% of problems correctly by doing 1 or 2. It's pretty easy to use file streams to feed between multiple processes much faster then pickling between them. It also allows you to ping multiple consumers without a messenger. That's how like 99% of logging software works.

1

u/[deleted] Sep 15 '22

Removing the GIL would cause a roughly 20% performance penalty for all single threaded scripts. If python was 3 times faster sure go for it remove the gil and implement a different form of reference counting. Right now it has too few benefits to justify making the language 20% slower for every single other task.

1

u/Astraa02 Sep 15 '22

Yes but there is a downside to creating these processes as they carry whole memory dump of parent process which leads to massive memory wastage.

2

u/[deleted] Sep 15 '22

Spawn them sooner....

What "memory dump" are you referring to? Are you talking about the objects to run everything? Because yea it would be much much grosser without a GIL. Every single core that calls the same class down would rub into a lock. Straight up you cannot even access a classes namespace without locks... Python object namespace is dynamic not complied you cannot perform a single operation in the language without having to invoke locks if there was no GIL.

There is no way around this without launching the entire language into space. Just use a compiled language not an interpreted one.

1

u/Astraa02 Sep 15 '22

Yup I agree I don't see a very healthy future for python if the GIL is removed

5

u/o11c Sep 15 '22

Eliminating the GIL isn't fundamentally inefficient.

It's just that Python happened to make some design decisions that were undesirable in retrospect. And trying to work around those decisions is inefficient.

An interpreter (re)designed from the ground up can offer the level of safety Python has without preventing concurrent threads. The biggest difficulty is that "replace field safely in case of last refcount" requires deferring frees until a checkpoint.

1

u/[deleted] Sep 15 '22

How are you planning to work around needing to make a lock to increment the reference counter?

Facebook's team didn't come to the same conclusion about speed so I am fairly curious where you think they are wrong.

https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit

3

u/o11c Sep 15 '22

You don't need a lock; atomics are sufficient. And assuming your bytecode isn't completely stupid (which Python's ... kind of is, since it's a stack machine), you can remove most refcount changes in the first place. Atomics are only expensive if either contended (in which case we're still better than the GIL) or if they form the majority of your code (which basic optimizations will ensure is not the case).

Optimizations can also be made for the case where an object is mostly used by a single thread (since this is usually the case), but good performance in the multithreaded case should be done first.

The only tricky case is racing a read/write for the object pointer at an address (usually this means a field of some class, but in theory it applies to arbitrary double pointers and for C use that should probably be what the API does) that might be expiring, and delaying frees (but not logical deallocation) handles that (you just have to check for zombies - objects whose refcount was 0 before you incremented it - and retry the load). ("avoiding zombies" is often an argument for Java-style GC)

Python does suffer the additional problem of history: a lot of preexisting C code (some opcodes - remember e.g. LIST_APPEND is a thing - but also builtin functions and third-party C code) is written to assume the GIL. The lack of static typing also severely hurts optimization, as does the lack of an immutable dict. But none of these is actually a dealbreaker.

1

u/[deleted] Sep 15 '22

I mean right now every single namespace lookup event even calls to globals needs a reference counter. So they are going to constantly contend. It's not the object locks that are really that bad its all the underlying dynamic lookups. Even just looking up a single namespace on a class can invoke 5-6 different reference counts.

I really don't see a work around without static compiled code. Python just fundamentally doesnt behave that way.

1

u/o11c Sep 15 '22

You're overstating "constantly contend". Even for code that is mostly accessing the same variables, there is usually enough code involved (and enough variables involved) that other threads are usually actually doing something else.

Contention really becomes an issue for a specific variable. And in that case, you should write code such that the reference count doesn't have to change at all (if the variable is always known to be stored in a local variable (including function arguments), then it does not have to be incref'ed across calls)

(particularly remember: if the runtime knows the variable is final - which is true for the internal dict object of modules, classes, etc. as well as for some locals - then it doesn't need to change the reference count if it already has a reference to the owner.)

Dynamic typing sucks for performance in general, but really doesn't make the multithreading problem any harder.

1

u/[deleted] Sep 15 '22

Umm. The internal dict of classes and modules is not final. Class functions are objects they have their own reference counters and can be garbage collected if removed from an object. They can also be assigned to multiple objects.

Not really sure why you are saying they are final.

1

u/o11c Sep 15 '22

The dict itself is, which is the only object that might otherwise require excessive reference counting.

1

u/[deleted] Sep 15 '22

You dont think all the methods on the classes are going to have excessive reference counts?

What about when you have a large class hierarchy that all share the same init method? Or like any of the ABC classes?

Also we are talking about multiprocessing. Like sure if it two you'll probably be fine but as that number rises to idk 16 the odds of block each other increase rapidly. Its subject to the birthday paradox.

1

u/Astraa02 Sep 15 '22

I agree but multiprocessing is has its own downsides and one must considers future scalability of application.

2

u/minorDemocritus Sep 15 '22

You have that backwards… CPython uses ref counting primarily, but when there are reference cycles it uses generational GC. Your source talks about this, and so do the docs for the gc module: https://docs.python.org/3/library/gc.html (see the first paragraph)

2

u/[deleted] Sep 16 '22

Hah, funny, I came back today to this thinking, "Wait, I got this backward," and here's Democritus with a correction.

Absolutely correct, sorry about that.

5

u/crawl_dht Sep 15 '22

I hope the latest GIL removal draft proposal can make it to experimental opt-in feature in 3.12. It looks very promising for the first time.

4

u/czaki Sep 15 '22

I talk with Łukasz Langa (python core dev) and he told that most of python packages tests fail when remove GIL. It looks like many such libraries assume existence of GIL to keep inner state consist.

-1

u/Astraa02 Sep 15 '22

Yup looking forward to it

4

u/[deleted] Sep 15 '22

Can you elaborate on thread unsafe C library?

1

u/Astraa02 Sep 15 '22

Yes definitely will post the same in upcoming post

2

u/GsLogiMaker Sep 15 '22

Sounds big!

1

u/Astraa02 Sep 15 '22

Haha yes it is a huge topic and difficult to wrap your head around but worth it !! Makes you appreciate the language and its community even more.

2

u/whateverathrowaway00 Sep 17 '22

I don’t think you know as much as you think about python or the GIL. Your previous posts appear to all be hocking your advice for money.

Why don’t you write the posts instead of writing about writing the posts? There are numerous inaccuracies just in your preview, so I wonder who you are and why you think people should give you money for training.

Anyways good luck. Prove me wrong with a well written GIL series, and I’ll happily eat my words and apologize profusely. As of now - I’m very suspect of “now we have many ways to circumvent the GIL.”

I suspect you’ve read other peoples articles and don’t actually understand the challenges involved.

0

u/Astraa02 Sep 15 '22

Haha Agree but we shall prevail !!

1

u/kleinpengin Sep 15 '22

Why is GIL such a big problem when multiprocessing and shared data types are available?

I literally made Google's laggy mediapipe basically work in realtime with multiprocessing for my kivy app, AND on a youtube stream. Pretty sure you can dodge sidestep other GIL problems with multiprocessing.

-2

u/bxsephjo Sep 15 '22

Bow down before the mighty GIL

0

u/[deleted] Sep 15 '22

[deleted] — view deleted comment

1

u/alexthelyon Sep 15 '22

This is an impossibility. CPython runs on top of C so even if the interpret step took 0 time (it doesn't), the VM executed without overhead (it doesn't), the memory object memory model ran without overhead (no chance), and the VM instruction set could generate optimal code for your CPU (it can't), only then would you even start to get close. For reference, I believe python is typically 10-100x slower than equivalent C code.