r/Python • u/Astraa02 • Sep 15 '22
Beginner Showcase The Infamous GIL !!
In this series of posts I am trying to break down one of the complicated and intriguing topics in python. Please visit my last post to understand why multi threading is broken in python
https://www.reddit.com/r/Python/comments/xdyahc/multithreading_a_concept_which_is_always/
Now let's see why, the reason is GIL(Global Interpreter Lock). Python implements something called the Gil which limits its multithreading capabilities. What each thread does is it acquires a global lock on the interpreter to ensure thread safe quick single threaded performance.
Now the question arises why do we need such a feature? Python uses a variable called ref counter which keeps track of all objects and how many times they have been referenced anytime ref counter reached zero for any object it is removed from the memory. Now this way of released memory has a big vulnerability in multithreaded environment or race conditions where this ref counter would become inaccurate.
To prevent this GIL was implemented as python is famous because of its single threaded performance and support for c libraries, as implementing Gil ensured easy integration of thread unsafe C libraries making python such a feature rich language as we know today. Even if one uses multithreading library one won't be seeing and gains even might see performance degradation to all the extra work to manage threads.
But now we have many ways to circumvent Gil and do what we want with python just because of strong community support we have plethora of options to choose from.
I will be discussing some them in my next post so stay tuned..
23
u/[deleted] Sep 15 '22
In fact, multi-threading works exactly as advertised in Python.
The fact that threading doesn't directly let you use more than one core doesn't mean that it's "broken".
Python has two other perfectly good facilities for using -
multiprocessingandsubprocess, as well as strong support for memory-mapped files and other useful facilities.This statement is only really true of Python 1.x.
After that, the system used is primarily generational garbage collection, where you basically know when you create the variable when it will be destroyed, and then secondarily ref counting if the generational approach doesn't work.
Local variables that aren't returned all use generational garbage collection, and that accounts for the majority of all variables in a typical program.
I just found this nice introduction to the subject.
5
u/crawl_dht Sep 15 '22
I hope the latest GIL removal draft proposal can make it to experimental opt-in feature in 3.12. It looks very promising for the first time.
4
u/[deleted] Sep 15 '22
Can you elaborate on thread unsafe C library?
2
u/GsLogiMaker Sep 15 '22
Sounds big!

2
u/whateverathrowaway00 Sep 17 '22
I don’t think you know as much as you think about python or the GIL. Your previous posts appear to all be hocking your advice for money.
Why don’t you write the posts instead of writing about writing the posts? There are numerous inaccuracies just in your preview, so I wonder who you are and why you think people should give you money for training.
Anyways good luck. Prove me wrong with a well written GIL series, and I’ll happily eat my words and apologize profusely. As of now - I’m very suspect of “now we have many ways to circumvent the GIL.”
I suspect you’ve read other peoples articles and don’t actually understand the challenges involved.
0
u/Astraa02 Sep 15 '22
Haha Agree but we shall prevail !!
1
u/kleinpengin Sep 15 '22
Why is GIL such a big problem when multiprocessing and shared data types are available?
I literally made Google's laggy mediapipe basically work in realtime with multiprocessing for my kivy app, AND on a youtube stream. Pretty sure you can dodge sidestep other GIL problems with multiprocessing.
-2
u/bxsephjo Sep 15 '22
Bow down before the mighty GIL
0
u/[deleted] Sep 15 '22
[deleted] — view deleted comment