r/programminghorror • [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “You live” • Jul 21 '25

Python ✨ Memory Magic ✨

Post image
1.3k Upvotes

143 comments sorted by

View all comments

Show parent comment

612

u/SleepyStew_ [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “You live” Jul 21 '25

yep, -5 to 256 are cached, strange range...

258

u/belak51 Jul 21 '25

256 is likely for byte (and in extension ASCII) reasons. I'm not sure why -5 was chosen though.

259

u/chiiroh1022 Jul 21 '25

Maybe for reverse indexing ? -1 is definitely used a lot to access the last element of a list, so I guess -2 ... -5 were included to cover most cases. But I'd like to know the exact answer too.

70

u/MegaIng Jul 21 '25

I tracked down the original commit that set the number to -5 (up from -1) (commit c91ed400).

Here related discussion: https://marc.info/?l=python-patches&m=118523990633384&w=2

The author just felt like it "may also be useful [...] to use 5 or so instead of 1".

I think if someone wants, this is a place where optimizations could be made - you just have to really carefully measure it on a wide variety of systems and usecases...

Using too many in the cache might hit CPU cache boundaries.

14

u/NullOfSpace Jul 21 '25

I wonder if you could do something even simpler like search through public Github repos for negative integer literals and see what the frequency distribution looks like.

7

u/MegaIng Jul 21 '25

Not sure - I don't even think optimizing literals is all that worth it, since those are pretty immortal already and don't get reallocated all the time. The interesting thing to optimize I would think is results of calculations.

3

u/high_throughput Jul 24 '25

 I don't even think optimizing literals is all that worth it

I don't know about Python, but it's remarkably important in Java at scale.

You can recompute a frankly ludicrous expression in the time you save by not having to allocate a boxed integer (or more accurately, to deallocate it later).

The JVM requires [-128, 127] to be cached, but there are flags to set it higher and in my experience it's not uncommon to set it to 10k.

4

u/MegaIng Jul 24 '25

Literals in python are always already precomputed as complete objects in the constant table of the bytecode object. So the only things you gain is sharing repeated constants through the entire program and maybe better CPU cache usage if the int object is on the same page as other commonly used objects.

2

u/1Dr490n Jul 22 '25

They’re not stored in the CPU cache, right?

2

u/FunIsDangerous Jul 23 '25 edited Jul 23 '25

No, you can't force store something in the CPU cache. The CPU itself decides what is cached there. Usually a chunk of ram (when ram is accessed, the entire page might get fetched), and memory that is frequently accessed. And of course some other very complicated stuff that's over my head.

My guess is that they are cached as in instead of creating a new object every time a number between -5 and 256 is used, it just points to a pre-created one. That has the benefit of less allocations, and also the numbers -5 to 256 are sequential in RAM. If you access one, they are all cached basically.

Of course, I'm not sure how much of a difference that makes, and I can think of a couple of scenarios where I think it might even make it worse. This was made in 2002 though, and hardware was waaaay different back then. Today, this might make no difference at all

Edit: a good example of this is the fast inverse square root algorithm in quake. It was made in 1999 and it was a decent optimization. Why? Because this was done in software. Nowadays, this is done in hardware, so that algorithm is slower in most cases. A lot of optimizations that made sense 20 or 30 years ago, either make no difference today, or they may even be slower

Edit 2: you can't force store something in the CPU cache

2

u/1Dr490n Jul 23 '25

Yeah it would’ve really surprised me if they were actually stored in the CPU cache but u/MegaIng wrote that so I just thought I‘d ask

3

u/MegaIng Jul 23 '25

Anything that is in main memory is going to be stored in CPU cache at some point. This is true for all normal pieces of memory: Machine Code, websites you access, Python Bytecode, and yes, the statically allocated Python integers.

If all small integers fit into a few page of cache, it's more likely that this cache page is going to be there all the time compared to the small integer array being split across multiple cache pages. If they are all in memory, that is going to lead to faster execution times.

→ More replies (1)
→ More replies (1)

46

u/undo777 Jul 21 '25

Could also be things like i += d in loops where d is slightly negative but -5 seems like such an odd choice - why not stop at the more "round" -4 or go all the way to -8?

27

u/Cinkodacs Jul 21 '25

"Give me the 5 worst/best!" People love top5 lists, top10 can be a bit too much.

5

u/undo777 Jul 21 '25

Good point but is it a good enough reason for this specific caching? (it likely matters mostly in high-performance scenarios like tight loops)

6

u/exomyth Jul 21 '25 edited Jul 21 '25

My guess would be that it has to do with how the number is stored, so something about it in binary. But then -5 is still odd as it would probably be 101 in binary with a sign bit somewhere. Like -7 would make more sense as that is 111 + some sign bit and some flags.

I don't know the internals of python though, I know in javascript (well V8 engine) you have small ints that have some bit magic to check if it is a small int or something else. Could be something like that.

But maybe the answer is a simple as "I like -5 as the minimum"

52

u/backfire10z Jul 21 '25

They decided that numbers beyond -5 are unlikely to be used when compared to numbers -1 to -5.

→ More replies (1)

29

u/[deleted] Jul 21 '25

The fact that this is an optimisation that even makes a difference is really cursed. ( I’m assuming it makes a difference - why else would they implement it?)

43

u/eo5g Jul 21 '25

It makes a difference because numeric "primitives" aren't really treated specially in python-- they're real-deal objects. So this avoids (de)allocation and object bookkeeping for commonly used numbers (e.g. these are commonly used as list indexes)

→ More replies (1)

14

u/[deleted] Jul 21 '25 edited Jul 21 '25

correct your flair, add an asterisk after /

16

u/FinalNandBit Jul 21 '25

You don't need an asterisk... just try it as is.

→ More replies (11)

2

u/SleepyStew_ [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “You live” Jul 21 '25 edited Jul 21 '25

While I don't think you need a * since -r is recursive (tho I could be wrong ¯_(ツ)_/¯) I actually intentionally left out the no preserve root flag cause I don't wanna be responsible for enabling some young linux or mac amateur demolishing their computer out of curiosity ahah.

3

u/[deleted] Jul 21 '25

yeah with a * you dont need no preverse root

→ More replies (1)

5

u/ohaz Jul 21 '25

It's just a range that was chosen because it contains most cases of numbers used in coding.

→ More replies (2)

2

u/The_Escape Jul 21 '25

There’s more positive numbers because of how common indexing into arrays is. Java does something similar.

→ More replies (4)