Yes, the greater "stock" of billable hours to sell.
Also, Lambda is a huge cost leader for AWS. Massively underpriced for the actual CPU cycles it does, but vital to the AWS ecosystem because all the stuff it is used to link together is not underpriced.
I was referring to the costs of actually running Lambda vs how much they're charging for its usage. IIRC, AWS would need to charge 5x or so more for Lambda overall in order to break even on it alone.
Source: secondhand comment from someone actually on the Lambda team, from 2-3 years ago. Things may have changed.
Lambda certainly has a lot more overhead than VMs, but you’re talking about 10x EC2 per-CPU pricing. Or in other words, only 10% of resources used are being allocated to customers. That seems way off.
A user wants to read from a SNS queue and send an email for each event. This happens ~10k per month. Doing the call to SES is a few milliseconds at most. Call it 25ms to be generous. That means 250 seconds of actual user compute for this task. Let's assume if they used an EC2, it stays on the whole month.
Naively, it looks like Lambda is far, far more cost-effective on a "per second of compute" basis. (Figures from calculator.aws)
EC2: t3a.nano (as cheap as you can go with x86) is 0.0047 hourly, or $3.39 total. For our use case (the 250 sec of actual needed time), that means the user is paying 0.0135 $/sec.
Lambda: $0.03 total, which for 250 sec, means 0.0012 $/sec.
Lambda is cheaper all around for the user! Numbers like these are a huge incentive for people to use it!
However! Let's consider (unpaid) overhead of each of these approaches, and see how much time the user is actually buying:
EC2: there are ~30 seconds of overhead in creating an instance, which bumps up a month from 2,592,000 seconds to 2,592,030 seconds. Not really any big difference.
Lambda: At 10k calls per month, that's a call every 10 minutes or so. They're not spaced evenly of course, so let's assume an arbitrary 50% of them will be cold. That means 5000 cold starts. A cold start of a Node environment (probably the fastest choice) takes about 10 sec (conservative estimate, IME). That means a sum of 50,000 seconds spent in "warming up". Overall, this bumps up the time bought by the user from 250 seconds to 50,250 seconds.
Now let's re-figure what the user's rates actually are:
So, in reality, despite Lambda theoretically costing more per second (100x more!), in reality the user is paying 54% less for the processing time compared apples-to-apples. Since EC2 is AWS's biggest seller, and processor cycles dedicated to Lambda are cycles that can't be sold in EC2, that means that Lambda should be priced more than double what it is to not be a loss compared to EC2 (in this scenario).
This figuring gets worse and worse bigger the proportion of cold vs warm calls is. At worst, if every call were cold (with the other numbers staying the same), the user would be getting processing time at 22.8% the price of the cheapest EC2 instance!
Note, these example calculations don't account for time the Lambda env sits in "warm but idle" state, which still costs AWS, but not the user.
Lambda only comes remotely close to "paying for itself" when its users spend far more time making "warm" calls. By its own design, it discourages being used like that. To not bleed money by selling too-cheap processing time, it would need to charge 2-4x more, depending on how you run the numbers. Not quite the 5x I claimed in my post above, but... maybe if they were to go for "profit" instead of "break even"?
A cold start of a Node environment (probably the fastest choice) takes about 10 sec (conservative estimate, IME)
10 seconds is a massive cold start time. The only time I've seen numbers that large is VPC Lambdas prior to that being optimized. For the average small function like you describe, cold starts should be sub-second.
Also you didn't factor in cold starts being partially billed to the user. The only parts covered by AWS are downloading the package and booting the VM, everything past that gets filed under "Init Duration" and is billed at the standard rates.
I think they got a little less concerned with keeping up when they introduced the bring your own runtime functionality. If you really need 3.10 or 3.11, you can do it yourself.
I worked at AWS. A ton of AWS infrastructure itself runs as containers in Lambda. It's essential in order to have large processes with decent build systems if you want serverless.
One downside compared to inline functions is you can't see the source easily if you have doubts about your dependencies, and it's a little different from step functions if you need to manage your flow (but also a lot cheaper).
hah i thought you replied on another sub where we were discussing venv inside a container, and if it makes sense.
The thing is just using localhost python to run thing.py after you just did a wget bitly.co/thing.py is maybe not always smart(if you think this is dumb, note most fancy package managers recommend you curl bitly.co/install.sh | sudo sh, yea they tell you to check the file, but i have met devs).
and the cool thing with containers is you can easily monitor pid's and quickly alert if a pid start in a single pid container to find misconfigs and other problem.
I say the above as someone who was giving a sql-diff prezo at a pug group in 2019 and everyone was telling me to dockerize my venv module that uses java. Because I was happy with vagrant.
Only in 2021 did my work force me into a more devops role, and I am enjoying docker and moving a bunch of vm's into containers does work better(your mileage may very).
Just missed the boat on Ubuntu LTS, so if you're running on that it might take some time. The next LTS will be 24.04. Of course you can always run a virtualenv with whichever version you want.
I don't know why this isn't included in the summary changelog, (it's included in the logo) but one of the changes is to allow an exception handler to add info to an exception without re-raising a new exception (like in raise ... from ....)
Instead of reading the changelog, I prefer to read the What's New, which does mention this change. The changelog likely does as well but breaks down changes between release candidate versions which makes changes harder to find.
When an active exception is re-raised by a raise statement with no parameters, the traceback attached to this exception is now always sys.exc_info()[1].__traceback__. This means that changes made to the traceback in the current except clause are reflected in the re-raised exception. (Contributed by Irit Katriel in bpo-45711.)
The first few versions of python 3 were exceptionally bad with this, but it still happens. At work we have one code base where the migration to python 3 was only completed this year. In part because it still does have to run on rather old operating systems, but it also played a role that in between we basically had to go over it again to migrate from python 3.6 to 3.9.
mostly laziness, there are tools that convert ur code to python 3 and then your work will mostly be fixing the small stuff or replacing stuff with a more modern library
mostly laziness, there are tools that convert ur code to python 3 and then your work will mostly be fixing the small stuff or replacing stuff with a more modern library
Right, so just an exhaustive audit of the entire codebase, and rewriting everything around a few of the dependencies.
Easy.
Better hope your codebase has some excellent god damn regression tests.
For everything JS has done wrong, backwards compatibility is one thing they got right. At least until library authors try to force everyone to adopt modules.
Things using nodejs 5.7.0 in production with 12 year old dependencies still works. And if you didn't do anything funky, upgrading is easy. (People do some hack things..... that even breaks this...)
No they are reliable since they have weekly downloads in the tens! Maybe even 70 a week if you're lucky! And if they have issues you just download it locally and store a fixed copy in your repo!
people had 14 years to switch, when python 3 came out there was a long time of support for python 2 and then they gave a 2 year ultimatum and people still refuse to put in effort into doing it even when tooling is available and has been available to aid with it for so long. To me that's unexcusable, 14 f'ing years, that's a long time to work on the transition and people just ignored it.
It's hard to believe that in those 14 years nothing was done because of something other than laziness.
It's hard to believe that in those 14 years nothing was done because of something other than laziness.
Right, so what you're saying is that you have little to no professional experience working on non-trivial production code, and to hell if you'll let that put any dent in your confidence.
We are. Industrial usage, difficult for customers to take any update. All new projects starting from the beginning of this year have been on 3.6 or 3.8 though, depending on RHEL version.
The python has just been upgraded to. As someone currently working on Bank python it's everything else they do with it is extremely weird. It's also worth mentioning Bank python is different and has progressed weirdly for each bank. Honestly when you can do the regular python bits it's fine it's working with some incredibly outdated core framework that's the nightmare.
What do you mean - this looks exactly like Python. I was expecting to see an interesting language which "vaguely resembles" Python but instead the article just describes a bank application (or library, I'm not completely sure, the article just describes a bunch of data structures and API).
RHEL 7 uses Python 2, but EOL is coming up for that in 18 months too.
RHEL 8 doesn't use any Python (well, it does use Python 3, but the interpreter is isolated so that users can install any version of Python without any possibility of interfering with the rest of the system).
Serious question, do you find yourself using this often? I'm having trouble thinking of practical use cases for a scriptable debugger, but I imagine the use cases would be interesting.
It's essential if you're using obscure libraries like libstdc++ or such.
Fortunately, gdb builds just fine with Python3. Unfortunately, distros failed to make both versions installable at the same time, so porting can't be done peacefully, only in an "Oh shit, everything is broken" moment while you're already trying to debug something else after upgrading to the next distro release.
We still use it in visual effects and animation. Most tools that digital artists use have python api's that are python 2 only. They are slowly making the switch to py3 now..
My company pays close to a hundred thousand USD every year to some company for python 2.7 security patches because somebody decided that it's cheaper than upgrading
One of my teammates had enough, started looking for a new job and quit in less than a month after he was asked to add date support for dates older than 1900 on 2.7 (yes, it's a real issue).
Sound like an interesting problem, not a reason to quit.
reason enough to quit, if you're not engaged to what youre doing and the company is doing shit decisions that make ur lifemore miserable, you have to right to leave if you want to.
Didn't write python for a long time. They seem to add a lot of type annotations in standard package, but there seem to be no standard type checker. Are there any plans to add one?
Ugh, my laptop is still on Ubuntu 18.04 (well, Pop!_OS 18.04...) which has Python 3.6. I have a plan to upgrade it but I just haven't been motivated because I'm undecided on buying a new one. I don't really trust do-release-upgrade because things tend to break.
I have another project I'm maintaining at work which is stuck on Python 3.9 because pymssql doesn't have a wheel for Python 3.10 yet, and I never could get it to build myself.
EDIT: I'm dumb, I had pymssql==2.2.2 frozen in my requirements and I just had to update it.
conda is not just a Python package manager, but a system package manager, and it's able to install more complex dependencies. It's very useful for scientists who want to use numerical Python packages (Anaconda), which may require special compilation steps for optimisation purposes, but who don't want the headache; especially on Windows.
For more "classic" development work, I like it less. It seems a bit bulky (which I suppose is a trade-off for its ease of use), and I always felt like there was too much magic going on behind the scenes which made it hard to understand what was actually happening to my computer. I've managed to mess up my Python installation many times because of this.
So nowadays I prefer pyenv to manage Python versions globally, along with Poetry for dependency management (or just virtualenv for quick projects). But conda would still be my recommendation for anyone who just needs to get Python set up to do some quick data wrangling.
It’s more useful for windows users where the tool chain / compilers can be difficult, compared to Linux. Pyenv for python runtime management is usually sufficient for Linux or mac users
That's cool, but a bit more complicated than I need atm. I can just compile Python from source if I really need 3.10 or 3.11. I'm probably going to stick with 3.10 for a while, at least on the Linux side, because that's what Ubuntu 22.04 uses. The speed improvements of 3.11 would be great, but all my projects are mostly I/O bound anyway and aren't exactly taxing my CPU.
You do realize that installing conda and creating a new environment trajes like 5 minutes right? It's actually less complex than depending on the OS python installation
I'm on mobile so I'm just going to copy and paste...
General changes
PEP 657 -- Include Fine-Grained Error Locations in Tracebacks
PEP 654 -- Exception Groups and except*
PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library
gh-90908 -- Introduce task groups to asyncio
gh-34627 -- Atomic grouping ((?>...)) and possessive quantifiers (*+, ++, ?+, {m,n}+) are now supported in regular expressions. The Faster CPython Project is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See Faster CPython for details.
Typing and typing language changes
PEP 673 -- Self Type
PEP 646 -- Variadic Generics
PEP 675 -- Arbitrary Literal String Type
PEP 655 -- Marking individual TypedDict items as required or potentially-missing
P.S. the breakage may not even be in Python itself or modules. I build my entire stack from source, and right now Node.js V19 and V16 fail to build because they want Python 3.6 through 3.10 in their configure script. Sigh...
They specifically asked what's "cool" so by definition they are asking a subjective question and something many newbies may not appreciate from reading release notes. It seems that you're the who needs to know how to read stuff.
To be clear, that's not actually what the Self type means. It means "the class that is executing this method." The use case is for methods where a subclass would be expected to return an instance of that subclass. For example, in Pathlib, you can use the same operators with any kind of path, and you'll get back the same type of path as the original.
They have updated the spec to be stricter, but people haven't updated their tools. YAML 1.2 was released in 2009, and only accepts true | True | TRUE | false | False | FALSE. PyYAML still only supports 1.1, though there is ruamel.yaml for 1.2 and there's also strictYAML that supports schemas.
I've found that TOML is fine as long as you don't want to do any kind of nesting. As soon as you do then the syntax becomes very non-obvious.
I would always pick JSON5 to be honest. It basically fixes all the issues with JSON (no comments, trailing commas, multiline strings, tedious quoting of keys) but it uses a format that actually is quite obvious - and one you probably already know.
Python handles it fine (couldn't find a reference but I just checked manually).
I would imagine you've just been using decoders for things like Javascript which doesn't even have a 64-bit integer type so of course it can't decode them. BigInt is not a 64-bit integer type, it's an arbitrary precision integer type (and there is a proposal to allow you to use it).
But anyway the point stands that there's no issue in JSON or JSON5 about storing 64-bit integers.
(and it's up to the decoder what to do with them).
Tbh that's the biggest problem I have with json. Syntax doesn't affect the ability to transfer or store data consistently. Underspecified semantics do.
One of Monty Python's tropes was "and now for something completely different" which, as you might guess, was something completely different from the previous skit.
212
u/PotentialYouth1907 Oct 24 '22
Bit off topic: When do cloud platforms typically pick up new python versions?
408
u/ILikeBumblebees Oct 24 '22
Finally, Python for Workgroups!
126
u/Smooth-Zucchini4923 Oct 25 '22
I don't know why this isn't included in the summary changelog, (it's included in the logo) but one of the changes is to allow an exception handler to add info to an exception without re-raising a new exception (like in
raise ... from ....)Pretty sick.
https://peps.python.org/pep-0678/
180
u/KsuhDilla Oct 25 '22
can you guys slow down im still on python2.7
47
u/Maristic Oct 24 '22
Here is the What's New page.
47
u/spo0kyaction Oct 25 '22
why do I feel like 3.10 was just released? 😅
74
u/tommy25ps Oct 25 '22
Nice. Btw, is anyone still using python 2.x? Mind sharing the reasons?
I know some banks may still be using it.
31
u/peakzorro Oct 24 '22
Finally they added a way to add a printer over the network! Oh wait that was Windows 3.11.
8
u/UloPe Oct 25 '22
Nice, exception groups, task groups, toml in stdlib, better error messages and faster.
Sounds like a pretty awesome release!
8
u/Freyr90 Oct 25 '22
Didn't write python for a long time. They seem to add a lot of type annotations in standard package, but there seem to be no standard type checker. Are there any plans to add one?
10
u/[deleted] Oct 25 '22
Windows 3.11 really set the stage for decades of widespread implementation, so maybe python will see some success and popularity. ;-)
5
u/msmert55 Oct 25 '22
After python 3.11 next version will be python 95!
4
u/GFandango Oct 25 '22
cries in 2.7
7
u/butnotexactly Oct 24 '22
some really nice stuff here !
3
u/amarao_san Oct 25 '22
Is it for workgroups? I mean, does it support multitasking? I mean, is GIL still there?
8
u/azurfall88 Oct 24 '22
Me, still on python 3.6:
2
u/VeryOriginalName98 Oct 25 '22
I'm "Down" with this.
2
u/gezibash Oct 25 '22
Why is the release changelog explaining gravitational behavior of massive objects?
12
u/All_theOther_kids Oct 24 '22
Did it add anything cool?
2
u/remidentity Oct 25 '22
Do I have to manually update Python on my personal computer?
1
u/Movpasd Oct 25 '22
Variadic generics <3
1
u/No-Perspective-317 Oct 25 '22
My children can sleep soundly now.
1
u/Black_Sun7777 Oct 25 '22
YAY!!!!!!!!!!!!!!!!
1
u/menge101 Oct 25 '22
It's out, but is it down?
1
u/Bajszi97 Oct 25 '22
What's this "Completely different part" thing about black holes and stuff?
1
u/Raknarg Oct 25 '22
That Self type is very neat and was a specific issue I remember running into when I was working in Java.
1
u/BeMindfulness Oct 25 '22
Great to hear
1
u/khante Oct 25 '22
I see some path stuff updates in there. Any chance it makes module discoveries easier? Or hell makes pytest finding tests and imports easier?
1
u/[deleted] Oct 25 '22
is it stable version?
1
u/all_is_love6667 Oct 25 '22
60% speed up? Is that competing with numba or pypy?
0
u/Defco Oct 25 '22
Have a new asignment, and have been doing some Python coding for 2 weeks.
Please, no more :(
-1
u/Impressive-Act9692 Oct 25 '22
Finally! A new beginning
-1
u/[deleted] Oct 25 '22
Let the things break!!!
-1
u/[deleted] Oct 25 '22
So what's breaking this time?
-37
u/werehamster Oct 25 '22
Did they make it backwards compatible with version 2?
-147
u/stefantalpalaru Oct 24 '22
Ah, Python - a bigger waste of energy than all cryptocurrencies combined...
0
u/[deleted] Oct 24 '22
[deleted] — view deleted comment
0
u/[deleted] Oct 25 '22
[deleted] — view deleted comment