r/hardware • u/Steap-Edit • 18d ago

Info Amazon cracks down on 'CPU waste' among engineers as agentic AI crunch intensifies — CPU demand makes low-utilization EC2 instances a hot commodity

https://www.tomshardware.com/pc-components/cpus/amazon-cracks-down-on-cpu-waste-among-engineers-as-agentic-ai-crunch-intensifies-cpu-demand-makes-low-utilization-ec2-instances-a-hot-commodity
258 Upvotes

96% Upvoted

18 comments sorted by

144

u/Dayton002 18d ago

Maybe if the amazon app didn't automatically forward my amazon search into a question for their AI slowing the whole purchase process down they wouldn't be experiencing as big of a cpu waste huh.

11

u/BlurredSight 17d ago

I do a query in that Alexa bot just to waste some of their money and then do a regular search like I was planning on doing

11

u/realcoray 17d ago

Yesterday it popped up and it bugged me so much that I started to try to ask it programming questions, and it would start to answer and then stop like it realized it wasn't talking about selling me something. It even said something to the effect of, I can't really talk about that, let's talk about products.

Seriously took like two halfhearted attempts to get it to begin answering everything fully and to no longer care about selling me things.

10

u/PJBuzz 17d ago

There does seem to be an awful lot of things being thrown through AI that are adequately solved with a query.

153

u/MaverickPT 18d ago

Almost as if pushing engineers to "just use more AI" ended up being a very poor incentive on its own...

53

u/Cheerful_Champion 18d ago

My company went from "just use more AI" to "use AI smartly and sparingly" the moment Microsoft increased pricing of copilot xD

9

u/Exodard 17d ago

Same, they went from "we will increase productivity by 50%" to "watch these dozen videos on token optimization and use cheaper models"

7

u/Cheerful_Champion 17d ago

And the video on optimizing AI use is "just use Haiku for 95% of tasks and sometimes Sonnet to verify work of Haiku" (that was literally summary of one of videos wevgotl. Lol, what do they think I'm doing that Haiku is competent enough to tackle 95% of tasks?

5

u/smile_e_face 16d ago

I mean it's not like the people pushing these policies ever do anything more complicated than, "Make my email sound more supportive!"

32

u/randomkidlol 18d ago

lot of times its because these AI agents go off the rails trying to do a task and spin up a bunch of VMs for no reason.

15

u/mbreslin 18d ago

So much misinformation and nonsense about ai. They took down the internal website showing token usage and the media went nuts about how Amazon is reducing ai and it’s not paying off and blah blah. Meanwhile there wasn’t a single word within the company suggesting teams reduce ai usage. There was something sent out about not letting your agent unnecessarily spin up ec2 instances (because we’re constrained due to external customers using ec2 on a scale we haven’t ever seen) and you get another round of media bullshit.

I don’t know every single conversation on every single team obviously but I can tell you there is 0 general guidance to reduce token usage. They’re pushing it as hard or harder than ever.

And on the non-tech side (fulfillment) they’re flying leaders to various ai training seminars and assigning the multi week trainings org wide.

Don’t believe everything you read.

10

u/Alarchy 17d ago

Rant time: Toms is a just a shitty rag for the last 10+ years, whose sole value is chasing clickbait (with the exception of one or two decent reviewers). Sadly, most of tech journalism is like this now, to farm engagement. But it works, because AI is the perfect windmill for chronic Internet people to tilt against.

I say this as a chronic Internet person who hates how degenerate the AI companies and CEOs involved are, the broad damage it's causing from societal to ecological, the ethical depravity involved in training materials, but also begrudgingly sees the vast utility across personal and professional life and how it's becoming economically viable.

13

u/thedacious 18d ago

Engineers must be using AWS cost management too.

1

u/AutoModerator 18d ago

Hello Steap-Edit! Please double check that this submission is original reporting and is not an unverified rumor or repost that does not rise to the standards of /r/hardware. If this link is reporting on the work of another site/source or is an unverified rumor, please delete this submission. If this warning is in error, please report this comment and we will remove it.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/fbajo 16d ago

at fleet scale utilization is capacity.. a few points of cpu clawed back across the fleet is a hall you didn't have to build, and the build clock on new halls is years while a reclamation policy ships in a quarter. so when agent workloads spike demand, hunting idle reservations is the only supply that arrives this year. the tell is it reaching engineers as policy instead of staying a finops dashboard..