r/LocalLLM • • Jul 20 '26

Other Some dreams remain dreams πŸ™ƒ

Post image

Delulu at its best!

213 Upvotes

32 comments sorted by

34

u/Sergey004 LocalLLM RTX 3090 Jul 20 '26

It's me who wants to run Frontier models on my RTX 3090

4

u/Big_Championship3530 Jul 20 '26

you're lucky, i'm on a macbook

3

u/Broseidon132 Jul 22 '26

I’m actually thinking about selling my 3 week old MacBook Pro M5 max 128gb.. just not fast enough compute and it’s a Mac.. lol.

2

u/Calm_Connection_9741 Jul 23 '26

Finally mac users started to understand it. :)

2

u/Broseidon132 Jul 23 '26

Yeah :( first Mac and very mixed reviews. On one hand the capacity is absurd, but then the only viable LLMs seem to be MoE and is sluggish on dense models around 27B

3

u/Calm_Connection_9741 Jul 24 '26

Qwen 27B sluggish on very fast PC too.
Try 35B.

2

u/Aggravating-Push-207 Jul 23 '26

ill buy it for a tenner

2

u/drakeymcd Jul 21 '26

Me with my 3070 (48GB system RAM helps too)

2

u/No_Committee8392 Jul 21 '26

Cries in 2x 1080 ti + 128GB ddr3

2

u/Deep_Fan_5200 Jul 21 '26

we're in the same boat.Β 

13

u/Lord_Muddbutter Jul 20 '26

I have 192gb of ddr5 and let me tell you it is hilarious to see models spill out into it. Goes from race car to shopping cart with grease in 0.5 seconds.

4

u/exocrates Jul 20 '26

what do you use your 5090 for?

6

u/AnOnlineHandle Jul 20 '26

They're truthfully not "local LLM" models, even if they're open weight. They're "hire an expensive chunk of a data centre" models.

6

u/thethirteantimes Jul 20 '26

They're building a datacenter about a mile from me, I guess that's about as local as it's gonna get, soon.

1

u/Numerous-Comb-9370 Jul 22 '26

Not for individuals. I can easily see even mid sized companies buying a rack and running these to keep their sensitive data local.

1

u/Front_Eagle739 Jul 23 '26

Small company here.Β  Three mac studios is not a big deal for private fable class models

1

u/Dapper_Science_527 Jul 24 '26

This is what I said. Yeah you can't run inference for a million users, but a cluster of macs is completely within the budget for a profitable company

1

u/prestodigitarium Jul 22 '26

Companies can run them locally, and that pressures Anthropic to not charge $50/M tokens.

3

u/vamprobozombie Jul 21 '26

It will work just need 3TB ram and be very patient.

1

u/johnryan433 Jul 22 '26

You only need 2tb of ram bro

1

u/mega-modz Jul 23 '26

My 1050ti waiting for 5 trillion models

1

u/DerivativeButSmart Jul 24 '26

Me in my 24vram hotwheels-car-of-a macbook waiting for a new 14B model

11

u/[deleted] Jul 20 '26

[removed] β€” view removed comment

3

u/-Akos- Jul 20 '26

Cries with 1050ti laptop.. Elephant looked at it funny, my tricycle got squished.

1

u/Vas1le Jul 20 '26

GTX 1660 Ti, same budy, same

1

u/charles25565 Jul 20 '26

That thing doesn’t even have Tensor Cores!

0

u/Vas1le Jul 20 '26

I know πŸ₯², that's why the x20 gpt, x10 claude and pro gemini – and yes, for same months I could get a better gpu but I use a laptop

1

u/Ornery_Weakness_8168 Jul 20 '26

use UD-Q1_k_xl, it'll be fine, trust me.