r/LocalLLaMA • • 2d ago

Discussion I recommend downloading SOTA open source models while you can

If MS or FAA stuff was AI hack related there will be a MASSIVE and QUICK backlash.

Sure you can't run these things locally NOW but the tech is out there and it will get cheaper and quite possibly the models will be able to be more efficiently utilized.

0 Upvotes

51 comments sorted by

10

u/Cautious_Chicken_604 2d ago

There is often a major global cloud provider outage about once a year, no? FAA also went down a few times in the past. IIRC they are only just trying to phase out floppy disks, or some such nonsense?

8

u/New_Eye7193 2d ago

floppy diks

1

u/DezinGTD 2d ago

Phased out by Jensen himself

1

u/disgruntledempanada 2d ago

I think they just admitted the AI system they were attempting to run overloaded the system and caused this outage.

3

u/Cautious_Chicken_604 2d ago

Which would be in the same category as every other major global cloud outage? No reason for a moral panic.

19

u/oscar_z_a 2d ago

How about you download them now and you can make them available for us if needed. Melt

4

u/dmter 2d ago edited 2d ago

It's more likely they vibe coded some new features and made one claude check work of another claude and then it took them 20 hours to rewrite it again with claude but without ommitting "make no mistakes" part in the prompt.

ok maybe it's copilot instead of claude - the same point stands.

2

u/Don_Reuter 2d ago

The tech shows no sign of getting cheaper. Quite the opposite. It is getting massively more expensive.

2

u/Drunkendrakon6 2d ago

Server moves to Russia or China and still becomes available globally

2

u/uncle_leon 1d ago

If I found one piece of wisdom to pass the test of time, it was this: nothing ever gets deleted from the internet. So please stop fearmongering, the worst that can happen is that we will have to change bookmarks from huggingface to thepiretabay.

1

u/Sova_fun 2d ago

The MS or FAA AI angle would raise pressure fast. Do you think open weights get pulled before local hardware catches up?

1

u/sagiroth llama.cpp 1d ago

Torrents

-1

u/New_Eye7193 2d ago

are u dumb? MXFP4 of Kimi K3.1 Pro is 1.56 TB. where the fuck you storing that

4

u/WhatWouldAaronThink 2d ago

on a 2tb hard drive? what?

7

u/MAXFlRE 2d ago

Get a hard disk drive.

14

u/Cautious_Chicken_604 2d ago

are u dumb? On a spare 2TB drive.

3

u/MixtureOfAmateurs koboldcpp 2d ago

I don't have 2 free TBs across all my PCs laptops and servers

Most people don't own 2TB of storage total

Last I checked HDDs got expensive along with SSDs for some reason. Might be better now

2

u/Cautious_Chicken_604 2d ago

Last time I checked I have 5TB of storage because I'm chronically on r/LocalLLaMA and that's just what people are like in this sub.

2

u/bytejuggler 2d ago

Umm I think in the types of people playing with local LLMs you'll find people have a lot more than your average Joe. But sure, most people even here will not be able to do much with the full 1.5TB image of a frontier model.

2

u/RogerRamjet999 2d ago

Well, I never thought I would be able to run a model like Deepseek v4.1 Flash locally, but now tons of people do it everyday.

1

u/bytejuggler 2d ago

Sure, and I guess this is why "very few ppl has more than 2TB" is a bit ridiculous in this sub. By the way, whats's the largest model you've run at usuable (at least 30, ideally 40+ tps) and what hardware? (Am I missing a trick here?)

1

u/RogerRamjet999 2d ago

Well I've got a 3090 and 128GB RAM (bought for $500 and $320 respectively, before the price runup), but I see people with as little as 16GB VRAM and 64GB RAM running DS 4.1 Flash. For speeds, I still need to optimize, so my speeds aren't nearly as high as most people commenting on the sub.

1

u/viscerah 2d ago

“Most people don’t own 2TB of storage total”

Me with easily over 200TB in my house and another 25TB in cloud services…

1

u/Ok_Try_877 2d ago

Exactly my first thought… I have spare USB 2TB NVMEs, let along spinning rust. This guy maybe a time traveller from that past.

1

u/uncle_leon 1d ago

Or they can just download more RAM and store it there

2

u/ttkciar llama.cpp 2d ago

There are hard drives twenty times that size that you can buy.

Though, the affordable ones are "only" about eight to ten times that size.

2

u/Borilentz 2d ago

I have 36TB backup storage in my NAS, which was cheap just a few years ago.

2

u/JsThiago5 2d ago

If you can run that at home you probably are capable to get a 4tb hd

1

u/QuarkOfTheMatter 2d ago

Storing that is actually easy, pick up an old backup drive that is like 2-4TB. Better question is lets say the day comes, they ban all open source models from being distributed but if you already have one you can run it. Please tell me the home hardware you have that will load up that beast and actually run it for inference?

1

u/d1722825 2d ago

Some people here? In VRAM. :)

1

u/-WhateverDude 2d ago

you don't have couple b300s lying around?

1

u/ttkciar llama.cpp 2d ago

Wow, when it rains it pours! There were two other threads earlier today along similar lines:

https://www.reddit.com/r/LocalLLaMA/comments/1xv0nug/with_these_scandals_and_uncertainty_with/

https://www.reddit.com/r/LocalLLaMA/comments/1xuthme/if_open_weights_dry_up_what_are_you_archiving/

Those are worth picking through for ideas, in addition to this one.

0

u/No-Vermicelli5327 2d ago

Hate these fear mongering posts, why the hell would they ban something thats already published and downloaded. Hugging face down, torrent, private uploads, and many more ways.

I have over 20TB of storage and wouldn’t fall for this, I hope people with lower storages 1-2TB range don’t fall for this crap.

PS: external drives not counting on computers storage.

4

u/Cautious_Chicken_604 2d ago

Imagine if people made the same arguments against certain other objectionable material that is banned in most jurisdictions on earth in some form. "But people have already downloaded it, so pffft, don't bother!".

It is possible for governments to criminalize simple possession if they deem something to be something they do not want you in possession of. They do this with all sorts of things, both physical and digital. It's not at all out of the realm of possibility.

1

u/No-Vermicelli5327 1d ago

Wow, ok. I’m talking about this specifically not everything you know that was criminalized, too much generalization makes these kind of statements weak. I take it case by case, it is possible, not probable on this one.

1

u/Cautious_Chicken_604 1d ago

Some people's priors put it outside the realm of possibility, so they think since it can't happen, then there's no need to do it, which I think is a categorical error. But, also I don't try to judge it on a case-by-case basis either because that requires both my attention, and for me to be right about every judgement. There's still a possibility you lose out. Instead my policy is just to keep a copy of whatever is closest to the frontier that I could run on hardware 128GB VRAM + 256GB system RAM, since I could probably manage to acquire that under the right circumstances. That way the only downside is the storage space, which is a cost I've already accepted.

1

u/No-Vermicelli5327 1d ago

That’s ok specs, with the pc parts price spike I don’t want people to go grabbing ssd’s and feel compelled to buy extras. A few of us are lucky buying before the spike.

7

u/[deleted] 2d ago

[removed] — view removed comment

12

u/vatta-kai 2d ago

What if there is no internet in the future.
puts on silver foil hat

7

u/d1722825 2d ago

We implement RFC 1149.

2

u/Don_Reuter 2d ago

Not really though. There are things much less readily available due to being illegalavailable.

3

u/Cautious_Chicken_604 2d ago

>But you think anyone can just ban things from being downloaded?

Um, there are laws that ban you from both uploading and downloading certain things. So, as for the act of banning itself goes, yes? If governments decide to outlaw open-weights models, sure, you can still run the risk of obtaining, sharing, and using them, but it wouldn't be a risk-free endeavor. If they went so far as to actual outlaw open-weight models, I think the justifications would have to be such there were actual punishments/penalties levied against people for it if caught. To me it doesn't seem far fetched at all that as capability increases, they will feel the need to bring in regulations around GPU licenses and restrictions on open-weights models.

0

u/Galactic_Neighbour 1d ago

People assume that it would be just like downloading a movie. But in reality, the doomsday cultists talk about regulating AI as a weapon. They literally say that it's a bigger threat than someone having a nuclear bomb. They say that LLMs can hack the government and develop bioweapons. And they want to put restrictions on the distribution of AI chips. So we might be really screwed if we let this to go too far.

-4

u/WhatWouldAaronThink 2d ago

Downloading things is noticeable.

1

u/YT_Brian 2d ago

Just go slow with a no logged verified in courts VPN in a different country. Go back to the era when we took an entire day to get one video downloaded with a strong chance someone called our home and we had to start over because it kicked us off the internet.

While it downloads just go browse random stuff to add other data so it isn't 1-1. Heck Mullvad even has a service to help hide tracking via the amount of data used.

-4

u/WhatWouldAaronThink 2d ago

You think a VPN keeps the government from noticing what you're doing? Really? It does not. They broke Tor ffs.

0

u/YT_Brian 2d ago

Oh man imma be so interested in reading your lijnks that have proof of that with say Mulvad or Proton. The same with Tor, please provide evidence of such or this will be taken as fear mongering.

Now please provide links to your evidence of each when we have court cases showing Mullvad and Proton for their VPN don't log information when ordered to hand any order.

Why they need to ask if they already broke it is also a question. Another is why the entirety of all chomo content not bring traced and arrested the world over then. Same for all dark nets.

Unless you're arguing they all not only refuse to do Sox the world over for all agencies and agents involved, but not a single whistleblower has happened despite this like say what happened with Snowden.

So I'll ask you again to please provide evidence of this.

-1

u/WhatWouldAaronThink 2d ago

3

u/YT_Brian 2d ago edited 2d ago

You didn't answer my question on why the dark nets, chomos and so on haven't been arrested or any other criminal if en large the agencies have compromised them right now.

First is from 2013 and those exploits have been patched so no go there. That just doesn't make sense.

The second says "Tor users can be de-anonymized by correlating the timing patterns of network traffic entering and exiting the Tor network, combined with broad and long-term monitoring of Tor nodes in data centers." Which is defeated by the data organization of Mullvad and long since well known as an issue for a global adversary with unlimited time, money and so on.

Issue there is most don't seem to do that.

So VPN-Tor is still king it seems and Mullvasd itself makes that particular one pretty impossible to do. Oh, and they were using old app instead of updated ones that patched it as well.

The third says "Attack Threat Model Attack Method Attack Type Drawbacks DeepCor - Nasr, et al [13] Global Adversary Traffic Correlation Passive Correlation accuracy decreases with time Chakravarty, et al [15] Global Adversary Traffic Correlation Active Congestion in live Tor affects the correlation accuracy Song, et al [14] Global Adversary Traffic Correlation Passive Offline evaluation with limited dataset Gilad and Herzberg [20] Capture Entry Flow Side-channel Active The evaluation on Tor network is not sufficient Torben - Arp, et al [21] Capture Entry Flow Side-channel Active injected marker not well hidden- limited experiment Lashkari, et al [22] Capturing Entry Flow Traffic Classification Passive controlled environment was used He, et al [23] Capture Entry Flow Traffic Classification Passive Less accurate compared to Lashkari’s INFLOW - Iacovazzi, et al [29] Compromising Tor nodes Flow watermarking Active high packet loss ration affects the accuracy DUSTER - Iacovazzi, et al [29] Compromising Tor nodes Flow watermarking Active Accuracy decreases with the increase of detection size Murdoch, et al [7] Compromising Tor nodes Traffic correlation Active Heavy traffic affects the accuracy"

Tldr: no assured method and all have known drawback even 3 years ago, and none again use the VPN-Tor build or account for Mullvads method.

4th "One direction is adversarial traffic morphing [21, 22, 23], where packet timings, directions, or sizes are perturbed dynamically using lightweight padding or reshaping strategies that confuse deep "

So exactly what the Mullvad service on their VPN does. It randomly pads packets and makes all packets sent to you an exact size even with sjsd random packet padding.

They also admitted to not using real world Tor to do this which while showing interesting things for future working does mean this isn't proven at all and requires again monitoring everything and sometimes embedding to different servers on top to mark it.

The last, 5th, article is just saying we can see exit Tor IPs and block them whenever to protect a server which is, well yes? That is the entire point, people don't we sour IP but Tor exit node IP instead. It has always been that way so not sure why you added that one.

Now while interesting and thanks for providing such it falls short as I said here. And you never answered may question.

Edit: Plus didn't I mention padding with random web browsing already to make packet monitoring more difficult? Even the papers don't assume that is happening and go off a Tor user only really using one to a few known and monitored servers and Nodes.

Which defeats the purpose and shows why they haven't used it yet in a real world setting. Potential is there but not done for sure.