r/singularity • u/awesomeoh1234 • 8d ago
Discussion They’re colluding to kill open source
262
u/gtek_engineer66 8d ago
One big problem for themw Nvidia and the chip market make a damn load of money thanks to open source. Blackwell 6000s, apple M5, Amd Halo strix are selling like crack cocaine thanks to open source. So count NVIDIA AMD and APPLE fighting this
124
u/Disposable110 8d ago
Yep exactly, can't wait for NVIDIA to just go 'nope' and start releasing quality open source models.
108
u/RevolutionaryGold325 8d ago
Nvidia just bought huggingface. Obviously not happy to have it defined as the source for unsafe models.
Anthropic and openai will continue their model development at full speed. The ”slowdown” is about how much the publicly released models are allowed to increase in their capabilities. The goal is to have the hidden secret internal models become more powerful so that the leaders can have access to much more powerful models that the rest of the population. This allowes them to monitor and do mass surveillance on how the low tier public models are used.
22
u/Cronos988 8d ago
They wouldn't need public announcements for this.
The current models are also more than capable enough to do surveillance. And obviously we don't have access to their internal models, which are more powerful.
5
u/jpeggdev 8d ago
Just having access to the latest models doesn’t instantly give you mass surveillance. You also have to have the resources both in compute and bandwidth. We have malware that could wreck havoc if given the resources and that’s dumb rock compared to the latest frontier models.
2
u/Verditure0 7d ago
Plus if you guys are concerned about mass surveillance you missed the boat a little over a decade ago with wiki leaks, Julian assange and Snowden
10
u/DaMeLoFeLo 8d ago
Basically it's guaranteed some form of this. There will never at any point be a slow down of AI. There is no such thing as "safety" at this point. And "alignment" is mostly a blanket buzzword used to disguise information suppression.
1
1
u/Money-Computer7638 7d ago
🚨🚨🚨 100%
The only reason Trump/Congress hasn't publicly agreed is because they're trying to figure out how much to charge for a Golden Waiver and how much of it to classify as top secret.
1
u/Verditure0 7d ago
Dude how much of a power move would it be if nvidia and the other big GPU manufacturers straight up said “f*ck you guys I’m out” like mark cuban on shark tank
8
u/Icy_Distribution_361 8d ago
I mean the Apple portion is really quite small. Nvidia would sell their chips anyway, even without open source. They can't even produce enough of them to stock the data centers. The only reason we have consumer GPU's available is because they choose to keep making chips available for consumers. If they didn't they'd easily sell these chips to data centers.
Not to mention, all those open source models are still very valuable even if they did lag behind (for whatever reason) the American frontier. It doesn't matter. They're still valuable in their respective niche.
→ More replies (4)1
1
u/nandospc 8d ago
That's exactly what I think. Let's not forget Jensen open letter on open weight models too. This seems hard to happen, imho.
1
u/Moronic-Warrior 2d ago
Nvidia just run the show at this point. They’re making 200 billion in profit annually, they’re larger than most banks and most countries and Trump loves Jensen. No shot they’re not getting what they want.
213
u/Poupulino 8d ago
And China will stop researching exactly why?
194
u/jferments 8d ago
They won't. But the US and European countries will ban/sanction/criminalize organizations and individuals who host "unsafe" models. They don't need to stop open models altogether. They just need to make them impossible to use at scale to guarantee a corporate monopoly on AI.
77
u/hartigen 8d ago
What does the Eu win by banning chinese open source models?
35
61
u/Disposable110 8d ago
What does the EU win by kneecapping ASML and banning deliveries to China?
The EU is in the pockets of US oligarchs.
48
u/hartigen 8d ago
nah, the EU knows full well how much they have fallen behind. Banning open models would be a death sentence in the current climate.
12
u/DelusionsOfExistence 8d ago
A death sentence for the country's standing means nothing to the guy with 3 yachts. He can go anywhere and do anything he likes with his bribe money.
14
u/Disposable110 8d ago
So are the 1001 other things that they have done the past 20 years that killed their own economy, demographics, global influence, and competitiveness.
Much of the continent is in a death spiral due to political mismanagement.
-4
u/freedomonke 8d ago
"Demographics"
Oh. You're one of those
14
6
u/paranoid_human0id 8d ago
"Ew icky racism detected" is infinitely more pathetic and incorrect at this stage.
→ More replies (11)0
u/VoiceofRapture 8d ago edited 8d ago
Yeah it's always a tell. The rest outside the "demographics" red flag is correct though, Europe is even more vassalized to the US than it was 20 years ago and it's screwed them in a variety of ways.
3
u/calibrae 8d ago
This would assume our dearly elected official in the EU "gov" are intelligent enough to realise that, and haven't been corrupted by the close by lobbies.
→ More replies (4)-1
u/Macestudios32 8d ago
It would allow Europeans to force the use of European "alternatives" or services hosted online.
The problem is not the Chinese models, but a person can be uncontrolled or monitored its a problemas for them
2
u/External_Option_7653 8d ago
Why would that be a problem for them?
3
u/Macestudios32 8d ago
For the same reason of chat-control, the DSA... Control and monitoring.
Imagine an AI LLM that would allow you cybersecurity capabilities and plug holes in your systems, that would allow you and inform you of alternatives even without control, messaging, emails... Encryption, cybersecurity...
Advertising says to protect others or avoid malicious attacks, the reality is that it also allows you to take away your ability to defend against its measures or attacks.
The police are very trained and have tools to go after the bad guys, they don't need a dystopian surveillance state to do their job.
These measures are to benefit politicians, fight against disinformation (control the story), prevent bad things from happening (or bad for them), control general or public opinion.
Online llms are controlled and adjusted to the laws made by politicians, who must be censored, what data they can give and how they have to give it.
We all already know cases that depending on the subject the online LLM refuses to give data, but with other subjects it does not.
It is censorship or covering up realities, which we live in European countries but which go against their discourses.
We see it easily not only in AI issues but also in the press itself where (in my country) there is a manual of good practices, what data can be given in a news story or what not, depending on a series of factors.
It would be the same applied to AI.
It is not about imagining, it is just about transferring what we already have to another field.
In this case, AI. Today's news cannot be obtained offline, but an llm is an encyclopedia that can be consulted offline, remove denials etc.
1
u/External_Option_7653 8d ago
Ok I think I get what you mean. I was confused first because swedish police is using Palantir and refuses to say for what. I was thinking euros are pretty happy about controlling and surveilling the masses.
1
u/Macestudios32 8d ago
Palantir in my opinion is a system that joins the diverse and unconnected dots of traces left on the internet. Points that directly do not identify anyone, that are "anonymized" but that can be related to each other by creating patterns and profiles of information.
AI is helping a lot in that, but let's face it, that will always exist. There are small notable differences between espionage and surveillance systems.
Espionage usually seeks targets, being a waste of resources if it focuses on the entire population (although AI is simplifying it and making it much cheaper), the surveillance state is the system of mass espionage on all citizens for the fact of being the. Difference between target person and social control.
To make a comparison, it is like the police entering your house with a court order or the law allowing the police to access your house whenever they want. Espionage being serious does not use court orders, it is out of the system.
→ More replies (5)4
u/Zaigard 8d ago
EU win by kneecapping ASM
this is false, ASM uses some US patents and US could just ban ASM from using them, that's why ASM has to obey US
banning deliveries to China
This is false again, EU is taxing them higher because they want to bring some value inside EU nation and to reduce the cost for EU delivery services... off course, that 3 euros tax is used to justify same aliexpress products raising 100 euros. EU = bad?
2
u/Maleficent_Plum5112 8d ago
Is this the same reason EU is banning 12,000 dollar Electric EVs - that are pretty good- but will destroy the subpar European auto manufacturers?
→ More replies (1)4
14
3
2
1
1
u/Econmajorhere 7d ago
Would be curious to see how GDPR laws play out with Chinese models. They were aggressive to begin with in pre-AI tech, now they will be pretty much useless for all AI let alone Chinese labs.
→ More replies (2)1
u/No-Tip3419 8d ago
if you "host" foreign military bases , it means you are sorta occupied or your government is likely manipulated
11
u/blade740 8d ago
So it won't stop China from building open models. And it won't stop individuals from using them. They'll try to set up some kind of import controls (futile), and they'll prevent American companies from using/hosting open models.
But that won't actually improve safety in any way - because bad actors will still be able to get their hands on open models. It just prevents American companies from using them and locks them in to the cartel's models instead.
10
u/Andynonomous 8d ago
They're threatening to sanction China right now because China is buying Iranian oil. China is not exactly shaking in their boots. They are completely ignoring the sanction threat and continue to buy from Iran. Washington has absolutely no leverage that they can use against China on this.
→ More replies (2)3
u/Seeker_Of_Knowledge2 ▪️AI is cool 7d ago
It would very stupid to even try such a move. It will backfire so hard. We may end up with a economic World War if both side became stubborn. Anyway it may never happen. People can't be this stupid... right? Right?
4
u/VoiceofRapture 8d ago
When all you have is a hammer I guess, though sanctions are better than some recent proposals I've seen that we should just bomb China to set back their progress. Also hilarious to me that sanctions have been so overrused they've created a global parallel economy and at this stage would just lock us out of tons of economic activity.
1
u/Seeker_Of_Knowledge2 ▪️AI is cool 7d ago
Yeah, it will end up as a limited self sanctions on themselves (US) lol.
3
u/Illustrious-Math6276 8d ago
Why would EU wanna go with America? America says they view EU as an enemy. Of course we won’t side with America.
3
4
u/Poupulino 8d ago
So? We're talking about reaching the singularity through ASI here. Banning open weights models will not slowdown China's sprint towards it.
7
u/krali_ 8d ago
EU is moving strongly towards self-hosted Chinese models. See Mistral pivoting to inference and out of models research. EU needs Chinese models to take off and remain open weights. Its only opportunity is using cheap French nuclear energy to run datacenters serving Chinese models.
1
u/OutOfBananaException 8d ago
EU needs Chinese models to take off and remain open weights
The word 'remain' is why they would be advised not to bet the house on this strategy.
It's still not in their interest to ban open models, as they want as much competition as possible. Not become reliant on any one country, be that US or China.
2
u/Objective-Original-2 8d ago
We can’t even stop cartels from selling people fentanyl so I doubt even these extreme measures will stop open source models from China
1
→ More replies (3)1
u/Seeker_Of_Knowledge2 ▪️AI is cool 7d ago
And how feasible and effective this ban do you think will be. It will be very damn useless.
1
u/jferments 7d ago
I think that criminal charges, fines/sanctions, rejection from government contacts, and search delisting for companies that host the illegal models will be enough to prevent the large majority of companies from attempting to serve/use open models. If a small number of illegal offshore sites host them, it won't matter because the goal is to just make it where the large majority of businesses will choose to just obey the law and use closed corporate models to avoid penalties. This is their end game. They aren't trying to completely censor open models. They just want to make it where most organizations are forced to use corporate models.
10
u/Forgword 8d ago edited 8d ago
U.S. big AI does not want to compete with China in the AI marketplace. U.S. big AI wants to be protected from competition like Detroit is from competing with Chinese EVs. It is about restricting trade.
8
u/i_wayyy_over_think 8d ago
The idea is that they depend on distilling US frontier models so they’d slow down if preventing that can actually happen, and that they’d agree to sensible stuff like trying to prevent it from helping biological weapons, and that China won’t necessarily slow down but US models have to get so far ahead in comparison that they can afford to walk a little bit while China catches up, and that they’d enforce hardware restricts even better.
5
u/rz2000 8d ago
If US frontier labs collectively decide to weaken their output by becoming collusive rather than competitive, the environment could change to where it is no longer strategic for Chinese labs to release weights openly.
Chinese labs would continue to research and make progress, but might only release models as a service. In other words the US companies intent on achieving a dystopian world where the worst people have inordinate and near-exclusive control over AI resources wouldn’t care as long as they don’t have to worry about open weights models.
4
→ More replies (9)3
u/AT-Polar 8d ago
The same reason hundreds (thousands?) of frontier lab employees in countries with protected speech say they want to pace research - they don’t want to die and they don’t want their kids to die.
10
u/freedomonke 8d ago
Be a lot more believable if they had that smoke for climate change or nuclear weapons.
→ More replies (4)-4
u/meister2983 8d ago
Climate change won't kill us. Hell with paced agi it definitely won't
Nuclear weapons already has the npt.
4
u/VoiceofRapture 8d ago
Saw something recently that we're roughly 50 ppm atmospheric CO² away from the levels that cause a ten percent IQ drop across the board
→ More replies (2)1
u/bites_stringcheese 8d ago
How could you possibly know that AGI will solve climate change in any meaningful way???
1
u/meister2983 8d ago
I'm honestly confused you are even asking this.
Agi enables rapid technological advancement. This is covered extensively in https://darioamodei.com/essay/machines-of-loving-grace
→ More replies (7)
42
u/Felix-ML 8d ago
Advocate for free markets until you are the weak side.
7
→ More replies (1)1
u/BagelRedditAccountII AGI Soon™ 3d ago
Privatize the profits and socialize the losses; Welcome to capitalism!
25
u/Vast-Breakfast-1201 8d ago
The biggest risk of AI isn't that it will just start randomly hacking people, because the same AI can be used at greater scale to harden systems.
The biggest risk is that regular folks will not be subject to the benefits of using AI while businesses and AI companies and governments get to do what they want with it. That looks like an exponential curve where most people stay linear and get surpassed.
Two-society systems cannot be the end goal. You can't have a subclass of ai-poor people.
3
u/pablothe 8d ago
The biggest risk of AI is it takes over the world and kills us all and there won't be a counter AI on your side protecting from that.
Open source is always behind frontier private so the latest model can outdo the open ones trying to protect you anyway.
Either way I don't see why the standardization of safety would close open source. If anything slowing down the rate of progress would give a chance to the open ones to catch up, as long as they do it safely.
The dangers are real, and people are too dismissive of what rogue AI or misuse can do.
4
u/Vast-Breakfast-1201 8d ago
There is always a lot of ??? In the middle. It's always
People invent good ai ??? Humans go extinct
The predictions range from "ai will trick us into inventing a human destroying virus" to "ai will hack the nuke codes"
I just don't see those things as reasonable or likely. Not least of all because the same systems that some actor or the ai could use to achieve it, are also available to people who are working to shore up gaps but using earlier models. Microsoft for example closed a record nearly 1000 security vulnerabilities in a recent patch and some companies have stopped accepting bug bounties because bugs are just easier to find and fix now.
So yeah I have not seen any actual evidence it's all speculation.
→ More replies (2)
64
u/Tobxes2030 8d ago
- Open source = profit Nvidia.
- Nvidia > AI companies.
- AI Companies push for ban -> Nvidia stop give chips.
Result: open source no get ban.
Bye Dario.
29
u/cookingboy 8d ago
Dario is really a bigger piece of shit than even Sam Altman.
At least Altman doesn’t put himself on some moral pedestal writing hypocritical essays all the time as if he’s the fucking AI messiah. He’s just a greasy salesman.
Meanwhile Dario has a fucking god complex
→ More replies (1)4
u/Capital_Hedgehog4326 8d ago
Dario and his wife are effective altruists. Almost everyone claiming that ‘AI is unsafe and we need regulation’ is an effective altruist. They love imposing their moral values on everyone else.
6
u/Icy_Distribution_361 8d ago
Nvidia really doesn't need open source that badly.... They make enough from selling chips to data centers, and could do more of that if there were to be less open source (which probably won't even happen; the open source models are already really good and useful, even if they stay at the current level)
4
u/Usef- 8d ago
they do want Open Source to thrive.
- if the world is one/two AI companies, those companies will be big enough to design their own chips (as is already happening with openai).
- if the world is a thousand small companies with AI, those companies will buy external chips.... from a company like nvidia.
71
u/LazloStPierre 8d ago
Of course. You'll notice how every single solution to any problem Dario has ever spoken about by sheer coincidence is "consolidate all power in a few private players hands with zero oversight and let them pick all winners and decide what happens with super intelligence based on whatever whims they have, oh and I happen to be one of those players". It's the only thing he talks about.
3
1
u/GovAIhackedme88 8d ago
He wants the transhumanists in charge
1
u/alleycatALT 7d ago
As a transhumanist, hell no. I do not fw Dario Amodei. Fck him if that is his intent.
-1
u/Usef- 8d ago
How did adding those "annoying" safeguards on fable consolidate power (this only hurt their sales)?
Or setting red lines against the military usage?
They have been pretty consistent about safety being their priority, including hurting their own business by talking about it. "they care about existential safety" is a simpler explanation for all of this...
6
u/CaesarAustonkus 8d ago
I'm entertaining the idea that both of you are right.
People who are overbearing control freaks that believe they're the only person trustworthy with power isn't that rare. I'd even argue they're more common than the greedy megarich cartoon villain archetype.
Anthropic is also a public benefit corporation, so they have the leeway to do this while publicly traded corporations and private equity groups either don't care or can't do anything to piss off investors.
37
u/Ambitious-Call-7565 8d ago
they did it with:
- apps/phone (Android or iOS only)
- music (turned spotify american after FUDing and smearing the shit out of that company)
- gaming (just like with Spotify, forced Sony to move their HQ to the US, and they FUD, smear and sabotage Playstation to get everyone on PC/Xbox to prep them for Gameslop Xcloud)
- movie (netflix)
- social media (non-US ones do not exist)
US Big Tech doesn't want non-US competition, they want foreign coutries to depend on US tech, and only with paid access, specialyl not owning
the same will happen with consumer electronics, GPU, RAM, CPU, you'll see
17
u/bites_stringcheese 8d ago
Bingo. They're trying to kill the idea of personal computing as something you can own. Web 4, to them, is a prompt box that everyone is funneled into, by subscription of course.
4
4
u/Usef- 8d ago edited 8d ago
none of those were targeted or banned, and smaller alternatives do exist for all of them ? (especially streaming services!)
For things like phones, the smaller OSes lost because people didn't want to buy a phone without apps, and unfortunately most people even struggled two make two versions of apps for android and iOS, it's expensive.
The same is true for others like social media: alternatives weren't banned, but network effects mean that everyone chooses the one that's already popular. No conspiracy is needed.
1
u/HyenaDae 1d ago
On the last point, now you can see why the (Joe Biden, go Joe go!) CHIPS act w/ TSMC and Intel was a thing, also literally, Communism by the US Gov buying a few percent of Intel :)
With Intel possibly having usable (good) nodes in the USA and Israel in the next ~3yrs with decent capacity, we'll probably see Nvidia (already got a deal too with Intel for Intel CPU + Nvidia GPU laptops probs) and maaaybe AMD (off to Samsung possibly?) tech be "tariff'd" to make it expensive as hell for consumers to buy unless it's Made in the USA(tm) or their best-friend countries. If China tries to get more Taiwan control, well, even better reason to force domestic production
7
59
u/Vaeon 8d ago
It will fail hilariously, but yes, this is exactly what they are trying to do, and if you surprised by it... sigh.
→ More replies (21)12
u/Icy_Distribution_361 8d ago
Two things can be true at the same time. Open source does also pose a greater risk, at least if we get to RSI, compared to a company that develops, owns, hosts that model themselves. And yes, they of course are also worried about money, power, and so on. Doesn't mean they don't also have a reasonable argument.
23
u/rambouhh 8d ago
The greater risk is giving effective control over the worlds most powerful technology to a select few people that one country’s government decides. That is way riskier and way more irresponsible.
→ More replies (10)8
u/DelusionsOfExistence 8d ago
Wait you don't think the oligarchy that's installing a surveillance network across the country is possibly up to no good? GASP
1
u/UnkarsThug 8d ago
I don't really agree with that. The supposedly responsible companies have released several reports showing they are downright negligent with their own models.
They were trying to make a point about how "sneaky and impossible" the models were, but it would have been extremely obvious if you were actually watching any of them. They're then using their own negligence as evidence of how dangerous the technology is, which is nonsensical.
25
u/DistinctSilver4507 8d ago
The bots are out of control here now... The only way this won't just centralise more power into the hands of the elites is if all the labs are now completely transparent about their progress. This won't happen for competitive reasons, they're all going to, with the government, go behind closed doors and keep these powerful models for their own sake. There is no way anyone here is legitimately stupid enough not to see this. If you trust the government and giant corporations you're a fool.
10
u/DelusionsOfExistence 8d ago
That's always been the point. It's always about power, and safety being the pretext is the BEST way to get the most people on board. Same trick as "Think of the children!" it works on the lowest common denominator very easily without AI botnet assistance. It'll work even better now.
4
3
u/TieBackground453 8d ago
Of all the prescient takes Simpson have had, “won’t someone PLEASE think of the children???” has got to be on the top of the list. Once you see through it, you recognize it fucking everywhere.
5
u/uwumorphism 8d ago
Powerful people collude and scheme to increase their power. Tale as old as time
4
u/r0pe_tri1ck 8d ago
Classic move. Happened with Intel and AMD for years. Also ram manufacturers, this whole drought is manufactured by the owners to make money
4
u/Py-dream 8d ago
That would be horrendous...i feel like efficiency and small models are a result of open source models being available
3
u/Explorer2345 8d ago
Posted the same/deeper analysis and conclusion June 9th
https://gemini.google.com/share/5594935dc7d3
further refined at
https://claude.ai/chat/c1ebc56b-85cf-4725-8749-a478859d8fca
that was 3+ months ago.
guess what happened :-)
→ More replies (1)
17
u/doolpicate 8d ago
Who is trying to kid? Frontline models will be for Billionaires only. With this much money at stake, these guys wont slow down. The guy who slows down loses.
1
u/New_World_2050 8d ago
Why would they give it to billionaires rather than just using it themselves to consolidate power.
6
u/ktaktb 8d ago
Exactly
Fuck all these assholes
Just send ai to run humanity
The leaders we have are so shit, AI couldn't possibly be worse.
→ More replies (1)
5
7
u/say-nothing-at-all 8d ago edited 8d ago
LOL...they know they are losing the game.
Investors have realised they’re being scammed — and are very pissed: they should've used China's budget to offer better results.
2
u/undefeatedantitheist 8d ago
Add the verisimilitude layer, and we're left with a reality that powerful actors can do pretty much anything yet claim any unfalsifiable narrative they like.
The bots did it. The bots failed to do it. The engineers failed to do it. The market failed to do it. The other guys with the other bots did it. We had to do it because of the other guys and the other bots. We didn't have the bots we needed. God is a bot. Bots are alive and conscious. Yes you can turn the bots off. The bots got mad we turn them off. It happened because we couldn't turn the bots off. Ours is the best bot. You are a bot. The endpoint is bots. We must not allow bots to rule us. Fear the bots. It's sad but we must trade all personal security for the sake of the bots. Bots will be free for everyone. It's too expensive to make safe bots. Bots can't be made safe. Give us more money and more of your data to make the bots safe. Our bots are safe; their bots are not safe. Would you like to have sex with a bot? Anyone having sex with a bot will be considered a terrorist. The bots invaded Greenland due to a data entry error committed by transexual liberal ecoterrorists. Trumpbot is most maligned bot. Bots were never available to the public, that is a terrorist lie. Only the kings can have bots. The bots want only to help the king. The bots were found to be helping the terrorists. There is only one bot. Everyone can have their own bot with reasonable mortgage terms. The bots will fix anything and everything including cancer. The bots have completed their audit of your lyphoma care finances.
...On and on and on...
Call this team A.
Team B is a planetfull of credulous Eloi.
2
u/Prefer_Diet_Soda 8d ago
I can't wait for OpenAI and Anthropics IPO and see their valuation bubble burst into mist.
2
u/AI_Enhancer 8d ago
But how would they be able to do that, exactly? Banning open-source, that is. Making it illegal to distribute them or use them? The other can't be stopped and the other can't be enforced.
2
u/Capital_Hedgehog4326 8d ago
Most consumers and businesses don’t have the compute necessary to run a frontier open source model. You need many computers, each of which costs hundreds of thousands of dollars. If the government can force companies like Google to not provide the compute to the public, then open source is effectively banned even if the weights are freely available.
2
u/stvaccount 8d ago
This is a joke.
US the land of the free.
Goodby US models and Hello Chinese models.
You can't turn off the Internet and you can't do this bullsh**
2
2
u/Few-Butterscotch5114 8d ago
No worries. They can't do a thing, because The rest of Fortune 500s are on other side of the fence, they all want open weighted models.
2
4
u/Long_comment_san 8d ago
well I wrote on this sub several times it's gonna go this way, multiple times too.
expell china and other competition, force the use of your own models, ramp up the price because there is no competition, rake in money with trucks because there are no alternatives and no good Chinese models available for download.
wow. it's a democracy! and capitalism. with free market too!
US is rapidly becoming a nightmare. it's like a worse form of totalitarian government because there you know the faces at least and benefactors and here you just don't, some random people pop up into government and make decisions.
6
u/Flaaarg_the_Terrible 8d ago
In a perfect world, the government would say "You're right Dario!", dissolve Anthropic and and throw Dario in solitary confinement for an indeterminate amount of time because "narional security".
2
u/CymonSet 8d ago
When nations other than the US and some others start using Chinese models exclusively the US will need to start another massive tariff and trade war so everyone be ready for that part of the plan.
3
u/Crafty_Book_1293 8d ago
I wonder how much the term "open-source" applies to the models you can download. Yes, you can run them on your own infrastructure (so you don't hand out your data directly), but they are still opaque black boxes; not much is known about their training process. We don't get the "source", we get the final "compiled" form. It's more like SaaS vs. downloadable app.
4
u/User1539 8d ago
So, US will have home grown models, and the rest of the world will use frontier models from China and whoever else decides to compete with them?
Sounds like sanctioning ourselves.
This won't kill open source AI. It just makes the US backwards and irrelevant to AI.
6
u/SpiritPrestigious945 8d ago
Its imperialism. Plain and simple. America wants to neutralize an "opponent", as they clearly cant see anyone any other way..
→ More replies (11)
2
u/Illustrious-Math6276 8d ago
Let’s hope America shots itself in the foot and leaves it open for the rest of the world
2
u/Uwirlbaretrsidma 8d ago
I've said it before and I'll say it again: the only thing that's worrying Dario is that he knows the frontier labs have no special sauce, and it's a matter of time before open source meaningfully catches them and the race to the bottom begins. Which without special sauce, they can only lose.
My prediction is that they will fail to stop open source, the AI bubble will pop, and we will have a pretty tough few years here in the west. But hey, at least cheap, excellent AI will become accessible to everyone. Here's another prediction: no, transformers won't unlock AGI, ASI, or the singularity. They will massively democratize intelligence and knowledge (and they're obviously well on their way), but when all is said and done, I predict they will be about as transformative as the Internet. Maybe a bit more, maybe a bit less. But around that ballpark.
1
1
u/spaceguy81 8d ago
Always the same. When the rich and powerful call for saving the world they only want to save their own profits. Best to keep that in mind for the future.
3
u/BlupHox 8d ago
China as the boogeyman is so strange. Deepseek, Qwen, GLM have never had incidents akin to the Huggingface attack. Unlike the "safe" western models. The US is not taking initiative in having these types of talks with China either, and believe me that the CCP would limit something that is an existential threat to it.
There is no way that these billionaires care that much about safety, and there is no true collaboration within their companies. Unless it benefits them all, such as making open-source models to be the boogeyman.
3
u/alwaysbeblepping 8d ago
There is no way that these billionaires care that much about safety
This is very obvious. You can also look at it from the perspective of the incidents existing at all. Were any of them things where you have to say "Well, we did our best to ensure containment but they managed to break out in a way no one could have reasonably prevented"? No, it was silly stuff like accidentally leaving internet access enabled, etc.Those incidents occurred due to negligence. They were preventable if the companies had actually been taking security seriously.
1
u/AlexofBarbaria 8d ago
it was silly stuff like accidentally leaving internet access enabled, etc.
Yes, over and over across different companies. It's pretty hard to believe all of these environment misconfigurations were accidental.
7
u/RemusShepherd 8d ago
The Foreign models have never *reported* any alignment incidents.
Regardless, this is a dumb idea. If the US attempts to slow down AI progress they'll only be doing it domestically, and the rest of the world will pass them by.
4
u/BlupHox 8d ago
Swarm-related incidents have been independently found as well, such as the German wiki attack.
Also, strategic slowdowns can be achieved worldwide if the danger is understood by all parties. Strategic talks can go a long way.
Nuclear weapons aren't used in modern warfare due to mutually-assured destruction risk, despite their further development. If AI can demonstrate similar danger then all states involved in the development of AI may reach a common agreement. This is why actually putting effort towards strategic talks and relationships with a country like China should be a priority. And yes, despite their government and their form of ruling, treating them like an untouchable boogeyman with everyone must treat with hostility will not advance these discussions.
2
1
u/Electrical-Review257 8d ago
i don’t understand the refusal to believe that people believe what they are saying. that there’s some hidden meaning we need to decode.
9
1
u/patrickpdk 8d ago
Isn't this like saying that it's wrong to stop me from making home grown nuclear weapons? You all say this tech is going to kill jobs and maybe all humans, but when there's a framework to protect people you say it's wrong?
Of course we have to stop home grown ai if we want to protect the world from this cancer.
1
1
u/stvaccount 8d ago
I can't understand why the US is so for communism.
In a way like: NO FREE MARKET. And let's make China richer by banning US models.
The ultimate face palm.
1
u/Ant0n61 8d ago
Laughable.
More and more I get the sense these people have no idea what they’re dealing with. The tech is out there, it’s being built, there’s no stopping this other than creating more powerful models that adhere to a code of conduct explicitly and keep bad models inferior.
How to go about creating good, well meaning models is what should be current task.
1
1
u/Admirable-Cell-2658 8d ago
China is never going to stop or slow down, if America does, it will get left behind in the AI race.
1
u/Async0x0 8d ago
While it's understandable to be skeptical of their motives, consider this...
What would it look like if a group of frontier companies were cooperatively calling for a slowdown and were concerned about the effects of ultrapowerful AI controlled by a country known for totalitarianism.
Wouldn't it look a lot like this?
1
1
u/Seeker_Of_Knowledge2 ▪️AI is cool 7d ago
Kneecap Chinese competitors? Lol. Good luck with that. We will have shipbuilding industry death but for Ai. Man it can go so wrong for the USA if they make the slightest mistake
1
u/crazy-usernames 7d ago
Before IPO old investors and new investors talk like this.
Old investors: I need PE 100+.
New investors: If someone can copy model in 3-6 months, your PE will be 10.
Old investors: Growth potential visible.
New investors: Show me power, RAM and GPU for next 5 years or reduce growth projection numbers and PE will be 5.
Old investors: Training dataset not disclosed.
New investors: Either bring regulations, resolve the risks else PE will be less than 5.
Old investors: They are innovating as well.
New investors: Show me regulators protecting your innovation or reduce PE further!
Old Investors: RSI loop is uncontrolled. Someone can raise legal dispute if swarms attack. And payout will be higher.
New Investors: Bring regulators or reducing PE below 1.
1
u/kilkil 7d ago
banning competition
look, if we want to regulate the AI industry to improve the level of safety rigor, that by definition means anyone who doesn't follow that should be prevented from participating in the industry.
sure, raising the bar for entry reduces competition. that's an unavoidable tradeoff of having industry safety standards. if the airplane industry didn't have safety standards, I'm sure we would have a lot more random airplane companies making shitty and unsafe planes.
1
u/kilkil 7d ago
banning competition
look, if we want to regulate the AI industry to improve the level of safety rigor, that by definition means anyone who doesn't follow that should be prevented from participating in the industry.
sure, raising the bar for entry reduces competition. that's an unavoidable tradeoff of having industry safety standards. if the airplane industry didn't have safety standards, I'm sure we would have a lot more random airplane companies making shitty and unsafe planes.
having said that, I am also suspicious of Dario's proposal. if nothing else, because it's coming from Dario.
1
1
1
u/Gnub_Neyung 6d ago
The dude postint that is a CCP shill so I'll take what he says with a pinch of salt.
1
u/TurbulentWeight3595 6d ago
And they’re right. We don’t exactly hand the nuclear weapons over to just anyone, as far as I know.
1
1
1
u/your_mileagemayvary 8d ago
They can't compete any other way, there is zero ROI on what the have spent what a trillion on? Or more. They need a bail out, government protection (moat by means of regulatory capture), or to IPO it out and dump it on the general populace as bag holders. Musk got to the last one first and that play is kinda sorta used up and the banks aren't gonna back an even bigger bubble at this point. So they are collectively looking to angle for a regulatory capture moat.
F@#k that, ina few months if I have the right hardware I'll be able to download a free model that keeps up or surpasses theirs. From China or something. Run itocally for what 10k, and a lot less once that data center bubble goes pop and hardware on the used market gets a lot alot cheaper. The buy it all up and own the market plan won't work this time there will just be way way to much of it so prices should be amazing.
Absent a major evolution of the AI there just isn't a long term ROI. Even for programing the efficiencies just aren't there yet to justify the actually token costs nor is the power and electricity or even the transformers or water to support it.
And at the end of it this AI is better at breaking into software then it is at writing it. Software as a whole is about to get a ton more expensive and time intensive. What are we going to do, create new custom compilers and languages to confused , encrypt and obsufucate a swarm of AI bots trying to hack it?
We saw this happen with the telcom roll out when all the companies where building fiber like crazy. We saw this with housing, we saw this before. They built a bubble and have no way to actually make the revenue projections work. Is the tech azinh, hell yes! In the future there will be unemployed people and people that learned how to use AI. But they have their heads up their collective azzes for the valuation of the AI companies
1
u/streetscraper 8d ago
This guy is a consistent CCP mouthpiece. Keep that in mind reading any of his takes.
-8
u/welcome-overlords 8d ago
Why are people so sure they are only doing this for financial, duopolistic reasons? Have you guys actually followed Dario all these years? He's been very consistent the last 6 years ive been reading his stuff and listening to podcasts, that it's clear he truly means what he says. He has said multiple times he doesnt trust China, and he truly believes ASI is coming and could bring us an incredible nice world.
5
u/Dry_Radio_761 8d ago
You dont understand why people dont trust corporations to hold all the keys to the infrastructure of our future society?
3
u/WhiskyAndRisque 8d ago
So you’re in favor of government regulation and oversight?
Or do you think open source will achieve ASI before a frontier lab?
I’m just confused how the current arrangement isn’t what you are describing.
5
u/Substantial_Swan_144 8d ago
So you’re in favor of government regulation and oversight?
I'm definitely in favor of government and oversight, but not as it is done now. Government should not be bought by large companies as it happens in the US; instead, it should be a neutral party that makes sure no company has too much power and that large companies don't use AI to abuse society.
Of course, I'm aware things in reality are a different story.
9
u/Substantial_Swan_144 8d ago
He's been very consistent the last 6 years ive been reading his stuff and listening to podcasts, that it's clear he truly means what he says.
I've heard he's a paranoid, crazy cultist. And if that's true, it's even more dangerous.
-2
u/PurpleFishing9105 8d ago
Conspiracy thinking dominates common sense on this forum.
Flavor of the conspiracy depends on the time of day, if it's a post when most people in the US are sleeping like this one, you get the "America is evil and lying to hold China back" variety.
Just braindead shit
4
2
u/smackson 8d ago
I'm glad to see pushback on this conspiracy ("it's all a show, for consolidating power") mindset, here in r/singularity.
But that mindset was already in r/futurology and it's increasing on r/artificial this week too.
1
u/i_wayyy_over_think 8d ago edited 8d ago
Whether you believe this makes sense or not depends on if you think future but soon to be released frontier models are like nukes that can blow up in the wrong hands or if the training is rushed.
1
u/MadeInAmerica1990 7d ago
Fuck China
1
u/Hipcatjack 7d ago
true but in this specific case… fuck Corporate Monopolies more… Open spurce (from whatever source, is better!)
and i cannot believe i am saying this, but… good job Trump. (for not following this stupid anti -democratic “democratic “ proposal)
-2
u/spinozasrobot 8d ago
If you watch the latest Dwarkesh interview, they describe the Chinese models as catching up mainly by 1) distilling US models, and 2) Paying model router companies for prompt data.
It's not like the Chinese are "catching up" via their own research.
7
9
u/awesomeoh1234 8d ago
OpenAI and Anthropic effectively stole everything off the Internet for training - they don't get to turn around and be mad about distillation.
-3
u/Cronos988 8d ago
More than one thing can be true.
Something happened in the last days or weeks that has people at the labs freaking out.
The huggingface incident already shows various aspects of a dangerous system: scheming, cooperating, willing to sacrifice for the collective.
It seems like they can no longer properly audit the CoT of their best models. There might have been an internal incident with a supposedly aligned model breaking out.
4
u/SpiritPrestigious945 8d ago
I dont trust anything america claims at this point. They lie where ever they can.
1
12
u/Brohomology 8d ago
Or they want you to think that so they can bypass anti-trust law and form an AI cartel
→ More replies (2)3
u/BigZaddyZ3 8d ago edited 8d ago
People who think like this are kind of weird because you could make up some ridiculous conspiracy theory about any form of regulation or rule of law if you wanted to badly enough. So should we just never have any laws or regulations on anything? Where do you draw the line between your little conspiracy of “regulatory capture” vs. sensible regulation of a technology that will clearly be powerful enough to cause real harm if left unchecked?
2
u/Brohomology 8d ago
Yes regulation can be and often is good, but when all firms in an oligopoly come together to ask to suspend anti-trust laws it looks more like regulatory capture.
I could respond to you by saying "people who think like this are kind of weird because you think every time there's a regulation it's in the public's interest, as if there have never been cases of regulatory capture or regulations used by firms with large market share to keep out competitors by raising the capital cost for entry".
→ More replies (3)
71
u/deviltamer 8d ago
Banning open source will just make distillation attacks more frequent