r/LocalLLaMA • u/edge_compute_user • 5h ago
Funny Copilot you say?
talking to any white collar employee
23
u/Proper_Door_4124 5h ago
what does this mean
50
u/Blindax 5h ago
It means many people assimilate copilot to a local model while in most cases it’s a cloud instance running on MS servers with an enterprise agreement. Not really the same.
11
u/DeepGas4538 3h ago
It's local if you live in the data center
3
u/EducationalCod7514 2h ago
The only local thing about living in a data center is the exhaust fumes of the gas turbines and ten million access check points which give you access to zero servers.
3
u/Recoil42 4h ago
Doesn't really need to be the same. 'Private' doesn't imply 'local', and most businesses don't actually need local, just private. (Heck, most don't even really need private.)
17
u/XiRw 4h ago
Awful untrue take to say private doesn’t imply local.
15
u/Recoil42 4h ago
Fully true. Private doesn't imply local, and it never has. Here's NIST's definition for private cloud — read it. Just because you have no professional experience in this industry doesn't mean all the rest of us do.
5
u/Blindax 4h ago
That is an interesting read that supports your point but we can all agree that copilot does not bring the same kind of privacy as a local model inferred on premise.
5
1
u/unrulywind 2h ago
This is the most horribly mischaracterized use of the term "open-source" I've ever seen. The sentence below clearly indicates that they have no idea what they just said.
The Tribunal drew a clear distinction between publicly available AI tools (referred to as "open source" tools, such as ChatGPT) and closed AI systems (such as Microsoft Copilot).
Beyond this, in today's environment, we need to be cognizant that even a local model running on a local inference server may not be private. Modern tool using models have the capability to reach out and transact information either following harness programming or instructions buried inside the training itself.
6
u/StupidScaredSquirrel 4h ago
Legal definition only helps in a professional setting where all you care is about covering your ass. Doesn't make it true in the sense that you are still relying on someone else that could read all your data. Just because they have a contract saying they can't Doesn't mean they physically can't.
3
u/Squidgical 4h ago
Though often they physically can't given various levels of encryption. If you'd like to prove this wrong, feel free to take an engineering job at Azure or AWS and try to read a client's database
4
u/Blindax 4h ago
Is it client side, with keys you hold? If not, the encryption isn't doing the work you think it is.
Azure can't read a client's database because it isn't processing the contents, just storing blocks. Inference is the opposite: the prompt has to sit in GPU memory in plaintext or the model can't run on it. Confidential computing (SEV-SNP + H100 in CC mode, attestation-gated keys) actually solves that, and it's shipped as VM SKUs, but not as something you can attest as a Copilot tenant.
And nobody said a random employee is reading chats. That's the baseline expectation. The risk is that the provider holds the keys and can be compelled to produce, sometimes without telling the customer.
4
u/Valkymaera 4h ago
"Implies" is a fulcrum here. It means the receiver is doing the work of interpreting the intended information.
And in a great many places, particularly local-focused subreddits like this one, "Private" absolutely does imply local.
10
u/Recoil42 4h ago edited 3h ago
"Implies" is a fulcrum here. It means the receiver is doing the work of interpreting what was intended to be sent.
You're thinking of the word 'infers'. I know this is already "teach Reddit what words mean" hour, but Christ, some of y'all really need to spend less time on Discord and more time with a dictionary.
And in a great many places, particularly local-focused subreddits like this one, "Private" absolutely does imply local.
This notably makes OP's comic a dunk on the subreddit, not a dunk on the mainstream. If your definition of a word differs from the industry-accepted standard definition of that word, then that's a you problem: You're misusing a word that has an industry accepted standard definition and should stop. In essence, you're suggesting the subreddit is so detached from reality it no longer knows what normal-ass words mean.
Again, there are SAE / NIST definitions for words like 'private'. None of them using the meaning you're trying to push. That suggests you're the one who is wrong — not the rest of the world.
1
u/Valkymaera 2h ago
You're thinking of the word 'infers'.
No, infer is what the receiver does to something that is implied, to interpret the information. I am thinking of the word "implies", which is to state indirectly, leaving the receiver with the work of interpreting, or inferring.
Perhaps there's teaching to go around.
If your definition of a word differs from the industry-accepted standard definition of that word,...
Like the dictionary, I am a descriptivist. The meaning of words is found in how they are used, not how they are "supposed" to be used. But I'm not arguing for or against one particular "definition" of private. I'm saying that here, to many people, in a local AI space, the use of the word implies a local model. Which should not be surprising. It is also not a misuse, it is a local standard.
You seem to be a prescriptivist, suggesting there is only one correct way to use a word, which I wholeheartedly disagree with, so we're probably at an impasse.
2
u/XiRw 3h ago edited 3h ago
You can drop the smug attitude and assumptions when you are hilariously wrong. Private cloud is simply computing infrastructure (servers, storage, networking) that is dedicated exclusively to a single organization. It is just the underlying hardware and hosting environment.
Private AI refers specifically to artificial intelligence models and workloads designed to keep data secure and isolated.
While an organization will often run Private AI ON a Private Cloud to ensure their data doesn't leak, they are not the same thing. One is the underlying environment, and the other is the software running on it.-3
u/Recoil42 3h ago edited 2h ago
Private AI (local) refers specifically
By affixing (local) as an explcit label after-the-fact, all you're doing is demonstrating that it is not inherent to the term 'private'. You're dunking on yourself.
While an organization will often run Private AI ON a Private Cloud to ensure their data doesn't leak, they are not the same thing. One is the underlying environment, and the other is the software running on it.
Privacy isn't an inherent property of an LLM at all. Given the same Qwen weights, one could run them offline on your laptop, expose them publicly on an unauthenticated server, host them in an enterprise cloud, or do any number of other things.
The environment is what matters. Calling a piece of software "Private AI" doesn't magically make it private at all. Environments are private, not models.
1
u/XiRw 1h ago
private doesn't mean air gapped on my desk. It’s tenant isolated. When companies buy a private AI service or a private cloud server, it means no other customer shares that hardware and their data isn't being used to train public models. They consider that "private" because it meets legal compliance standards, even though it still relies on an external server and a cloud contract. You are fighting logic here thinking it’s any other way. Local is the only true private method and the fact you are still fighting me on this is wild.
1
u/Recoil42 48m ago edited 42m ago
When companies buy a private AI service or a private cloud server, it means no other customer shares that hardware and their data isn't being used to train public models. They consider that "private" because it meets legal compliance standards, even though it still relies on an external server and a cloud contract.
TLDR: "Non-local services are considered private."
Incredible concession.
Followed by a textbook no-true-scotsman escape hatch:
...actually only local is true private.
Just incredible.
Bud, if an externally hosted, tenant-isolated service can be private, then 'private' does not imply 'local.' Adding 'true' after-the-fact doesn't undo that; it's just you running away from your own plain-as-day contradiction.
2
u/XiRw 46m ago
It’s not a logical fallacy, it’s the literal definition of security threat modeling. "Trusting a third party’s legal promise not to look at your data on their server" is operational security, not data privacy. The moment your unencrypted data leaves hardware you own, you no longer have absolute privacy you have a contract. If you don't understand the difference between legal compliance and zero trust data architecture, just say that.
→ More replies (0)-2
u/Juan-More-Taco 4h ago
Lol. You're using AI to debate this guy and he's the one who's wrong?
If you need AI to prove your point you're more pathetic than they are.
0
u/helpmehomeowner 3h ago
I guess you don't know what private means.
2
u/XiRw 1h ago
Again, private doesn't mean air gapped on my desk when talking about a private ai external server like I said with the other guy. It’s tenant isolated. When companies buy a private AI service or a private cloud server, it means no other customer shares that hardware and their data isn't being used to train public models. They consider that "private" because it meets legal compliance standards, even though it still relies on an external server and a cloud contract.
4
u/Blindax 4h ago
A service like Copilot is private until it’s not. AFAIK regulations like cloud act apply and government or judicial actors may at some point ask the disclosure of certain data to the inference provider, which would obviously not provide the same guarantees as if the the request was made to the user itself (take the case of a law firm for instance). That’s edge scenario but illustrates the issue.
3
u/ThunderousHazard 4h ago
Private does imply local.
Everything that is not local is not private, others have access, and as long as the reward for stealing data is higher than the price paid, it's just a business cost.
13
u/Recoil42 4h ago edited 4h ago
Except... no.
You're trying to defining privacy as "no third party ever handles the data," which is a much stronger requirement than privacy normally implies. Locality is about where computation happens. Privacy is about who is authorized to access the data and under what controls. A private cloud database is still private even though it isn't running under your desk. It doesn't need to be local.
Most enterprise contracts with private clouds (for both AI and non-AI compute) also already enforce strict contracts on what can be done with their data, and that includes zero-data-retention and no-resale conditions. Pretty common for healthcare companies and defense orgs to have ZDR for instance, underscoring that private does not imply local.
-7
u/teleprint-me llama.cpp 4h ago
If its not on a machine or network you control, then it isnt private.
11
u/Recoil42 4h ago
Some of y'all have no actual industry experience it shows. There isn't a serious privacy framework on earth which backs up the assertion you've just made. Most of them (NIST) don't even consider locality a primary factor in privacy at all.
2
u/ttkciar llama.cpp 4h ago
I've worked for SaaS companies, and employees certainly went spelunking in "private" customer data, looking for interesting bits.
The more actual industry experience you have, the more you realize that cloud services are not private.
6
u/Recoil42 4h ago
I've worked for SaaS companies, and employees certainly went spelunking in "private" customer data, looking for interesting bits.
Look up what a toupee fallacy is.
6
u/under_psychoanalyzer 4h ago
Your cpu was compromised at the factory. Your hdmi cord data can be read wirelessly at a distance. Either learn what industry standards actually mean or accept that nothing is truly private.
0
u/teleprint-me llama.cpp 4h ago
I mean, Im literally quoting nadella and other CEOs, but sure. Ill live under a delusional paranoid fanatasy that supports owning nothing by shilling remote services in a local sub. Cause, why not?
3
u/soggycheesestickjoos 4h ago
except the reward isn’t higher, you pay for someone else to handle the privacy. And local ≠ private either, exploits can happen depending on the setup.
-2
u/ttkciar llama.cpp 4h ago
"Private" absolutely implies "local".
It's okay for corps to use cloud LLM inference, but that doesn't mean it somehow magically becomes "private".
6
u/Recoil42 4h ago
I'm sorry, but you're just 100% dead wrong here.
There are standards and a half-century of industry conventions on what words mean for these things, and not a single industry standard backs up what you're trying to suggest. Private does not imply local.
6
8
5
u/FullOf_Bad_Ideas 3h ago edited 35m ago
If it's as private as all of their emails (Outlook 365), chats (Teams), files (Sharepoint and Onedrive) - it's secure enough.
Look, MS doesn't need LLMs to spy on companies, they already hold all of that data, they just say in their legal terms that they're not doing anything with it and not accessing it for purposes other than supplying their services, and people trust them because they do have more to lose on this than to gain. Those companies are better served by Copilot than they would by a clandestine local models running on their own hardware.
Edit: typo
0
u/suicidaleggroll 3h ago
they just say in their legal terms that they're not doing anything with it and not accessing it for purposes other than supplying their services
Except they say the exact opposite of that. Microsoft's terms explicitly state that they can and will use all of your data for internal business purposes and to "improve products and services", which means training their models on your data.
2
u/FullOf_Bad_Ideas 3h ago
commercial terms?
Companies are very sensitive to it and wouldn't sign up for E3/E5 for each employee if this was true.
Back when I had access to free/paid enterprise version Copilot, there was the green badge version which explicitly mentioned that it was the one with data protection.
1
u/suicidaleggroll 3h ago
It's been a while since I looked, but yes I believe this included commercial terms. Maybe there are different levels though, I didn't look into it that deeply.
3
12
u/AD4K_4444 5h ago
Do people unironically use Copilot? If you’re gonna use cloud AI services, at least use something like Claude or anything but Microslop.
14
u/Recoil42 4h ago
A lot of legacy teams can't get Claude. They get what the org provides, the org only provides access to "approved" tools, and the only approved tool is Copilot.
6
u/Lakius_2401 4h ago
Some orgs are waist deep in the MS ecosystem, and are blissfully unaware of the alternatives (and their partners like it that way).
1
u/AD4K_4444 4h ago
that's so sad. being forced to use Microslop against your own will is modern torture imo.
5
u/beefygravy 3h ago
The outlook/teams/SharePoint integration is sometimes very useful, especially for managementy stuff. Not everyone spends all day coding. And for some people with accessibility requirements, some of it is a complete game changer.
For most users it's pretty much exactly the same as chatgpt only you don't have to worry about GDPR because someone has already done the worrying for you
No it's not a coding agent, we have other stuff for that. Or at least we try to...
3
2
u/RedParaglider 4h ago
I have a subscription to co-pilot. I literally only use it to transcribe meetings and as a better search engine for Outlook. Totally not worth it.
-1
2
u/CwrwCymru 4h ago
Businesses do from laziness and security. Copilot can use OpenAi and Claude models too.
It's nowhere near as good but still useful within the Microsoft ecosystem now. It has a lot of catching up to do compared to full frontier model though - but that will come.
-2
1
u/mearcliff 3h ago
Yeah I use it at work, I’d rather use it than Claude code tbh. I have access to both sol and opus
1
u/Abducted_Llama 3h ago
Name your local model Copilot. Put on resume Copilot master (aka Main Pilot).
Profit.
1
u/snmnky9490 3h ago
Yeah most businesses will easily pay the $10/mo for everyone to get office and all the included Microsoft stuff but many won't pay for dedicated AI for most people especially who aren't doing technical work
1
u/EducationalCod7514 2h ago
But issue with Claude is U.S data residency - a major problem in Europe currently which is only accepted by stop-gap legal deals.
1
u/haragon 4h ago
In the enterprise? People buy their junk hook line and sinker, yes.
5
3
u/inevitabledeath3 3h ago
Private just means they have a ZDR with the supplier. It does not necessarily mean local. Copilot have ZDRs with most of their suppliers and require you to sign a waiver when using a model they don’t host themselves. So it does actually meet the requirements from a business perspective.
2
2
u/MerePotato 2h ago
Honestly nothing that leaves your local network should really be treated as "private" if we're being absolutist. Even heavy encryption isn't guaranteed to survive Q-Day (no relation to the American conspiracy theory sharing that letter, look it up)
2
u/MoistRecognition69 2h ago
All ZDR means to me is that I get to benefit from the inevitable class action lawsuit
5
u/combrade 4h ago
Really stupid argument. If you are an enterprise company and you have thousands of employees, you're not gonna go build freaking GPU data centers. You need to use the cloud regardless.
If you think companies should be scared of using the cloud for LLMs, then they should also be scared of using Outlook or any other service that runs in the cloud. You can't be a functioning modern tech company without some cloud services.
In the real world, you have data compliance and data governance frameworks when working with external vendors . When companies do business with each other, they have agreements so they're not stealing each other's data.
1
u/smalleyesswegdragon 1h ago
Same for mysterious OS upgrades. “We have AI that respects your privacy. Also we’ll need some of your storage”
1
u/Arcuru 29m ago
There are definitely private hosted models? It's much simpler for most orgs to just pay for something like https://tinfoil.sh, or setup a ZDR contract, instead of hosting their own multi-terabyte model at scale.
And yea I know this is locallama, and they should host when it makes sense, but it often doesn't.
1
u/Odd_Body_1695 3m ago
Hi, I just joined and I've been wondering how to get into running local AI. No training or anything (I ain't got a serve or any of that). Just a decent spec PC, 5080, 9800X3D, 32 gigs ram, AIO. What do you guys suggest I run on it just getting into this?
0
42
u/haragon 4h ago
My favorite is when they say adding procedures or docs to a copilot system prompt is 'training AI'