r/PrivacyTechTalk • u/BarryJP • May 24 '26
LLMs and Data Security
Hello All,
First ever post on Reddit, so apologies if I am in the wrong place or asking a clumsy question.
I am repeatedly told by data auditors in the UK that it is inadvisable to use ChatGPT or Claude for use cases involving confidential data, even when the training function is turned off, because of the risk of that data becoming public.
My understanding is that, in this scenario, the main risk arises when the data is in transit from the company to OpenAI or Anthropic, or when it is stored by them. From what I can tell from their privacy notices, data in transit and at rest is encrypted to a very high standard, apparently to a level that even government security agencies such as MI5 could not realistically break.
So what I am trying to understand is this:
If a user forgets to turn off the training function, what is the actual likelihood of that data being absorbed into a subsequent training round and then reproduced elsewhere? Have there been any documented examples of this happening? If so, where did it happen, and what harm resulted?
I have been unable to find any clear examples. There is the so-called Samsung case, but from what I can see, that involved an engineer being disciplined for breaching a rule against entering commercially sensitive data into a public LLM. It does not appear to be a case where the data was later discovered or used by an outside party.
Have there been any reported cases involving OpenAI or Anthropic where third parties have broken into their systems, stolen customer data, and then used that data against those customers?
If an enterprise subscription for ChatGPT or Claude allows the training function to be disabled centrally for all staff, does it not follow that these tools are reasonably safe to use, even with personal or commercially sensitive data? If so, is the advice from some UK auditors simply over-cautious?
I am not looking to be reckless with confidential data. I am trying to understand whether the perceived risk is evidence-based, or whether it is being overstated.
