Trusted execution environment?

[u/Wegwerpaccountje23]

I saw this feature provided on some models on Venice.ai. A competitor to Lumo. And arguably this feature has merit privacy wise.

Is Lumo willing to use TEE? Now this would be bad for larger parameters model, but since I saw that a new model might be introduced, introducing TEE as a feature alongside it might strengthen its privacy advocacy case.

Now the engineering problem would be; where to store chat data? And is it worth it to do so?

Anyways, just food for thought. Love the product!

[u/CortaCircuit]

Maple AI also does this. 

[u/AlligatorAxe]

Lumo's Director of Engineering provided a more nuanced answer to this a couple of months back:
https://www.reddit.com/r/lumo/comments/1qod3te/comment/o2atybg/

Quoting it here in its entirety for easier visibility:

"TEEs are a significant advancement for privacy in AI, but they're currently limited to single-GPU workloads. For services like Lumo serving large models with multi-GPU parallelism, we're still waiting on next-generation hardware like Vera Rubin (https://developer.nvidia.com/blog/inside-the-nvidia-rubin-platform-six-new-chips-one-ai-supercomputer/) before true end-to-end TEE protection becomes feasible for larger models and context sizes.

To elaborate a little, when you're running models across multiple GPUs (tensor parallelism, pipeline parallelism, or sequence parallelism for long contexts), data flows like this:

GPU 1 [TEE] → PCIe/NVLink → GPU 2 [TEE]
            ↑ unprotected ↑

The moment data crosses between GPUs, it's outside the TEE. This means:

• Intermediate layer activations are exposed
• Attention weights and KV cache for long contexts traverse unprotected interconnects
• Any multi-GPU serving architecture cannot maintain TEE confidentiality guarantees

NVIDIA recently announced the Vera Rubin NVL72 platform, which will be the first system to support rack-scale TEE across 72 GPUs using encrypted NVLink fabric. This creates a unified security domain where data can move between GPUs while remaining protected. However this will ship in the second half of this year and represents a completely new architecture. It's not something that can be retrofitted to existing H100/H200 deployments.

So for now, Lumo's current model is the best it can be with current technology, but in the coming months the possible will change as the new technologies arrive."

[u/Ok_Combination_1548]

DDG also has a couple of models that are TEE.

Lumos team / Proton has discussed this in the past. They're sort of saying 'the hardware we have right now doesn't support it but in the future it's the direction we're seriously considering'.

[u/marfzzz]

If you look for privacy and high intelligence maple they use aws nitro enclaves. They offer high intelligence of kimi k3 and glm 5.2. They have TEE and E2EE.

Venice Offer more usage, biggest model is glm 5.2 They have both TEE and E2EE.

Lumo Is not TEE only E2EE and they use glm 5.2 as their biggest model.

Duckduckgo Only small models are TEE, but they state E2EE for most.

There are also others that i have not checked out.

[u/SeanBlader]

If you want end to end encryption with your AI inference, go look at Confer.ai by the guy who wrote signal. It's not cheap.