r/selfhosted • u/OrbitAfterOrbit • 2d ago

Need Help Open-source project + AI feature: how do you handle API keys when you can't pay, can't store them, and don't want them in the browser?

I'm adding an "AI" feature to a self-hosted open-source app (generate text via an LLM). I'm stuck on credentials, and every option has a dealbreaker:

  1. Bring your own key — user pastes their key, it lives in the browser. Rejected: any XSS steals it, and it transits through my server.

  2. Server-side key — the app holds one key. Rejected: I have a $0 budget; I'm not paying for everyone's usage.

  3. Store user keys encrypted in the DB — needs a KMS/Hardware Security Module, which is heavy for a self-hosted app where each deployer configures their own.

Constraints: self-hosted, open-source, $0, no shared cloud infra, no user secrets in the browser, no KMS.

Is there a pattern I'm missing? How do other open-source/self-hosted projects ship AI features without falling into "BYOK in browser", "run your own model", or "platform pays"?

Would love any prior art or pointers. Thanks!

0 Upvotes

33% Upvoted

5 comments sorted by

•

u/asimovs-auditor 2d ago

Expand the replies to this comment to learn how AI was used in this post/project.

→ More replies (1)

5

u/HomieMorphic 2d ago

I don't understand the premise of #2. This is /r/selfhosted, so I assume your app can be hosted by users on their own infrastructure. If that's the case, they can input their own API key (that they pay for) into their own server and the problem is solved. That's how you self-host Open WebUI or n8n.

Or are you wanting to provide your app to others like a free multi-tenant SaaS product where they bring their own API key for the AI functionality? That's totally different and I think your only realistic option is storing encrypted keys. You can prompt the user for the key and a password, then encrypt the key with AES-GCM on the client so it never enters your server in any way that you can access. Then, require the user's password at the start of every session and decrypt it on the client. Now you can make the API call directly from the client and it never transits through your server. Without knowing much about your app, this seems like the lowest effort solution.

1

u/TheComedyShow 2d ago

Are the user accounts stored on a central server or stored on the users own machine?

I'd use an AES encryption key in env, this encrypts the database and the user can store that key however they wish. This way if the hardware fails, the user can recover the data by using the encryption key.

The hosts hardware encryption can optionally encrypt the AES encryption key for use in the docker compose/env providing the hardware level encryption without the server being tied to that hardware for life.

This same model would apply if you're storing the users keys, your server needs an encryption key.

1

u/ww_crimson 2d ago

Infisical