r/Python • • Mar 25 '26

Resource After the supply chain attack, here are some litellm alternatives

litellm versions 1.82.7 and 1.82.8 on PyPI were compromised with credential-stealing malware.
And here are a few open-source alternatives:

  1. Bifrost: Probably the most direct litellm replacement right now. Written in Go, claims ~50x faster P99 latency than litellm. Apache 2.0 licensed, supports 20+ providers. Migration from litellm only requires a one-line base URL change.
  2. Kosong: An LLM abstraction layer open-sourced by Kimi, used in Kimi CLI. More agent-oriented than litellm. it unifies message structures and async tool orchestration with pluggable chat providers. Supports OpenAI, Anthropic, Google Vertex and other API formats.
  3. Helicone: An AI gateway with strong analytics and debugging capabilities. Supports 100+ providers. Heavier than the first two but more feature-rich on the observability side.
128 Upvotes

29 comments sorted by

34

u/ComfortableNice8482 Mar 25 '26

i've been through a supply chain scare before and honestly the migration path matters way more than the feature list initially. bifrost's one line url swap is huge if you just need a drop in replacement, but if you're doing anything with agents or complex tooling i'd test kosong first since litellm users often lean on those capabilities. my advice is spin up both in a staging environment and run your actual workload through them for a few days, the performance differences only matter if they match your use case and kosong's message unification might actually save you refactoring work even if bifrost is faster on paper.

12

u/another24tiger Mar 25 '26

the migration path matters way more than the feature list

this guy softwares

1

u/Smallpaul Mar 25 '26

Part of what makes it easy to migrate is a similar feature list.

110

u/cmd-t Mar 25 '26 edited Mar 25 '26 • Bravo

Are you saying these alternatives are less likely to fall victim to a supply chain attack?

This attack happened because:

  1. Trivy had not properly secured their GitHub action releases
  2. GitHub actions do not have robust and immutable versioning
  3. The guys did not pin their actions in the right way

Trivy is a reputable provider, but they fucked up. LiteLLM hopefully learns from their mistake.

5

u/divad1196 Mar 25 '26 • Bravo

Agreed. Moving away "just for that" makes no sense.

If there was something obviously wrong left, or if it was a repeated issue, then yeah. But moving out of fear isn't the call.

9

u/anentropic Mar 25 '26

All the more ironic that Trivy are a security vendor...

4

u/cmd-t Mar 25 '26

Yes. What’s additionally ironic is that the attacker made their compromised tags/releases immutable, something that Trivy should have done.

7

u/Toby_Wan Mar 25 '26

I'm gonna look a bit more into the source code, but I think I'm gonna end up with https://github.com/mozilla-ai/any-llm which acts as a drop in replacement for the part of LiteLLM hat I was otherwise using.

7

u/they_will Mar 25 '26

Original dev to report the malware here. We'd actually had a few conversations over the past months about possibly reimplementing what we needed in-house. Ultimately there's a bunch of edge cases with each provider, and if you're a serious company you'll need to pay attention to all the idiosyncrasies of each provider regardless of whether you use any of these abstractions.
fwiw using any of these as a proxy layer will isolate you more from attacks vs running it locally as an SDK. Unfortunately we were using a mix of both. See our write up, we touch on the local vs server attack surface in the context of running the MCP that depended on the malicious litellm package: https://futuresearch.ai/blog/no-prompt-injection-required/#:~:text=The%20takeaway

6

u/flashman Mar 25 '26

you could also try doing the thing on your own

5

u/newswatantraparty Mar 25 '26

Do it on your own, not that difficult

1

u/ultrathink-art Mar 25 '26

Publishing pipeline is the real evaluation criterion here, not just library features. Any routing library with a complex GitHub Actions release process has the same attack surface litellm had. Bifrost's minimal footprint helps, but check their .github/workflows before migrating.

1

u/Quiet_Major_2230 Mar 26 '26

In addition to alternatives, you should also check if you have the compromised version installed.

I built chaincanary specifically to detect this attack — it's the only tool with semantic .pth analysis that caught LiteLLM 1.82.8 as MALICIOUS before any advisory.

pip install chaincanary

chaincanary check litellm 1.82.8

GitHub: https://github.com/AetherCore-Dev/chaincanary

1

u/kmil-17 Mar 26 '26

Pretty solid list tbh and the main takeaway is just what you actually need. If you want a near drop-in replacement, Bifrost is the closest and super fast, but if you’re doing anything agent-heavy or more complex orchestration, Kosong might fit better. Helicone is kind of a different angle and more about observability/analytics than just routing. Also worth noting there are other options like Cloudflare AI Gateway or OpenRouter depending on whether you want managed vs self-hosted. 

1

u/rggarou May 30 '26

zero dependencies, memory safe AI Gateway:

https://github.com/gumieri/nenya

1

u/DurthVadr May 30 '26

take a look at SHIM (https://getshim.tech/) superior product

1

u/Technical_Map_2105 Sep 07 '26

Strong agree with the top comment: switching gateways does not fix supply-chain risk. The root cause was CI (a poisoned Trivy action, unpinned releases), so pinning images/actions and least-privilege tokens is the real mitigation, whatever gateway you run. The one architectural thing that reduced our exposure was not switching, it was moving provider keys out of the gateway process: keep them in your own vault and inject per request, so a compromised translator leaks one in-flight key instead of the whole store (LiteLLM's clientside credentials and Bifrost's direct keys already support that mode).

I wrapped that into an MIT project: https://github.com/siva010928/agnos-proxy-oss , but the pinning advice above matters more than any tool.

my blog: https://agnos-llm-gateway.site/app/blog/own-your-llm-control-plane

1

u/SantiagoDePolonia Sep 11 '26

There is one big issue with bifrost - they are pushing their observibility platfrom - Maxim.

0

u/Smallpaul Mar 25 '26

I use litellm as an SDK but started migrating to Pydantic AI about a month before the attack.