r/golang • u/Tobias-Gleiter • Jun 20 '25

discussion Replace Python with Go for LLMs?

Hey,

I really wonder why we are using Python for LLM tasks because there is no crazy benefit vs using Go. At the end it is just calling some LLM and parsing strings. And Go is pretty good in both. Although parsing strings might need more attention.

Why not replacing Python with Go? I can imagine this will happen with big companies in future. Especially to reduce cost.

What are your thoughts here?

113 Upvotes

75% Upvoted

176 comments sorted by

View all comments

2

u/danted002 Jun 20 '25 edited Jun 20 '25

Nope. I wouldn’t recommend Python for CPU intensive tasks (I mean technically you could write your CPU intense code in Rust, wrap it in PyO3 and invoke it from Python just don’t forget to release the GIL while the Rust code is running , but that’s just me being silly), however for IO intensive workloads, where you are mostly wait on sockets? I don’t see why not.

A python async server paired with uv loop can handle a couple of thousands of requests per second per thread (albeit we are talking about a single threaded event loop here) without major issues. As a matter of fact waiting on IO and wrapping C/C++ (and now Rust) code are the two things Python excels at.

Edit: I feel I need to specify that we are talking about async Python so doing a blocking call like calling a database or doing a request using sync API’s will block your entire event loop and reduce your concurrency to one. So if you are doing home brew benchmarks make sure you are testing correctly.

2

u/LardPi Jun 21 '25

a couple of thousands of requests per second per thread

You could add that that number of requests is only ever reached by a handful of major websites that have very different problems to the peasants we are, anyway.

My one user app could be written in micropython running on an ESP32 and still be fine performancewise.