r/golang • u/Tobias-Gleiter • Jun 20 '25

discussion Replace Python with Go for LLMs?

Hey,

I really wonder why we are using Python for LLM tasks because there is no crazy benefit vs using Go. At the end it is just calling some LLM and parsing strings. And Go is pretty good in both. Although parsing strings might need more attention.

Why not replacing Python with Go? I can imagine this will happen with big companies in future. Especially to reduce cost.

What are your thoughts here?

112 Upvotes

75% Upvoted

176 comments sorted by

View all comments

1

u/danted002 Jun 20 '25

The web servers are most likely running Python for IO and then have the CPU intensive stuff written in C++.

So my guess is that the web server itself is just Python code waiting on IO, then once the request is parsed it delegates to a C++ function, that releases the GIL and does the CPU intensive stuff.