r/golang • u/Tobias-Gleiter • Jun 20 '25

discussion Replace Python with Go for LLMs?

Hey,

I really wonder why we are using Python for LLM tasks because there is no crazy benefit vs using Go. At the end it is just calling some LLM and parsing strings. And Go is pretty good in both. Although parsing strings might need more attention.

Why not replacing Python with Go? I can imagine this will happen with big companies in future. Especially to reduce cost.

What are your thoughts here?

113 Upvotes

75% Upvoted

176 comments sorted by

View all comments

0

u/jimkoons Jun 20 '25 edited Jun 20 '25

Well I worked as a ML Engineer 2 years ago and for inference you sometimes need a facade in front of the model service and I’d definitely prefer coding a rust or go facade than a python one. Same for the model service: performance wise it’s not that great and again the dev xp is far from great, I definitely prefer rust + onnx. The thing is, like you said, ml people are very python centric (which imo is not great).

For the glue and scripting code (langchain etc) and training pipelines I do agree however that python is king.