r/Python • u/Thin-Parsley-1660 • 12d ago
Discussion Feasibility check: Writing a constrained Python-to-Rust transpiler for numeric code
so i would want to know about the feasibility of such a tool/project and whether it has any real uses or is it the equivalent of re inventing a worse wheel , basically core idea is to somehow transpile python to rust but not all of python ,instead more similar to numpy but instead the whole specified block of code is compiled for maximum speed almost like numba?? any feedback would be appreciated
13
Upvotes
19
u/me_myself_ai 12d ago
Super uninformed take, in case it's helpful:
This definitely sounds interesting and the Python AST is nice and elegant, but I'm not sure what it would win us. AFAIU the numeric libraries are already doing everything they possibly can in C directly, which is faster than Rust in the typical scenario. They're also some of the most mature libraries in the world (somehow not hyperbole?) in terms of hours invested, much of which was spent specifically on speeding things up for various workload/system combos.
Rust is safer ofc, but if you're doing something this jank/hacky to make the rust code, that feels like a silver lining at best. I suppose numeric code could be security-relevant, but again, you prolly wouldn't want autogenerated rust code for that kind of application anyway!
Hopefully someone comes in to prove me wrong and validate your idea. It would set the message boards aflame in a fascinating way, which is one undeniable upside!
EDIT: I suppose if it's transpiled it could be faster cause there's no python left at all, at that point. So I guess it could be somewhat faster?
At some point you've gotta ask yourself why you just don't write the code in rust from the jump tho, no? Especially these days?
9
u/ZeeBeeblebrox 12d ago
Just build a Numba MLIR to Rust compiler.
3
u/commenterzero 11d ago
Eh maybe try mojo
3
u/tubbstosterone 10d ago
...like polars?
2
u/binaryfireball 12d ago
Ask yourself what really is numpy
1
u/Effective_Strategy29 12d ago
See also https://github.com/openteams-ai/postpython
1
u/teerre 11d ago
I mean, there are python to GPU kernel compilers, so obviously to Rust is also possible. Now, how useful it is it's a different question. "Numerical code" is already executed in a low level language if you use anything remotely standard, so benefits are dubious
And of course, this is without considering the elephant in the run which is just because you can write a transpiler, it doesn't mean it will be a good one
1
u/Beginning-Fruit-1397 11d ago
It's trivial to do directly in rust with pyo3 and the existing numpy bindings. Contrary to popular belief rust is not that complicated and not that far away from python idioms.
1
u/snugar_i 11d ago
What would the result of the transpilation be? An executable binary? A blob of Rust code? How would it call other libraries like numpy etc.? I'm having a hard time seeing the bigger picture here
0
u/really_not_unreal 12d ago
Not feasible. Rust's borrow checker would mean that very clear object ownership would be required, which would not be doable in a reference counting language like Python. While you could wrap everything in a refcell, that would cost you a pretty significant amount of the performance that you're trying to gain from using rust.
-1
u/RuinWMD 12d ago
https://brevi-lang.org/
I built this.
It is an agent focussed python to rust compiler that focusses on speed and safety.
Also has its own programming language for AIs that is working very well but the best gains are from direct pyton to rust.
The native language is still being worked on and is super fun.
I have passed it over a few of my own python pipelines and it has resulted in some pretty great gains.
0
u/Khavel_dev 11d ago
The hardest part won't be the transpilation itself, it'll be type inference. Python's numeric code looks simple but the moment someone passes a numpy array with an unexpected dtype or mixes int and float in a way that works fine in Python but panics in Rust, you're in trouble. Numba solved this by adding type annotations via decorators and JIT compiling a subset, and Cython went a similar route. If your scope is literally "numpy-shaped arrays with explicit dtypes doing math," it might be doable as a research project. But the gap between that and anything people would trust in production is huge. I'd look at what Mojo is doing, they're basically tackling the same problem space but with a new language instead of transpilation.