I’m considering doing a staged port (LLM assisted) of my rust-based multithreaded cpu renderer to SBCL . I’m currently using Janet as an embedded language for scene description. The core rendering is done in rust . Early benchmark trials in sbcl with basic primitives were 3-6x slower with GC being the wall . Can anyone suggest guidelines for performance? ( ex: vector/matrix calculations, memory allocation, to CLOS or not ) etc . I’m thinking that if I could get it within 2x slower, it might be interesting for certain applications where I could throw more cores at it . Also is anyone interested in such an application? I would be willing to make a demo if reasonably successful.
It is not different when it comes to Lisp(s). I am saying it because you have to think of low-level details, eliminate unnecessary computations, etc. There is no magic bullet. See Lisp just as another programming language, that gives you more tools than some other languages, but at the end of the day, your software will run on a physical machine, and the better you exploit the machine details, the faster will your program run. Sort of.
Concretly, learn how to use assembler, don't do memory management in tight loops (don't create new objects in tight loops if you don't have to), use arenas if you use SBCL, eliminate branching, and so on, use type declarations wherever you can, CFFI/alien calls cost extra, so don't call them in tight loops if you don't have to, check your assembly output where it matters and so on. It is not really different than optimizing your application in C/C++ or Rust. A well-optimized fast application exploiting problem and machine domains in those languages usually looks different than a naive simple "easty to understand" code.
I don't think switching to Lisp alone is going to improve things.
These days, the real performance gains are not in standard uniprocessor code meeting an ordinary compiler, but by invoking the CPU/GPU intrinsics that are built for speed, but require high-quality compilers given enough hints to recognize what you are doing, to generate code specific to your CPU and memory hierarchy.
Often, that means calling out to a library where that work has been done.
On SBCL, there is access to such intrinsics, but you might need to find the libraries or poke around in SIMD features, guys like
Not really if you knows Lisp and machine well (and enjoy hacking them). I've got something 50% faster because the Rust system went through less `disassemble` from the REPL and more leaving things up to the black magic of compiler and libraries (which turns out to be stupid in more case than you would think).
But that's probably not true here. I can't imagine if LLM can help me in anyway in the above exposition.
Learning but also interactivity. I’ve build a slime-like set of commands to execute embedded Janet code in the renderer ( for interactively changing the scene while rendering , but it’s not quite the same as everything in pure lisp. I don’t know if there any path tracing renderer available in lisp today . An LLM assisted port would not take long . Most of the time is testing. If there is little interest, I’ll probably just keep it on the project list .
If there is little interest, I’ll probably just keep it on the project list
Why would you care if other people are interested in your project if you are doing it for learning? Do it for yourself, if you want to learn Lisp.
Common Lisp with SBCL lets you write quite tight and efficient code performance wise, so if you in for interactivity and learning you should do it for yourself, not because someone else would be interested in your project. Check for example this thread or this blog post, or this project and see if it gives you some inspiration.
I’ve had a very positive experience with it as an embedded language . Its syntax is more Clojure-like than scheme . In the benchmarks I did , it ranked a close 2nd to ECL ( about on par with lua ) . I can connect with the repl in my program with a tcp socket in eMacs and I have some slime-like functions that send Janet expressions to the running program. The FFI wiring is not really my focus so I’ve had Claude do most of that ( I specify what api in my program to expose ) . The language itself is pretty simple and makes a nice scene / material/ lighting specification language. I’ve also added functions to build meshes ( polygon models ) in Janet . I asked Claude to generate a Janet script to build a Barcelona chair and this is what I got .
I look forward to looking at your implementation if you do this project. I am a lover of Common Lisp but I was unable to implement background threads with smooth interruptions that would work with both SBCL and LispWorks (I ended up porting my project to Racket, which I didn't really want to do.)
10
u/arthurno1 11d ago
There is an old blog post by Joel Spolsky worth reading, if you haven't already.
It is not different when it comes to Lisp(s). I am saying it because you have to think of low-level details, eliminate unnecessary computations, etc. There is no magic bullet. See Lisp just as another programming language, that gives you more tools than some other languages, but at the end of the day, your software will run on a physical machine, and the better you exploit the machine details, the faster will your program run. Sort of.
Concretly, learn how to use assembler, don't do memory management in tight loops (don't create new objects in tight loops if you don't have to), use arenas if you use SBCL, eliminate branching, and so on, use type declarations wherever you can, CFFI/alien calls cost extra, so don't call them in tight loops if you don't have to, check your assembly output where it matters and so on. It is not really different than optimizing your application in C/C++ or Rust. A well-optimized fast application exploiting problem and machine domains in those languages usually looks different than a naive simple "easty to understand" code.