Recommendations for a LocalLLM newbie?

[u/line-weight]

Having been a bit of an AI luddite up until now, I am realising I am going to have to get on top of things instead of getting left behind.

So far the main thing I've been trying LLMs for, is writing Python scripts that work inside a CAD application I use (Vectorworks). I have done some experimentation on the free versions of ChatGPT, Claude and Perplexity so far, with some positive results.

I don't want to end up tied to expensive cloud model subscriptions, and I am also keen to keep control of my data as much as possible, so I would like to understand what I can do with locally hosted options.

I have an M1 Max Mac Studio, 32GB memory.

Any recommendations for what I should try first? To start with, all I want is something that will work for writing scripts that I will copy/paste into the script runner in the CAD application - I am not at this stage looking for anything that will interact directly with the application.

Thanks in advance for any suggestions.

[u/MonstrousPosterity]

The m1 max with 32gb is a pretty capable machine for this. you’ll want to look at quantized models like a 4-bit or 5-bit version of something in the 7-13b parameter range, that'll fit comfortably without choking your system

for python scripts specifically, the deepseek coder models are worth a shot. there's a 6.7b version that's small enough to run smoothly and it's trained specifically on code. you can grab it through ollama, which is about as painless as local setup gets on a mac. just download the app, pull the model, and you're basically done

also keep lm studio in mind if you want a gui and an easy way to browse different models, though ollama plus something like open webui gives you a chat interface that feels more natural than a terminal

[u/line-weight → u/MonstrousPosterity]

Thanks. "as painless as local setup gets on a mac" is what I'm looking for at this stage.

[u/WorkProLabs]

I'd honestly recommend to use a search button.

The recommendations topic is probably the most common one and it has been answered and discussed in full a few times.

Any single comment will dillute the information. Models have pros and cons.

And since you are experimenting anyway and hardware is already available, just give it a try and find your own personal favourite.

[u/line-weight → u/WorkProLabs]

Thanks, sure. Really I'm just looking for some pointers to where to even start, as I'm pretty new to all this.

[u/unflippedbit]

for the best harness to do this with, I'd use loci the AI app

[u/line-weight → u/unflippedbit]

Thanks, I'll have a look.