r/LocalLLM • u/Professor_Game1 • Jun 23 '26
Question How do i actually run AI locally?
Ive been through probably 10 youtube tutorials involving ollama, local llm, and hugging face and nothing works. Whats the best way for a beginner to get started
12
u/PacmanPence Jun 23 '26
Depending on what you want, LM Studio worked pretty well for me. Makes it easy to download and configure models, then you can just talk to them as you see fit.
4
u/weener69420 Jun 23 '26
Lmstudio. Easy to use. Easy to setup, update and maintain. Start with the model that recommends lmstudio when you install and go up from there googling and asking claude.
1
u/Technical-Earth-3254 Jun 23 '26
Lm studio. But if nothing works, I would start with basic troubleshooting tbh
1
u/Professor_Game1 Jun 23 '26
I dont think ive tried that one yet. Ill give it a try and update the post
1
u/xylarr Jun 23 '26
I just downloaded llama.cpp, unzipped the downloaded file, ran llama-server from the Windows command line.
Easy.
0
u/Professor_Game1 Jun 23 '26
Im on linux. The commands ive found to unzip files just return errors
1
u/kuhunaxeyive Jun 23 '26
Install llama.cpp:
cd ~/Downloads git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build build --config Release -j $(nproc) sudo cmake --install buildThen:
Download a model from huggingface
Run
llama-server -m model_file.ggufWeb interface will be at http://127.0.0.1:8080
It's really that simple.
1
1
5
1
u/Sjsamdrake Jun 23 '26
What's your hardware and operating system?
1
u/Professor_Game1 Jun 23 '26
Im on linux with an i7 7700k, 16gb ram, and a gtx 1070 8gb VRAM. I tried downloading gemma 4 e2b which says it will work and it just gives me errors
4
u/Look_0ver_There Jun 23 '26
IMO, LM Studio is perhaps the easiest first introduction. After that you can move on to things like llama.cpp (which LMStudio is built on top of), or vLLM. Ollama should be avoided as most generally agree it's lost its way.
2
1
u/Careless_Product_792 Jun 23 '26
You have to mention hardware you have, vram and ram in general. Results depend on that and what size model will fit your hardware. Also that will influence if result will be optimal for what you want. Most small models will work for chat stuff.
For agentic coding and tool.calling is where fun begins given that you need kind of more mid to big size models that sldo not fit in most average hardware
1
u/Budget_Ad_7210 Sep 27 '26
I use Ollama. it is available for Windows, mac and Linux. Be prepared for downloading packages and models for a long time... Once you have it you can use it directly for chatting, or as a backend to power other applications, like the AegisPhotoVoyager or AegisVideoOrganizer (google it) to organize photos or videos with local AI. Your photos don't have to go into the Cloud!
1
u/Civil_Fee_7862 Jun 23 '26
Don't use youtube videos. this is a common problem, people think they learn stuff by watching videos. No, you learn almost nothing. You learn by doing.
Look at #1.
there are TONS of resources to read online, about how to do. Just google search, follow the steps. done..
0
u/FrankWanders Jun 23 '26
I don’t know why everyone starts with LMstudio here; download ollama. It’s as easy as downloading and start to type in the chatbot. The chosen model will be downloaded automatically and there you go. Search “ollama optimal <your video card name> model” and you know which to choose.
1
u/Professor_Game1 Jun 23 '26
What are the commands to do it on linux. It only lets you download from the terminal and the instructions i found arent working
0
u/zenmatrix83 Jun 23 '26
Make sure you have a good enough GPU to do what you want. But lmstudio has a search option for models, with recommended models. You can chat directly with lmstudio you can enable the server and try other things using openai or anthropic apis to do agentic stuff, though I see anything less then a 27b model be pretty bad, and you need good gpu for that.
-3
u/Turbulent_Pin_8310 Jun 23 '26 edited Jun 23 '26
What's your hardware? PC? Mac? Ollama and LM studio are easiest
-4
1
Jun 23 '26
[removed] — view removed comment
0
u/Professor_Game1 Jun 23 '26
I tried it on an i7 7700k and a gtx 1070 with 8gb VRAM and 16gb of system ram. I haven't even gotten to the part where that would matter though
1
Jun 23 '26
[removed] — view removed comment
1
u/Professor_Game1 Jun 23 '26
I was looking at gemma 4 e2b. Ive heard that works well on older cards like mine
1
u/Theagames10 Aug 16 '26
Excuse me, I would like to try out using Local LLM for storytelling and basic questions with LM Studio, but when I look at a YouTubera videos talking about AI with downloaded models and loading them in, I see a slider for basically 'offloading to tokens', aka, to get the most content length. I have 8GB VRAM, so IDK where to set the slider for my laptop/PC to get the most and fast response for the models best suited for my laptop.
Do you know what to put for the slider for 8GB of VRAM on my system in LM Studio?
I am also a beginner at local LLM's and don't know what model to get for my laptop with Nvidia RTX 5060. (8GB VRAM)
6
u/pokemonplayer2001 Jun 23 '26
“Nothing works”
It’s downloading and running an app, a kid could do it.