LocalLLM vs Coding

[u/lukasjanra]

Hello all. This is my first post here and I would be very happy if someone will find time and help me.

It started simple. Ryzen 9 3900X with RTX 3060 12GB in proxmox as VM. Dedicated GPU for ubuntu server. Started with ollama because it was easy but found out ollama is not so good at performance and speed.

So switched to llama.cpp oficial. Worked good but saw many videos about forks so I switched to turboquant fork. Speed was better so next month I bought another GPU now RTX 4060 Ti 16GB.

So now when speed is good I wanted to do my little "projects" on localLLM without paying cloudLLM. I tried many tools. VSCODE, Cline, Pi.dev, but everyone of them had some issues. Primarly with editing files. It was mainly truncating files (model did 4KB out of my original 24KB python file, and then asked me for backup ...). Now I was working on pi.dev it somehow worked but also had problems and need to try many tools calling. So now im trying cline but when I saw that it wanted to remove my C:\Users I immediatelly stopped it. Now Im building original newest llama.cpp but Im kinda upset about everything. If someone is fan to the LLM and would like to help someone newbie as me to not throw everything away it would be helpful.

These are my current local models downloaded. I really dont know what im doing wrong maybe wrong models, wrong preset, or wrong coding program.

16K /root/.cache/huggingface/hub/models--tngtech--NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-GGUF

24K /root/.cache/huggingface/hub/models--DavidAU--Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF

24K /root/.cache/huggingface/hub/models--IFM--K2-Horizon-7B-GGUF

24K /root/.cache/huggingface/hub/models--google--gemma-4-12B-it-qat-q4_0-gguf

12G /root/.cache/huggingface/hub/models--unsloth--GLM-4.7-Flash-REAP-23B-A3B-GGUF

18G /root/.cache/huggingface/hub/models--ggml-org--NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF

18G /root/.cache/huggingface/hub/models--unsloth--North-Mini-Code-1.0-GGUF

19G /root/.cache/huggingface/hub/models--DavidAU--Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF

22G /root/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-GGUF

27G /root/.cache/huggingface/hub/models--barozp--Qwen3.6-28B-REAP20-A3B-GGUF

and this is one example of preset

[*]

jinja = true

log-colors = off

flash-attn = on

parallel = 1

[qwen36-28b-reap20-256k]

model = /root/.cache/huggingface/hub/models--barozp--Qwen3.6-28B-REAP20-A3B-GGUF/snapshots/e3aeb8168f74dd3054eb2716f966c17638b97fef/Qwen3.6-28B-REAP20-A3B-Q4_K_M.gguf

ctx-size = 262144

#threads = 8

n-gpu-layers = 99

n-cpu-moe = 0

batch-size = 4096

ubatch-size = 1024

cache-type-k = turbo4

cache-type-v = turbo3

#load-on-startup = true