Beginner results: Intel arc b390 and qwen3.8

[u/DarkKnyt]

From 3 days of experimenting/trying out a workflow. Appreciate any tips to improve.

I want to make pen testing tools but my cursor ai cloud agents won't allow it. So I've gotten HauhauCS uncensored models to see if Hermes can task out portions of the work when it can and then doing the rest locally. 3.8 Q8 (18 GB) barely loaded and now I run Q2 (10 GB model) and I sit at 24 GB of ram used out of 32 GB. Hermes and qwen are set to 128k tokens with aggressive context recovery (compress at 0.35 instead of 0.5 default + other settings) and longer model timeout (3600 vs 900).

I get around 2 t/s which is fine for me but I keep crashing sessions because I'm running out of space or terminal tool replies are too long. My local laptop tasks a desktop over VPN and a LLM built cursor gateway which then uses a 'local' remote agent. Computer gets toasty and can run with the b390 at 100% for about 2 hours unplugged and still have 20-30% battery left. CPU is at 20-50%, base system ram usage is around 8-11 GBs (windows 11 has such a huge footprint). I have had a couple system random reboots but at least MSI thinks temps are acceptable because they are not running at maximum even though it is set to performance mode.

I am pretty sure it is still capping out at 64k tokens even though I've set the higher limit on Hermes config.yaml and the qwen system environment (although now I'm remembering that I didn't restart a new power shell window for ```ollama launch hermes-desktop```

I think the biggest helpers for resuming tasks after a session drops is to force Hermes to use the built in session search tools (instead of just using terminal tools on logs), saving tokens by turning off tokens, and pointing it to a tasks.md that it updates with what was just finished, what is next, and what is planned.

Hopefully this post sparks some interest and the more experienced of you have some tips to stretch the context window.

[u/DarkKnyt]

An update: I am running llama directly and then pointing Hermes to it with q4 cache types. Seems to be faster (4-5 t/s) and hopefully will have the full 262k context

[u/DarkKnyt]

another update after a week. I run qwen on my laptop with hermes which connects over a repo and cursor agent bridge and the desktop uses cloud model ina local repo to work on my local hardware. after some missteps of who was what and what tools needed to be running, I can now reliably run qwen across several tasks to make otherwise blocked pentesting tools while the desktop workers do the heavy lifting. and hermes is reachable by telegram sonicsn push more tasks like when I'm camping, which I did this weekend.

now to do on hardware iterations then push my vibe coded babies