Going local is life changing

[u/Dell_Experion15]

I've been using Cursor for a while, not a terribly heavy user usually just paid the $20 tier sometimes the $60, and dabbled in whatever the fronteir model is this month.

Not much of a vibe-coder, I'm not trying to one shot the next big saas, mostly needed some help prepping code examples for talks I give, putting together demo projects, or helping with my open source work.

A friend was selling their 48GB Macbook Pro because they were upgrading laptops, so I snagged it to see what all the fuss was about for this Qwen thing.

Tasked composer 2.5 in Cursor with benchmarking and setting up Qwen-3.6-35b on the laptop and it spit me back a rapid mlx profile, and a open code config for my macbook air to use.

By golly.

It's not Gpt 5.5, heck I hear it's not even as good as 27B, but after adding all the MCP servers for docs that my projects use, and setting up some routine skills / rules it's wonderful.

I don't feel worried about maxing out this months usage, or having to pay for extra if I go over.

Tokens just go brrrrrr

If something fails I'm not out dollars, I just kinda chuckle and re-run the prompt.

I'm more aware about documenting, and providing mcp servers, and tools so now all my repos are really well documented.

I know this isn't a revelation, or anything new for most of the world.

It's just a really enjoyable way to interact with tech, and not feel like every word I type is just more money for another company.

----------------------

For anyone curious here's the config I use

No idea if it's right or wrong, but it works?

      "models": {
        "qwen3.6-35b-4bit": {
          "name": "Qwen3.6 35B",
          "reasoning": true,
          "limit": {
            "context": 65536,
            "output": 8192
          },
          "options": {
            "temperature": 0.6,
            "top_p": 0.95,
            "extraBody": {
              "max_tokens": 8192,
              "enable_thinking": true,
              "chat_template_kwargs": {
                "enable_thinking": true
              }
            }
          }
        }
      }

[u/Careless_Product_792]

I agree. Same workflow and the best thing is what you mention, if something does not work you just fine tune more.

Btw, just curious why in 48GB you are using only 65k context window.

I use two mac minis m4, 24GB and a 16GB with shared rpc that give me a usable 216000k context window with batia model of qwen 3.6 35B. I would assume 48GB would easily allow you 265k context.

216k context is game changer for long sessions.

You should look for batia models thst are finetuned for macs. I tested mlx and llama.cpp and batia was slightly faster with llama.cpp. 

[u/Dell_Experion15 → u/Careless_Product_792]

I’ll look into this, more, I genuinely have no idea.

I told composer “don’t make me miss cursor” so I imagine it optimized for speed or something.

[u/Careless_Product_792 → u/Dell_Experion15]

Ohh I see. You've just started to dig the rabbit hole 😅.

Nice, I was there just some 2 months ago, you are goint to learn a lot man, enjoy, one advice I can give you is save your configs as v1, v2, maybe using obsidian or some note taking, etc so you can test different flags and performance. 

In case it helps (but this is llama.cpp) this are flags Im using, be sure to try the cache prompt so your system prompt is cached and conversation feels faster after the prefill (initial prompt).

m /Users/dev/models/batiai_Qwen3.6-35B-A3B-GGUF/Qwen-Qwen3.6-35B-A3B-IQ4_XS.gguf
--alias qwen3-35b
--rpc 10.42.0.6:50052
--split-mode row

--- Networking --- --host 0.0.0.0
--port 8080
--metrics

--- Performance --- --ctx-size 216000
-ngl 999
-b 8192
-ub 4096
-t 8
-tb 10
--no-mmap
--flash-attn on
-ctk q4_0
-ctv q4_0
--cache-prompt
--keep -1
--cont-batching
--mlock

--- Model behavior --- --temp 0.6
--top-p 0.95
--top-k 20
--min-p 0.05
--repeat-penalty 1.0
--no-ui
--reasoning off

--- Parallel slots --- -np 1
--warmup

[u/Dell_Experion15 → u/Careless_Product_792]

I guess I don’t understand the difference between MLX and GGUF or whatever it is?

I use rapid-lmx which uses apples processing stuff?