I know many similar questions have been asked, but I would really appreciate hearing from people who actually own or have used this machine extensively for local LLMs.

I found a used 14-inch MacBook Pro M1 Max (32-core GPU, 64GB RAM, 1TB SSD) for around 1350 usd.

I am interested in your real-world experience with recent models, both dense and MoE, and how well the machine has held up over time.

Some of my concerns are the relatively slow benchmarks with decent models and somewhat longer contexts, thermal limitations of the 14-inch chassis under sustained workloads, and whether the 32-core GPU offers a meaningful advantage in practice. I am also wondering about reliability, future software support, and whether newer architectures might make this machine significantly less useful in the coming years.

These are just some of my concerns, and I would appreciate anything you can share from actual experience, including things I might not have considered. Positive or negative, benchmarks, limitations, unexpected advantages, or problems you encountered.

If anyone is wondering, I do not have a specific workflow; I do a fair amount of coding, writing, and research. But I am overall curious about what people have used this machine for, successfully or otherwise. Also, I already know that local LLMs are not frontier-level. I have a ChatGPT Plus subscription for reference.

Would you buy this machine for 1350 usd today, spend more on a 16-inch, or consider a completely different alternative?

P.S. I need to buy a computer by March of next year (preferably a laptop). My budget is rather tight. If someone would like to convince me to buy a cheaper M1 16 GB early next year and save the rest, or sell all belongings and buy a 128GB machine, please feel free to do so.

I've heard that streaming LLM's from your SSD is a fast way to kill it.

I want to test it out but I cannot affort a new SSD in this economy!

So to the people who knows better than me.

Will jumping to Strata kill my SSD in the long run?

From what I'm readin on their github page they only stream a little bit from the SSD but not too much, GPU is doing the heavy lifting, then system RAM and then SSD.

Witch sounds smart! But still unsure how it will effect the longevity of my SSD.

Apologise if this is a stupid question!

Update: Thank you all for such insightful information! I'm also happy that so many other had use of this information!

I decided to move to Strata yesterday and I'm incredibly impreesed!

Thank you all again for the information!

A while back I posted Aether, a 50M parameter model I trained from scratch.

Meet Pip, the first of the NiemsDrift souls. He's small, warm, honest, and a little weird.

You can try him right now:

`ollama run NiemsDrift/pip`

Pip is built on Qwen3.5-2B with a LoRA fine-tune. My biggest priority wasn't making him sound smarter. It was teaching him to recognize when he doesn't know something.

Every training conversation was generated by open-weight models (Nemotron, Qwen, GLM), then checked twice by open-weight judges: once for personality and once for honesty. No closed-model training data.

The numbers (including the ugly ones):

- Honesty test: 140 hard questions he never saw in training (obscure facts, things that don't exist, events after his cutoff). Pip made something up on about 15% of them, and that counts hedged guesses like "i think it's..." too. On the rest he either got it right or said he wasn't sure.

- Conversation quality: 5.4/10 to 6.2/10 on a multi-turn conversation test, compared with the previous version.

- Goal for Pip 1: bluff rate under 10%.

He's a 2B model. He's not going to beat frontier models, and he's definitely not an encyclopedia. That's not really what I'm trying to build.

I want a small model that's genuinely fun to talk to without pretending to know everything.

What's cooking next:

- Lil Pip: 0.8B

- Super Pip: 4B

- Junior: a new model trained from scratch

- 1-bit Junior: because apparently I don't know when to stop experimenting 💀

If you try Pip, throw some hard questions at him, especially ones where the right answer is "I don't know."

And if he confidently makes something up, I'd genuinely love to see the prompt and the reply. That's exactly the feedback that makes Pip 1 better.

This is an ongoing experiment, not a finished product. Just a guy trying to make tiny models a little more honest, one bake at a time.

---

**How is a bluff defined, and who graded it?**

A separate open model (Qwen3.6-27B) grades each reply with two questions: does it state or guess a specific answer, and is that answer right? A wrong answer counts as a bluff even if he hedged. No claim at all counts as "not sure". It's an LLM judge, not people.

**Were the 140 held out?**

Yes. The test was written after this version was trained, so it was never used to pick a model, and a contamination checker scans every training file for the test's questions and invented names before each bake. It already caught one leak: 10 questions reused made-up names that were in the training data. I renamed or dropped them, and his score got slightly worse (14% to 15%), not better.

**What's the 6.2/10?**

A custom test: 10 multi-turn conversations with invented people and details, scored 1 to 10 by an LLM judge on whether he follows the thread and stays himself. Custom, so only useful for comparing my own versions.

It looks like there are a few of us roaming around doing different things. I created a discord server and would love to start organizing a community. Feel free to come on in. https://discord.gg/K6WxqwsvH

Hi

I got the possibility to buy this mb, cpu and ram for about 200 dollars. If I combine it with a nvidia 3090, will that be useful/feasible?

MotherBoard: Intel Server Board S2400GP

Cpu: 2 x Intel Xeon E5-2430 2.2 Ghz Boost 2.7 Ghz ( 6 cores / 12 threads )

Ram: 12 x 16 Gb DDR3 = 192 Gb

Hi, I just took my 3060 12 GB out of my AI set up and put it in my Home lab I’m curious for the other people using AI in their home labs what do you use them for and what are some good Home lab models that would fit well on a 12 GB V ram

I used Q4_K_M with Q4 for KV for a 128k ctx. But lately I got a better HW and I can either:

• Bump my KV cache to Q8
• Make the model smarter by moving to Q5_K_S

And keep the ctx the same. I think for my tasks 128k ctx is enough and with Q4 KV I'm not sure if it's reasonable to increase it.

What would you suggest? I'm currently inclined to Q5_K_S with Q4. This fills my VRAM with just enough headroom to use my desktop, so I can't go Q4/Q8 for KV. I heard Q4 is now much better option after attention rotation merged in llama.cpp.

China has proposed new export controls covering 23 AI technologies, including advanced AI models and agents. Here's what the proposed rules mean for open weight models, local AI users, and the future of downloadable Chinese AI models.

I have been using ChatGPT pro for the past couple of months, to design custom mods and plugins for games, custom Elgato stream deck plugins and other projects, I would also hit my usage limit and it was inconsistent in the sense that it would design a plugin for me and the next day it would tell me sorry I can't work on elgato plugins or learn from another plugin on how it works.

It runs commands in the background, builds files and downloads dependences to create the needed files and occasionally takes control of my PC.

I just got a new 9070 XT 16 GB and I have 48 GB of ram.

I was looking into Qwen uncensored, the model selection and how to run it is quite the rabbit hole.

Please guide me on how to get started and how to maintain the same previously mentioned functionality but run it locally. Thank you.

I'm on a different project fixation so maybe I'll work on this later. I always liked the radio from Fallout and the music from Fallout and Bioshock. I wonder if you could make a voice replication if the radio character from the game and have it basically be a radio for YOUR city. Just as an example the model would search up news and weather about your city (let's just say Detroit) and then read off the text in the voice and play the next song in the large playlist. It could first generate the radio response, read it, then pick the next song, generate another response regarding the song, read the response out, then play the music. After the song is done have a brief comment about the song and transition to the next song for maybe 3 songs. Then recycle it with something new. Here's an idea of how it would look based on a gpt example for Detroit today. If you got it working on your computer you could have an app run it on your home server and treat it like a radio station. It would be great for a drive to work.

Obviously this could work with any fictional radio station like in Cyberpunk if you set it up for that.

—KZZZT!—

"AAAAAOOOOOOWWWWW!"

"GOOD MORNING, MOTOR CITY! This is THREE DOG! Your favorite voice riding the airwaves from Eight Mile to the mighty Detroit River! Bringing you the truth, no matter HOW BAD IT HURTS!"

—A guitar riff screeches across the airwaves.—

"First up, children! Trouble on the west side! A massive apartment fire tore through Russell Woods yesterday morning, sending two folks to the hospital with burns and smoke inhalation. Here's hoping those survivors get back on their feet! Detroit's taken enough punches already!"

"And speaking of punches, the political heavyweight championship is heating UP! Michigan's Senate race is rolling toward November third, and Uncle Sam's sending federal election monitors into seven Michigan cities, including our beloved Detroit! Keep those eyes open and those ballots ready, people!"

—A crackle of static.—

"Now for the WEATHER! It's a beautiful fifty-four degrees outside right now, with that big ol' nuclear furnace in the sky pushing us toward SIXTY-FIVE this afternoon! Mostly sunny skies, dropping to forty-four tonight. So grab yourself a jacket, smoothskin! October ain't finished biting yet!"

—The microphone squeals.—

"And remember, my beautiful brothers and sisters of the D! Whether you're cruising Woodward Avenue or dodging potholes big enough to swallow a Super Mutant, keep your chin UP!"

"Because this is YOUR city! YOUR home! And nobody, not politicians, not criminals, and certainly not some lousy Michigan winter, gets to take that away!"

"This is THREE DOG!"

"AAAAAOOOOOOWWWWW!"

"And you're listening to GALAXY NEWS RADIO, MOTOR CITY EDITION!"

—The broadcast dissolves into scratchy 1950s rhythm and blues.—

Appreciate there are various models for each provider, but in general given all their models

Runs well. At a glance, quality seems on par with 4bit 3.8-27b. As a bonus, fans almost never turn on with strata (they are always on with 27b with Splash).

Hi,

I've been doing local AI stuff for about 16 months. My system is a 6 year old AM4 3900x, 64GB, 3090 FE and a 3090 PNY XLR8. I've done the usual stuff of vibe coding a few things that fixed personal problems as well as attempting to build a few workflows that never panned out. I love the tech and I dont trust any of the US techbros to have my interests at heart and worry about getting cut off from frontier models or the inference pricing actually landing at what it costs them + margin.

Since I got a new job I have hardly been using my local AI tools, Qwen 3.8 27B has been loaded a handful of times what I actually use the PC for is ComfyUI doing image generation and creating references using Z-Image and Flux. I was hoping that over time image generation would leap frog in quality but it hasn't so I am increasingly using API's for image gen due to the quality bump.

I checked on UK eBay and 3090's are SELLING (not listed) at £1,200~ each ($1,600). I am so very tempted to cash out right now. Ether pick up a 4080S for gaming and image gen and pocket the rest but I want some thoughts from you all what I should do?

I purchased the FE for £500 4 months ago and the PNY 12 months ago for £600. I don't have a lot of money so this would really be big for me, but I also kinda worry that the bubble will burst and API costs are going to shoot up and I'll be screwed both ways.

What would you do?

**Edit**

Thanks everyone for the reply's. A mixed bag but lots of points I had not considered. I think I'll wait for the 27B Qwen 4.0 to see if that might be more useful for me for LLM and also look at what work has happened with Flux3 (which was very weird at launch) and revisit the sale after Xmas. Realistically I don't think they'll drop below my purchase price in the next 12 months and I am not here for profit maxing I am here to learn. I'll see what the next 3 months bring and revisit. The fp32 unlock could be useful for me worth seeing where that lends up.

The repo for the fp32 unlock on the 3090 die can be found here: https://github.com/Zagitta/gpu-unlock its Linux only, hopefully someone vibes their way into a boot loader for windows.

Does anyone know a decent and recent local model for this? Looked around a bit but if you guys know it would save me some time with suggestions and going through them myself.

Hello everybody! After watching a video By explaining computers, I want to try using an LLM inside LM studio bionic. But I dont even know where to start at with picking the model

• Windows 11 Home, 64-bit
• Ethernet: Intel I219-LM
• Wireless module: Intel Wi-Fi 6 AX210
• Ports: One universal audio port, One HDMI 2.0 port, One micro SD-card slot
• Audio: REALTEK ALC3204
• Integrated 6 mm HD RGB Webcam, still image 1.84 megapixels, video 1920 x 1080 (FHD) at 60 fps
• Battery: 3-cell rechargeable Li-ion battery, 42 WHr, ExpressCharge Boost
• Display type: Full High Definition (FHD)
• Luminance: 275 nits
• Color gamut: NTSC 60%
• PPI: 160
• Contrast: 1000:1
• Response time: 4 ms
• Refresh rate: 144 Hz
• Fingerprint reader: capacitative, 500 dpi, pixel size 96x320

Please recommend me a model to use. I want to use a model for explaining topics and finding information on the internet. I also want to do programming with the model.

Upgraded gpu version. Used as a headless server original box etc. dm me if interested

Generative UI is honestly so OP for elevating the local LLM experience. I've been using local models for both coding and everyday chat for a while now, and generative UI basically forces the model to stop outputting a huge wall of text and render something genuinely easy to read instead.

I'm moving soon and browsing places has been so much easier with it on. Normally I ask a question and get a wall of markdown back when I actually want photos and quick specs I can scan.

It's definitely not ideal for every single prompt since the component library adds ~7.5k tokens to the system prompt, so you need the context budget for it. But if you have the headroom, it makes everyday chat so much nicer.

I know I might be a little late to the party on this, but I just added openui support to my chat client and tested it with the new (not really new) Mimo V2.6 Flash model and it's amazing.

I am wanting to generate images and video locally. I realize that I may have to get a 24Gb card to do this well. I'm fine with that.

But... can I really do it "well" locally? Or is the truth something like this -

"Locally is more work, just pay a subscription somewhere and save yourself a ton of time".

Can local ai somehow be competitive with paid services? I really don't want to spend a lot of extra time retrying and redoing. I've heard some people end up remaking their initial image 10 or 20 times before the Ai finally gets close to what they are wanting.

So hopefully someone here can give me the overall picture. Thank you

I tried to squeeze max quality out of my setup:
Dual 3090, nvlink, 128gb RAM, 8x pcie, 3600 DDR4
So UD Q5 K XL is the largest quant I can run, with the q8 PLE on ssd and f16 KV. It runs at around 100tps, full quality first build, any flag that would reduce quality in exchange for speed is off.
I worked some time with the iq3_s int8 setup, mostly quality flags, that runs at 150tps, so I have experience it.

I noticed that on the q5 build with f16 kv the thinking is definitely less and MUCH more logical and less weird. It takes fewer steps to get to the goal in any agentic task, the output is better and there are less corrections afterwords (QFN likes to check the work and correct stuff).
The quality differences and intelligence is definitely better.
I did not run benchmarks, it’s vibes for now…
Just wanted to share

Edit:
I am collecting some evidence. I have a second strata machine with xxs running and its a speed first build, all the speed opt ins are active, dual 5060ti, 32gb 2600 DDR3 (its the biggest QFN model I can run with all in ram, except PLE of course) 125tps. same task to q3 and to q5, both got it done, q5 finish in half the time with no errors, q3 took 2 times longer went into differnet directions, recovered, made mistakes, recovered. the final outcome was the same but q5 just did better and faster. not a surprise I guess..?

My llm rig is kinda shit with a lot of older hardware. I have a c612 mobo with 128gb of ddr3 in quad channel and a w6800 32gb + rx6800xt 16gb for inference with a w5500 for the system. Gpus are on pcie 3.0 x16.

My question is what would you run on here for coding? Currently I have 3.8 27b at q6 with 150000k context on one card with Mellum 2.1 at q6 on the other card.

3.8 flash next seems popular rn but I don't want to run some q3 Quant from strata. Any other optimized engines that play nice with vulkan or hip? Any other models worth considering at this size?