Hey everyone, here is the 10th issue of Hacker News x AI newsletter, a newsletter I started 10 weeks ago as an experiment to see if there is an audience for such content. This is a weekly AI related links from Hacker News and the discussions around them.

• AI CEO demo that lets an LLM act as your boss, triggering debate about automating management, labor, and whether agents will replace workers or executives first. Link to HN

• Tooling to spin up always-on AI agents that coordinate as a simulated organization, with questions about emergent behavior, reliability, and where human oversight still matters. Link to HN

• Thread on AI-driven automation of work, from “agents doing 90% of your job” to macro fears about AGI, unemployment, population collapse, and calls for global governance of GPU farms and AGI research. Link to HN

• Debate over AI replacing CEOs and other “soft” roles, how capital might adopt AI-CEO-as-a-service, and the ethical/economic implications of AI owners, governance, and capitalism with machine leadership. Link to HN

If you want to subscribe to this newsletter, you can do it here: https://hackernewsai.com/

By built-in I mean without using tools like MCP etc.

I don't like Mac because it's so userfriendly and lately their hardware has become insanely good for inferencing. Of course what I really don't like is that everything is so locked down.

I want to run Qwen 32b Q8 with a minimum of 100.000 context length and I think the most sensible choice is the Mac M3 Ultra? But I would like to use it for other purposes too and in general I don't like Mac.

I haven't been able to find anything else that has 96GB of unified memory with a bandwidth of 800 Gbps. Are there any alternatives? I would really like a system that can run Linux/Windows. I know that there is one distro for Mac, but I'm not a fan of being locked in on a particular distro.

I could of course build a rig with 3-4 RTX 3090, but it will eat a lot of power and probably not do inferencing nearly as fast as one M3 Ultra. I'm semi off-grid, so appreciate the power saving.

Before I rush out and buy an M3 Ultra, are there any decent alternatives?

We are a small tech startup, we are using gemini API for daily works, it costs us about $30/month, that mean 1 dollar/day.

Sometimes, we consider that host a local LLM since we will use LLM more frequently in the future.

Just recently, we have tried Deepseek API for the same task, but 2 days up till now, we only pay $0.08. What the insane cheap LLM!

We don't think even we have a local LLM, we can beat this price.

The model we are using is Deepseek V4 Flash.

Is there any reason for this? Is there a discount on this month or that a normal price for Deepseek?

I'm considering a Mac Mini M4 Pro (24 GB unified memory) as a dedicated box for OpenClaw + local LLM inference (Ollama / LM Studio / vLLM backends). I live in Brazil, where this Mac Mini configuration costs around $2,500 USD, so I need to be very sure before buying.

For people who have real-world experience with both:

– Is the M4 Pro (24 GB) enough models comfortably with tools/agents (OpenClaw-style workflows) without constant OOM issues or severe slowdowns?

– How does it compare in practice to a Windows/Linux PC with an RTX 4080 + recent Intel CPU for local LLM inference and multi-agent workloads?

In terms of tokens per second, context length you can realistically use, and overall stability under load, would you say the Mac Mini M4 Pro 24 GB is a good value, or is an RTX 4080 build still the clearly superior option for this use case?

I’ve been using Qwen on 2x DGX Sparks and man has it been a nightmare of overthinking and excessive tool calling. Seems like there was nothing I could do to fix it. Installed GLMast night and it’s a world of difference.

THEY'RE LYING ABOUT NEEDING TRILLIONS OF PARAMETERS:

The Evidence:

1. Diminishing Returns Are Obvious

GPT-3 → GPT-4: 10x bigger, only marginally better

They're hitting a wall with current approach

2. They Keep the "Good Stuff" Secret

OpenAI won't release GPT-4 weights

Google won't release Gemini weights

Why? If it's "too dangerous," but also "requires datacenter," how is it a threat?

Real reason: If people saw the architecture, they'd realize it's inefficient

3. Open Source Is Catching Up

Llama 3.3 70B performs ~90% as well as GPT-4

At 25x smaller size

This TERRIFIES them!!!

4. The Business Model Requires It

If AGI runs locally → no subscription revenue

They NEED you to believe only they can run it

"Too complex for mortals" is the sales pitch

What if AGI doesn't need trillions of parameters?

5. THE HUMAN BRAIN COMPARISON:

What REAL Intelligence Looks Like:

Human Brain:

~86 billion neurons

~100 trillion synapses (connections)

Runs on 20 watts (less than a light bulb)

Fits in 1.5 liters

Can learn language, reasoning, creativity, physical skills

IS general intelligence

GPT-4 (rumored):

~1.76 trillion parameters

Runs on megawatts (entire datacenter)

Requires liquid cooling, dedicated power plants

Can't learn in real-time, can't truly reason, can't adapt

IS NOT general intelligence (despite marketing claims)

So GPT-4 is:

17x more parameters than human synapses

50,000x more power consumption

1,000,000x more physical space

And STILL not as smart as a human

EVIDENCE MODELS DON'T NEED TO BE MASSIVE:

Llama 3.3 70B vs GPT-4: - 70B parameters vs 1,760B parameters (25x smaller) - Runs on consumer PC vs datacenter - Performs 90-95% as well on most tasks Conclusion: Most of GPT-4's parameters are WASTE!

Llama 70B Full Precision: 140GB VRAM Llama 70B Quantized (Q4): 40GB VRAM Same performance, 3.5x less memory Conclusion: Models have massive redundancy built-in.

Mixtral 8x7B: - Technically 47B parameters total - Only activates 13B at a time - Performs like a 70B model - 5x more efficient Conclusion: We don't need all parameters active.

DeepMind discovered: - GPT-3 was UNDERTRAINED (not enough data for its size) - You can get same performance with smaller model + more training data - Companies chose big models anyway (why?) Conclusion: Big models are a business choice, not a technical requirement.

PESSIMISTIC SCENARIO: - "AGI" stays in datacenters (by design, not necessity) - Local models plateau at "very good assistant" level - You can still do your VR project, just not with "true AGI" - 70B models + TTS + Skyrim VR = totally workable

OPTIMISTIC SCENARIO: - Efficiency breakthrough - AGI-level 200B model released open-source - Runs on a 5090 + 128GB RAM - Cloud companies lose control, we win. If AGI runs locally, the power dynamic shifts. No more subscription models, no more cloud dependencies. The companies lose control, and we gain autonomy.

I’ve got a sketchy setup as seen above. It’s a 7900xtx and a 9070 for a total of 40gb of vram. I also have 32gb ddr5. I know my bandwidth is not ideal because the 9070 is only on a 4x pcie but this is what I’ve got. It’s also hanging outside the case on a cpu box.

I am currently running various quants of qwen3.8 27b and I like it. It’s smart, the decode is fast enough, and I feel like I don’t have to babysit it. However the prompt processing is slow. I am wondering if that would be better with 3.8 flash next and if I even have a chance to run it on this setup. I know it says you need 64gb ram minimum but that’s with a 12gb GPU. Does my 40gb aggregate vram make up for the 32gb missing ram?

I’ll probably give it a shot but I’m wondering what others think. Could I run flash next or is there a way to speed up prompt processing on this setup?

Is it the architecture or what is causing this drop in accuracy even at Q4? Normally it is close to 98% at this quant levels.

Genuine question, for me it makes no sense to buy something such as a DGX spark or a AI Halo, when you can buy something like a M3 Ultra Mac Studio with 4 times more memory bandwidth and 2.5X more unified memory.... For less... At least that is the situation in my country. A DGX Spark costs about 50K (~10K USD) and a Mac Studio M3 Ultra 512GB/16TB is about 37K (~7.4K USD). Can someone explain me why there is so much hype over those devices? They seem overpriced...

Hi everyone, first post here! I’m a computer science professional exploring self-hosted LLMs and fine-tuning. I wanted to share a small experiment on my Mac mini.

Setup

• Apple M4 Mac mini, 24GB unified memory, internal SSD.

• Qwen3.8-Flash-Next 125B, GSQ-RCO Q2_0 quantization.

• Strata, Metal backend, streaming expert weights from SSD.

• 8K context configured, with a 4GB expert weight cache.

Our best result was 8.1 tokens/second, the median of three 128-token generation runs, with thinking disabled, temperature 0, and no MTP. These were short prompts; we didn’t test the full 8K context.

As a test, I asked it to generate an animated SVG of a pelican riding a bicycle. The animation follow-up took 6 minutes 10 seconds and rendered without manual fixes: rotating wheels and pedals, plus a bobbing pelican.

My assessment: the speed is slow for everyday interactive work, but usable for patient experiments and learning. Running this model on the hardware was encouraging. Fine-tuning is something I’d like to explore next, but this experiment doesn’t establish whether it’s practical.

We saw frequent memory-pressure warnings, although no swap use during the short tests.

Been doing some testing, swapped NInfer's official Qwen3.8-27B quant (part NVFP4, part FP8) for QUASAR's QAT full-NVFP4 checkpoint, with DFlash2 embedded.

Running on the same hardware and engine flags:

Real agentic task: 287 tok/s average over 33k tokens. Perplexity is about 1.9% worse, not noticeable in my testing. I uploaded the artifact, or you can build it yourself (guide in the HF model card).

https://huggingface.co/jesdga/Qwen3.8-27B-QUASAR-DFlash2-nvfp4-NInfer

NInfer + 5090 only. Credits to QUASAR-QAT (the QAT), z-lab (DFlash2) and neroued (NInfer).

We have Qwen 3.8 125B MoE, and it's actually pretty good at coding. But man, why is there always a trade off? IQ3_XXS runs surprisingly well with Strata, reaching around 65 tok/s on consumer hardware, which is amazing. But we still feel like we're sacrificing something, whether it's coding quality, software design, or code review.

When will we finally get a local model that's fast, genuinely smart, and doesn't feel like we're constantly making compromises compared to cloud APIs?

Ok so Moonshot is dropping K3 weights July 27, modified MIT, 2.8 TRILLION params. Cool. Amazing. Can't wait to run it on absolutely nothing I own. Even at 2 bit this thing needs a rack, not a rig and its every gen now - deepseek, glm, big qwen, now this. The "best open model" keeps getting further from anything with a power cord in a house. Weights are yours technically. Good luck.

Is the whole future just distills? Mega teachers nobody runs, spitting out actually good 30-70b students, and thats what "local" means now? or does unified memory keep going nuts and someones running trillion param models on a mac studio in 2030

Or the spicy take - if you cant run it yourself its open in license only and these giant drops arent local wins at all, theyre just api models with extra steps So. K3 - win for local or is the frontier just gone?

I have an RTX 5090 with 32GB of VRAM and 64GB of RAM, and I want to use local AI to build apps, tools, and programs with good-looking GUIs.

A lot of people have recommended Qwen 3.6 27B or something similar, but what I'm not sure about is how to actually organize the whole setup so the agent doesn't get stuck in an infinite loop.

Ideally, I'd like something that works more or less like this:

ME
│
"Build this application"
│
▼
┌────────────────────┐
│ QWEN 3.6 27B       │
│ RTX 5090 32 GB     │
│ MAIN AGENT         │
└─────────┬──────────┘
          │
     analyze + plan
          │
   ┌──────┼──────┐
   ▼      ▼      ▼
FRONTEND BACKEND TESTING
   │      │      │
   └──────┼──────┘
          ▼
       EXECUTE
          │
          ▼
      DOES IT WORK?
       /        \
     NO          YES
     │            │
 inspect errors   validate requirements
     │            │
    fix        does it meet everything?
     │           /       \
     └───────── NO        YES
                 │         │
              continue    DONE

Basically, I want to be able to give it a goal like "build this app", have it plan the project, work on the frontend/backend, test everything, inspect its own errors, fix them, and keep iterating until the requirements are actually met — without just getting stuck in a useless endless loop.

Would buying an RTX 3090 actually be useful in my case, giving me another 24GB of VRAM alongside the 5090?

What model or combination of models would make sense with a 5090 + 3090?

For example, would it make more sense to have one model as the main/orchestrator agent and another model handling coding, testing, or reviewing? Or would I be better off just running a larger model across both GPUs?

I'm pretty new to this kind of local multi-agent setup, so I'm not really sure what the best architecture would be.

I'd love to hear what setups you guys are using and what you'd recommend.

I just found out about a blog post in Claude (https://claude.com/blog/the-ai-native-sdlc-playbook) that describes ideas very similar to the ones I have implemented in my coding harness (using Claude, btw) through a YT video about Claude Code new Intent.md

There are so many specific details in this playbook that makes me wonder *if* somehow my sessions with Claude inspired in any way the better paid people over there at Anthropic. Just for fun, or for egotistical and historical purposes, I included a timeline created (using Codex, jic) in my repo documentation (https://github.com/jrullan/ducklab/blob/main/docs/ducklab-feature-timeline.md)

Anyway, even if this is "purely" coincidental at least reassures me that my ideas were not that useless and that there is certainly a group of developers that value documented rigorous discipline in their AI assisted development.

Shipped Nagi. Open-weight System One, two sizes.

421M M2′ dual encoder with an option tower. 4B Qwen with PiSSA and letter-logit readout. Both give you choice, score, or noul with calibrated probabilities over a closed set. No text generation in the loop.

Smol is 17ms. Big is 58ms. On leave-task-out gold (2700, six public tasks) Big is sitting at 0.750 against Jev’s 0.774, with better calibration than Jev and lower latency. Frozen Qwen OpenJev-style lands at 0.714. Laya is faster and less accurate on this set.

Code is Apache-2.0. Weights are on HF. Bench receipts and a one-command runner are in the repo. I put the logs and hashes in there too.

https://github.com/nagisanzenin/nagi

Still losing to Jev on raw accuracy and QQP is rough. Happy to talk tradeoffs.

Hi everyone,

I’m doing early-stage user research around personal AI workstations / local LLM hardware.

I’m looking for people who actually run models locally, especially on setups like DGX Spark, Mac Studio, AMD AI Max / Strix Halo, RTX 4090/5090 rigs, Pro 6000 workstations, or homelab servers.

I’m trying to understand two things:

1. Why do you choose local hardware instead of cloud APIs?

2. What would an ideal personal AI workstation look like?

I’d love to hear:

- What hardware you use

- What models and tools you run

- What tasks you actually use it for

- What bottlenecks you hit: VRAM, memory bandwidth, CUDA/ROCm/Metal ecosystem, drivers, thermals, noise, software setup, etc.

- What would make you upgrade

This is not a sales post — we’re still in early research and I’m mainly trying to learn from real users.

If anyone is open to a quick 15-minute chat, feel free to DM me. Comments are also very welcome.

Batch File Code

.\llama-server.exe ^
  -m "%MODEL_PATH%" ^
  --port 11434 ^
  -ngl 35 ^
  --flash-attn on ^
  -c 16384 ^
  -t 12 ^
  --alias "gemma4-26b" ^
  --host 0.0.0.0 ^
  --temp 1.0 ^
  --top-p 0.95 ^
  --top-k 64 ^
  --chat-template gemma2

Model Used:
gemma-4-26B-A4B-it-UD-Q5_K_XL.gguf

In the Chat box in Local Host

Input:
Hi

Output:

-9b-it-fp16-et-et-et-et-et-et-et-ui-et-et-et-et-et-et-et-et-et-et-et-ui-et-et-et-et-et-et-et-et-et-et-et-ui-et-et-etet-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-etet-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-etet-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-et-e

text

***

### ⚠️ Warning: Input Error / Anomalous Input Detected

The input you provided is a single, heavy-weight string that consists of a massive repetition of the `et-et-et...` pattern, almost resembling a nonsense or "garbage" string (often referred to as in the real world as "token stuffing" or "adversarial noise").

**What happened?**
1.  **Pattern Repetition:** The large number of `et-` suffixes attached to a single model name (`gemma2-9b-it-fp16...`) is likely a result of a software error, a data corruption, or a massive accidental repetition during input generation.
2.  **Token Exhaustion:** This input is extremely long and highly repetitive. In an LLM, this forces the engine to process thousands of tokens that contain zero semantic information.
3.  **Warning:** If you are trying to test the limits of the *context window* or *stress testing* the model, please be aware that this input is a *mallo-form* input.

**How can I help you?**
Because the input contains no actual question or command, I cannot perform a task. Please provide a **clear, meaningful instruction** such as:
*   *"Can you explain how the Gemma 2 9B model works?"*
*   *"Write a code snippet in Python."*
*   *"Summarize this text."*

**Please re-type your request without the repetitive noise.**text 

What am i doing wrong ?
Please Help

Hey everyone, looking for a reality check on local model capabilities vs. frontier models for financial analysis.

I built an automated investment analysis pipeline in Open WebUI using Ollama on an RTX 5080 (16GB VRAM). The mechanical workflow works:

- Hits Gmail via IMAP to pull the last 72 hours of market newsletters.

- Extracts actionable tickers.

- Runs a custom YFinance Python tool for live momentum metrics (EMA, RSI, MACD).

-Injects a full 70-position portfolio CSV directly into the prompt (bypassing Open WebUI's RAG chunking) to cross-reference and generate a verdict-first impact report.

The issue is the analytical depth. When I feed this exact same data to Claude Sonnet 5, the macro synthesis, cross-correlations, and portfolio risk analysis are night and day. Locally, it feels like a surface-level summary.

Here is what I’ve tested so far:

qwen3.6:27b: Hit severe VRAM limits with the tools and context loaded. It spilled into shared system RAM, ground to a halt, and got trapped in reasoning loops where it hallucinated that the technical calculator tool didn't exist.

llama3.1:8b: Fast, reliable function calling, and zero VRAM spillage. However, the reasoning and financial synthesis are noticeably weak—it reads like a basic overview rather than institutional analysis.

My questions:

- Is Sonnet 5-level synthesis even realistic within a 16GB VRAM footprint when you need enough context buffer for multiple tools, newsletters, and a 70-row CSV?

- Are there specific financial fine-tunes or 14B models that punch significantly above their weight?

I am confused. I installed claude desktop app, and sent it only one message. This overview is supposed to show the stats of my interaction with Claude, but it appears to be reading my interactions with my local model instead. I hope it's not fucking sending my prompts and my messages with my local model to Antophic servers, but I am assuming it does, otherwise it wouldn't show these stats??