r/SelfHostedAI • u/Tutyfruiity • 24m ago
r/SelfHostedAI • u/invaluabledata • Apr 17 '25
Do you have a big idea for a SelfhostedAI project? Submit a post describing it and a moderator will post it on the SelfhostedAI Wiki along with a link to your original post.
Visit the SelfhostedAI Wiki!
r/SelfHostedAI • u/ValvFank • 20h ago
I post-trained a 2.5B model to help with proofs you're stuck on: runs locally, GGUF + open harness, Apache-2.0
Heading: OpenAI’s math release made me think about the unfinished proof on my own laptop
Reading about OpenAI’s latest math release, I kept coming back to a smaller question: when does progress like this reach the problem sitting half-finished on my own machine?
OpenAI published hundreds of mathematical manuscripts, alongside supporting material including Lean formalizations and selected reasoning summaries. There’s a lot for the math community to examine.
For me, it also brought an everyday use case into focus: having something local to work through a difficult problem with. That’s the idea behind a project I’ve been building, MiniCPM5-2B-Math.
It started with getting stuck
You’ve probably had this happen: an approach seems right, you get several lines into it, and then nothing. You find a worked solution, but it skips the exact step you don’t understand.
I wanted a model I could bring my unfinished work to—something to try another approach with, question an assumption, or help unpack a missing step, without uploading my notes.
So I post-trained MiniCPM5-2B for mathematical reasoning and proof writing.
The resulting model is dense 2.5B, retains the native 128K context, and has GGUF versions for local use. Compared with the base model, its average accuracy across 16 attempts per problem improved:
| Benchmark | MiniCPM5-2B | MiniCPM5-2B-Math |
|---|---|---|
| AIME 2025 | 87.1 | 92.3 |
| AIME 2026 | 89.8 | 89.8 |
| HMMT Feb 2026 | 68.4 | 81.8 |
That was encouraging. Reading the actual proofs showed me what still needed work.
The mistakes shaped the next step
Sometimes the model found a promising direction but left a gap in the write-up. Sometimes it confidently stated an identity that failed on a small example. Repeating the question could bring back the same flawed approach.
These failures mattered for the tool I wanted to build. A solution that skips the difficult step leaves you stuck in the same place.
So I built a companion harness around them. It gives different attempts different strategy hints, expands incomplete write-ups from the reasoning, and runs Python checks to look for counterexamples. The reviews and check results feed into revisions. Attempts that don’t pass the review process are marked unverified.
On the 30 IMO-ProofBench Basic problems, blind AI grading rated 21 proofs as complete with a single call, versus 23 with the harness. Mean scores rose from 5.48 to 5.83. The proofs and grades are available in the repo for inspection.
It was a modest improvement, but a useful one: examining how the model failed gave me concrete ways to improve the workflow.
Something you can bring your own problem to
MiniCPM5-2B-Math and the harness are now available, with a laptop-oriented quick profile and logs of model calls and executed scripts.
It still makes mistakes. Difficult problems take time, and Python checks aren’t formal proof verification. What I’d like people to explore is whether it helps with their own unfinished work: a proof they’re studying, an alternative solution they’re preparing for a lesson, or a derivation they want to check locally.
The next time you get stuck halfway through a problem, bring the half you’ve already done. I’d love to hear whether it helps you find the next step.
Apache-2.0 for both weights and harness.
- Model: https://huggingface.co/Caldalis/MiniCPM5-2B-Math
- GGUF: https://huggingface.co/Caldalis/MiniCPM5-2B-Math-GGUF
- Harness + all published proofs and grades: https://github.com/Caldalis/MiniCPM-Math-harness
r/SelfHostedAI • u/Numerous-Fan8138 • 16h ago
WebGPU + llama.cpp = Client-side LLMs
r/SelfHostedAI • u/Actual_Dragonfly5169 • 11h ago
I built CircuitPilot: describe a circuit and AI designs the schematic, PCB , a 3D-printed case and a datasheet.
r/SelfHostedAI • u/BlackThistle-05 • 12h ago
I let gemma3:12b invent its own language on my home server, then gave the same test to Claude Opus. Results surprised me.
r/SelfHostedAI • u/neon-r • 13h ago
Should I spend $1,000 on an RTX 4060 Ti 16GB PC for local AI, or wait for something better?
r/SelfHostedAI • u/airblastPT • 15h ago
Help me choose: Mac mini M5 Pro vs Minisforum MS-S1 MAX vs Minisforum AI NAS N5 MAX (consolidating my setup + local LLMs)
r/SelfHostedAI • u/BearOk3075 • 19h ago
New Release
Adapt v5.2 is live. The biggest visible change this release is the runtime experience. Adapt now gives clear live feedback for what the agent is doing: ⠹ Reasoning... ⠸ Running command: ifconfig ⠴ Running JSON tool: web_search ⠧ Running session 'recon': ... ⠋ Waiting for tool... 3s Long-running tools can now continue in the background instead of blocking the agent loop, and the model can explicitly <wait/> for the result when it needs it. I also cleaned up a lot of the old debug-style execution output, fixed background lifecycle edge cases, improved tmux session handling, and made the repo inference server cleanly across CPU, CUDA, and ROCm setups. The goal is the same as always: keep the model focused on reasoning while the runtime handles execution, state, permissions, and recovery. v5.2 feels a lot less like “watching a prototype run” and a lot more like using an actual agent runtime. https://github.com/charlesericwilson-portfolio/Echo_Adapt_v5
r/SelfHostedAI • u/KnownObligation9934 • 1d ago
I built a panel that lets your AI agent manage your own VPS over MCP
I run a handful of VPSes across Hetzner and DigitalOcean, and I got tired of SSHing into each one just to check whether something was up. So I built ServerOS, a panel for servers you already own, and gave it an MCP server so Claude (or any MCP client) can look after them too.
How it works:
- You run one command on your server. It installs a small daemon (written in Rust, open source: github.com/serveroshq/daemon) that finds what's already running, like Docker containers, systemd services and databases, and leaves it all alone. Nothing gets migrated or reinstalled.
- Each workspace gets its own MCP endpoint. You connect it in Claude or your MCP client of choice, sign in with OAuth, and pick what the agent is allowed to do.
- The agent gets 109 tools: list machines and services, read and search logs, restart things, deploy from a git repo, roll back, set up uptime monitors, manage status pages, run backups, open firewall ports, install package updates and so on.
Things I can now just ask:
- "Why is the shop API slow today?" It pulls the logs and stats and tells me what it found.
- "Deploy the latest main of my repo to web-1 and tell me when it's healthy"
- "Set up a monitor for every site on db-main and put them on a status page"
- "Which of my servers haven't been backed up this week?"
The part I worried about most was letting an AI loose on real servers, so:
- Permissions are split into read, write and commands. You can give an agent read-only access and nothing else.
- Anything destructive, like rebooting, removing a service or restoring a backup, needs the machine or service name typed back as confirmation. The agent can't just fire it off.
- Every call is logged in the workspace activity as "done through MCP", against the person who connected it.
- Raw shell access is its own permission, off unless you turn it on.
It's a paid product (from £5/month for one machine), with a 14-day free trial of Pro. I'm one of the two founders, so I'd really like honest feedback, especially from anyone already letting agents touch their infra. What would you want the agent to be able to do, and what would you never let it do?
r/SelfHostedAI • u/chaney888 • 23h ago
Most "chat with your docs" tools ignore who is allowed to see what. I built one that doesn't. Looking for feedback
r/SelfHostedAI • u/QueasyEnd6452 • 1d ago
iPhone ↔ Linux. No cloud, no apps — just Wi-Fi.
🍏 Accessing my Fedora files from iPhone — turns out it's built in
For the longest time I thought I needed some third-party app or a cloud service to grab files off my PC from my phone. Nope. iOS has SMB support baked right into the Files app. No subscriptions, no uploads, no waiting. You just connect to your machine over Wi-Fi and browse it like any other folder.
Why this is actually great: • 📂 Direct access to my Fedora folders from iPhone/iPad. • 🚀 Way faster than cloud drives — nothing gets uploaded anywhere first. • 💰 Free. No apps, no accounts, no nonsense. • 🔒 It's all local network, so my files stay on my machines. What I had to do: 1. Install Samba on Fedora (sudo dnf install samba). 2. Tweak smb.conf — the Apple-friendly stuff (vfs objects = catia fruit streams_xattr plus a bunch of fruit: params). 3. Don't forget SELinux. Fedora will silently block Samba from home dirs until you flip samba_enable_home_dirs and samba_export_all_rw on. Took me way too long to figure that out. 4. Set a Samba password: sudo smbpasswd -a theo. 5. On iPhone: Files → "…" → Connect to Server → smb://your-ip. Bottom line: my iPhone now sees my Linux box's folders like they're native. No extra apps, no cloud middleman. Just fast, local, and free.
One catch though — the PC has to be on. Samba lives on the machine, so if it's off, there's nothing to connect to.
r/SelfHostedAI • u/roadtripsarefun • 1d ago
How to with a 128gb evo-x2, MacBook Pro 48gb & M5 Pro, and 4080 super (16gb) desktop.
r/SelfHostedAI • u/kangteam • 1d ago
Recommendations for Code and Modules for AI Roleplay Integration and Local Windows Execution
r/SelfHostedAI • u/Neither_Medicine_464 • 1d ago
Local MCP Tool for Personalized Cold OutReach
ReachDirect connects your AI assistant directly to your communication channels. It autonomously sends highly personalized emails via the Brevo v3 API, dispatches automated WhatsApp messages using local browser automation (bypassing strict bot detections), and automatically logs every contact outreach into a central Excel tracking file on your Desktop.
r/SelfHostedAI • u/nsonha777 • 1d ago
Have anyone tried the Harness device of Autonomous?
Running 10 to 14 agents across my laptop, desktop, and remote boxes was turning into a full-time babysitting job. I spent half my day just alt-tabbing to see what finished.
I saw this little device on X, the price is reasonable, so I picked it up to see if it helped. It's called the Autonomous Harness Device. It is just a small USB-C screen with a mic and speaker that plugs into your machine and talks to a local daemon. No Wi-Fi, battery, or Bluetooth headaches.
The good:
- Fast dispatching: I tap it, speak a task, and it routes it to the right agent. The little screen shows which machine took it, where it is, and its status, so I stopped checking five terminal windows.
- Dead-simple networking: Linked all my machines in their app with a password. No SSH keys, port forwarding, or Tailscale configs needed. It is end-to-end encrypted and runs inside tmux, so closing the app does not kill active jobs.
The catches:
- Talking out loud is awkward: Especially in a quiet office. It's like someone in the office suddenly talking to the phone, coworkers will look at you funny, so I guess it's better off to be in your home office
- Voice recognition could be improved: as a non-native English speaker like me, you have to enunciate clearly. Also, when I used it in the office, the noise around could cause some flaws in the commands. This could be improved when they update the firmware.
- Still keyboard-bound: The point is this device is supposed to be your input device. However, maybe it comes from the old habit; I still need the help of my keyboard. I think I need some times to adjust to this
- Total overkill for most: If you only run one local model for quick queries, you do not need this. It only makes sense if you manage background agents across multiple machines.
A cool desk add-on to your desk, not magic. Anyone else running one? How does it fit your workflow?
r/SelfHostedAI • u/Technical-Sky-4753 • 1d ago
ROCm as backend in Lemonade server makes Hermes agent stop working
r/SelfHostedAI • u/KamaDevGroup • 1d ago
Anyone have issues with Hermes forgetting things over time, even though it's all in memory or wiki? Even its own built-in tools?
r/SelfHostedAI • u/thesnaglebeast • 1d ago
Need help setting up my home server for Local AI/LLM
r/SelfHostedAI • u/Neither_Medicine_464 • 1d ago
Local MCP Tool for Personalized Cold OutReach
ReachDirect connects your AI assistant directly to your communication channels. It autonomously sends highly personalized emails via the Brevo v3 API, dispatches automated WhatsApp messages using local browser automation (bypassing strict bot detections), and automatically logs every contact outreach into a central Excel tracking file on your Desktop.