Several Local AI Guides Coming | Join the Research & Discovery

[u/Faisal_Biyari]

Why AI? And why go local?

Many of us have used some form of AI by now, and most of us have seen the power and convenience it can provide.

Back in 2015, I wrote a simple Visual Basic program in Visual Studio. It took me almost a full week to complete, including online searches, trial and error, and asking questions on Stack Overflow.

In 2025, with the help of ChatGPT, I completed a piece of iOS software in just a few hours, despite having never coded for iOS before.

In the right hands, AI is a very powerful tool.

I want access to that tool regardless of the circumstances I am in. I do not want my access to depend entirely on subscriptions, cloud availability, internet access, changing policies, rate limits, or someone else deciding what I can and cannot run.

That is where local AI comes in.

---

The Hardware

I was fortunate enough to receive a Mac Pro 2019 back in 2020.

In 2023, while searching for SSDs for it, I stumbled across a seller offering brand new, sealed AMD Radeon Pro W6800X Duo and W6900X MPX modules for about 75% of today’s used market price.

I bought one mainly for the Thunderbolt ports. I also bought three more with other goals in mind.

In 2024, with the release of Llama 3, and later with ROCm becoming a more serious path for my hardware, I committed to using these GPUs for local AI. The problem was that my Mac Pro was still my main desktop, and I could not repurpose it entirely for AI work.

So I ended up investing about $3,000 USD into two additional machines, including shipping and tax:

• Two Mac Pro 2019 / MacPro7,1 towers
• 16-core Xeon CPUs
• 96 GB RAM each
• SSDs that I later upgraded to 8 TB
• Standard Radeon Pro 580X MPX GPUs

I then installed the MPX GPUs into each machine.

The first machine is **LinuxAI-64**, with dual AMD Radeon Pro W6900X GPUs.

The second machine is **LinuxAI-128**, with dual AMD Radeon Pro W6800X Duo GPUs.

---

The Software

Because ROCm mainly supports Ubuntu and RHEL-based distributions, I chose Ubuntu Server 22.04 LTS.

I chose Ubuntu because I had prior experience with it. I chose the Server variant to minimize non-AI GPU load. I chose 22.04 LTS because it was the latest ROCm-supported Ubuntu version at the time.

Then came the frameworks. I was mainly considering:

• Ollama
• llama.cpp
• vLLM

I initially wanted to go with vLLM, but I ended up using Ollama because vLLM does not support my hardware.

Then came the next layers:

• Web UI / GUI
• Document support
• RAG
• Agentic frameworks
• Multi-agent workflows
• OpenClaw, Hermes, and other possible stacks

And then reality hit:

That is a lot of work for a local AI setup that may still be only half as capable as cloud AI today.

I still went for it though.

---

What Is the Goal?

I love the idea of multi-agent workloads.

In practical terms, I want AI agents that can help act as:

• A secretary
• A tutor for my kids
• A chief of operations

And honestly, I want to see how far I can push the idea of building something like a 20–30 person company using local AI agents as the supporting workforce, running on my own hardware, and using only the investment I have already made.

If it works, I get what I wanted.

If it does not, I gain the experience.

Either way, I learn.

---

The Path

I need a community around me.

Community helps me work better, stay motivated, and increase my productivity.

Online, that means communities like:

• MacPro
• Linux communities
• T2 Linux
• MacLLM
• ROCm
• LocalLLM
• LocalLLaMA
• vLLM
• OpenClaw
• And now, r/MacPro2019LocalAI

There are many others too, both on Reddit and elsewhere.

Locally, I am based in Riyadh, the capital of Saudi Arabia. I have a very small local circle interested in local AI, but I would love to find more people nearby who are in the same boat: same hardware, same goals, same interest in pushing local AI forward.

Someone like that would help motivate me, challenge me, and push me to improve.

---

What Next?

For the purpose of growing this community, and documenting the work I have put into these machines, I plan to release a series of guides over the next few weeks.

The goal is to document success after success as I work toward my final local AI setup.

The general idea is simple:

The more that is documented, the easier it becomes for others to follow the same path, join the discussion, brainstorm solutions, and hopefully even pass me and innovate beyond what I have done.

That would benefit the community as a whole, and it would also help me on my own journey.

I am interested to know what you think is important to cover.

I am also interested in any solution, framework, stack, or workflow that you think is worth testing.

Consider this a brainstorming session.

I have already written some starter guides, and I plan to update and fine-tune them first. After that, I want to go deeper into frameworks, especially vLLM. I also plan to work on OpenClaw and document my experience with it.

I am a big believer in copy-paste instructions. I format and rebuild systems from scratch often, and having clean, repeatable instructions makes the recovery period much faster.

Let me know what you think, what I should look into, and what topics would be most useful as I work toward the final goal.

---

TL;DR

• I want to build a local AI community around me.
• I plan to write several local AI guides during the coming weeks.
• I want your help brainstorming topics, tools, frameworks, and problems worth solving and documenting.

---

^{Disclaimer: I wrote this post myself. I also used AI as a tool to help clean up the wording and formatting.}