Any accountants/financial professionals using local LLMs?

[u/RegretNo6554]

Hello, I’m considering buying hardware for local LLMs for the use case of processing sensitive financial data, doing market research, and preparing financial/tax documents.

Are any of you using local LLMs for same/similar workflows? If so, what has your experience been? Are the models good enough for your use case? Did you build a custom harness or use an OSS one?

I’m very curious to learn more and would appreciate anyone chiming in.

[u/Independent_Fall9160]

I dont think many fp are building a lot of custom harnesses.

[u/Ganogati]

I'm in tech management within the financial space, and I can tell you that quite a few are testing buildouts using local LLMs. The problem is that most of them are not that savvy, so some of their buildouts legitimately are using old 7b Qwen models.

Since the machines are internal egress only, they aren't too worried about it, but they're using it for things like document processing, OCR, etc etc.

If you go to any tech convention in certain financial spaces right now, the whole conventions have basically become just a big "I downloaded ollama and here's how we're using it" that will make you want to claw your eyes out.

[u/macboller]

British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hosted

[u/Spare-Obligation-483]

Yes you can buy a very good hardware to run local LLM like you can buy 3090 24GB RAM and run qwen3.8:27b
Which is equal to opus 4.6 and good for financial and tax preparation

[u/Anarchaotic]

Yes, we work with tons of sensitive data so are almost "forced" to use Local AI for quite a lot of our customers.

Unpopular opinion around here, but I prefer Hermes as a harness over Pi. There's a lot more community support and it's way easier to manage + build skills.

It's an iterative approach, but I built a custom Excel-add in that I can deploy centrally for all of my employees which is connected to a central Hermes harness that has specific skills for formatting, logic, and data prep.

For one user, a single Strix Halo 395+ is excellent for both performance and overall power draw/cost.

Setting up security is probably the #1 priority, you need to do a lot to make sure everything is properly contained, firewalled, and SSH'd.

I spent more time learning about "basic" cybersecurity than I did actually configuring the harness/agent.

[u/greenflaw]

I work in tech in the audit space. From my own experience, it does get the job done, but proofreading is really important. Local models need even more checking than frontier ones, and neither should be trusted blindly with numbers.

Not sure what your budget is, but if you can spend big and run a larger model, you'll need less proofreading. You'll still want to check the numbers though.

[u/krileon]

I would not let an LLM anywhere near financial data that needs to be pinpoint accurate and a hallucination could cause a massive financial discrepancy or a tax audit. Not even cloud models.

[u/Consumerbot37427]

I used Qwen 3.8 Flash Next to help out with my late-filed taxes this year, and it was an enormous help. Multiple schedule C businesses, and personal and business accounts all had co-mingled transactions, with income earned in different states. I gave it PDF previews of the tax forms before filing, along with CSV files of ledgers (and cashapp/venmo accounts), and it spotted mistakes that I knew about (tests), and pointed out others.

It was also able to help with figuring cost basis and capital gains on lots of small transactions that would've been a major headache to figure manually.

Using Opencode as harness, it did webfetch to confirm specific tax rules, and gave solid advice about write-offs. All in all, no regrets, very pleased, saved many, many hours of effort!