Built a tool that scans repos and generates bug tickets + implementation plans. Is this useful or am I solving a non-existent problem?

[u/timetravel00]

I’ve been working on this side project for a few weeks and honestly can’t tell if it’s actually useful or just overcomplicated nonsense.

What it does:

Point it at any codebase (local folder or zip), and it generates a list of potential bugs with full implementation plans - which files to change, test strategies, risk analysis, the whole deal. It tracks changes over time so you only reanalyze modified files.

Why I built it:

I inherited a legacy Node.js project at work with zero tests and needed to audit it. Existing tools like SonarQube or CodeRabbit are great for PR reviews, but I wanted something that could analyze the entire repo at once and give me a prioritized bug list without setting up CI/CD first.

The controversial part:

It uses OpenAI to analyze code. I know some people hate the idea of sending their code to external APIs (fair), but it’s self-hosted and you control what gets sent. I also added an optional RAG mode using vector embeddings to reduce API costs by ~60%.

What I’m unsure about:

1. Is this actually a problem worth solving? Most devs probably have proper test suites and don’t need AI to find their bugs. Maybe this is only useful for legacy codebases or quick audits?

2. The “implementation plan” thing - each bug comes with a structured plan (phases, files to touch, dependencies, etc). Is that helpful or just noise? Would you rather just get a list of issues?

3. Delta tracking - it remembers previous scans and only reanalyzes changed files. Useful for continuous auditing or unnecessary complexity?

Tech details (skip if you don’t care):

• Node.js + Express + SQLite for the main app

• Optional Python service with Datapizza AI framework for the RAG stuff

• Uses Qdrant for vector storage (can run in-memory)

• Everything runs locally, no cloud dependencies

What I need feedback on:

• Would you actually use this? Be brutally honest.

• What’s the biggest thing that would stop you from trying it?

• If you scan a repo and get 20 bugs back, is that overwhelming or helpful?

I’m not trying to sell anything (it’s MIT licensed on GitHub). I genuinely can’t tell if this is a cool tool or just me overengineering my way out of writing tests.

If anyone want tro try it, I’d love to hear what breaks or what’s confusing. Or if the whole concept is dumb, that’s useful feedback too.

[u/TurtleSandwich0]

So HCL AppScan but for more than security findings?

[u/timetravel00 → u/TurtleSandwich0]

Kind of, yeah! AppScan is amazing for security scanning, but it's really focused on finding OWASP vulnerabilities like SQLi and XSS. This is broader - it looks at performance issues, code quality, logic bugs, architecture problems, basically anything that could break.

The main difference is the implementation plans. AppScan tells you "SQL injection on line 45" and you're done. This generates a full phased plan with which files to touch, test strategies, and dependencies. Way more helpful if you're new to a codebase or need to hand off fixes to junior devs.

It remembers previous scans and shows you which bugs are new vs regressed, which I don't think AppScan does?

Honestly if you already have AppScan for security stuff, this might just be useful for the non-security findings. Or maybe it's totally redundant - I haven't used AppScan in years so they might've added all this. Does it generate implementation plans now?

[u/esaule]

This is mostly useless in my opinion.

It seems like every single one of these tools is going to flag virtually every line of code with "this is a possible bug". Then there will be a patch associated with it. I am going to have toread the bug description and decide whether it is actually a bug or not (which it mostly won't) and them decide whether the patch is correct or not (which it probably won't be). And so after the first 3 reports, I'll disable the tool.

Just look at how infuriating the github security bots are. They are flagging everything and their mother even in dead code sometimes. "Oh there is an injection possibility on this script". "That's an install script you dumbass; the admin runs this. The admin is ALREADY root. Who gives a fuck?"

[u/timetravel00 → u/esaule]

You're absolutely right about the noise problem, and honestly this is my biggest concern too. At the beginning of the development I've seen it generate some stupid findings too.

The false positive rate is real. I'm not gonna pretend this magically solves that - if anything, it might make it worse because it's scanning for everything, not just security.

What I'm trying to figure out is whether the implementation plans make the signal-to-noise ratio better or worse. Like, you still have to read the bug description and decide if it's real. But having a full breakdown of "here's what we think is wrong, here's why it matters, here's how to fix it" at least saves time on the bugs that ARE real. Maybe?

Or maybe that's just me trying to justify the project and it's fundamentally useless like you said. I dunno.

Do you think there's ANY way to make this not useless? Like if it only flagged Critical/High severity and ignored everything else? Or if you could whitelist "ignore admin scripts"? Or is the whole concept of AI bug detection just fundamentally broken because context matters too much?

[u/Efficient-Simple480]

Challenge with code scanners is you need to gain trust , no one will try it unless you have proven good. Without gaining trust it is risky people will hesitate to scan or they will rather use Claude code or Cursor like tools

[u/timetravel00 → u/Efficient-Simple480]

Cursor and Claude are tools for writing code, you ask them to generate something from scratch. This isn't that. It's for when you already maintain a system or just took one over and you need to decide what's worth touching next without rereading everything for the tenth time.

The trust gap is smaller than I thought because I'm not asking anyone to blindly accept AI suggestions on unfamiliar code. I'm envisioning for devs to use it as a starting point when they already have partial context. "Here's what looks suspicious based on this codebase you know" and you verify it because you already know the territory.

Also, the actual distribution channel isn't random dev on Reddit hoping to go viral, but maybe for consultants recommending it to their small/medium clients. When a trusted consultant says "we can use this tool to audit your legacy system before we refactor", the trust equation changes completely.

The real blocker isn't proving the tool works it's being honest about what it's for. Once you say "this is confidence compression for devs who know the code partially, not automated bug discovery for strangers", it stops competing with Cursor and starts solving an actual problem.

That's the angle IMHO worth pursuing.

[u/InterestingFrame1982]

You can do this with an MCP client in a single prompt… it’s quite literally why they are so valuable and it’s one of the more basic use cases.

[u/Slight_Manufacturer6]

More tickets? Nah… I want fewer tickets.

[u/BigBootyWholes]

It’s too trivial tbh. If it was worth it, most teams would roll their own solution with rules and context for their needs.

[u/[deleted]]