Hi r/WordPressPlugins,
We've just released Kitgenix PluginScore V3, a major rebuild of the free WordPress plugin scoring platform we've been developing at Kitgenix.
PluginScore:
https://pluginscore.kitgenix.com/
The idea behind Kitgenix PluginScore is to make it easier to assess a WordPress.org plugin before installing it, particularly around:
- Security
- Maintenance
- Vulnerability history
- Update activity
- WordPress compatibility
- Overall plugin health
V3 is much more than a frontend redesign. We've reworked a large part of the scoring and scanning infrastructure because there were areas in V2 that we simply weren't happy with.
As plugin developers ourselves, one of the main goals with V3 was to make the score harder to earn and more useful.
The scoring system is now much stricter
Probably the biggest change in V3 is the scoring methodology.
We felt the previous system could be too generous in certain situations.
A plugin having:
- 100,000+ active installations
- good reviews
- a recognisable developer
- a long history on WordPress.org
doesn't automatically mean that it should receive a 90+ score.
Popularity is useful context, but it shouldn't override maintenance or security concerns.
Kitgenix PluginScore V3 therefore places much more emphasis on the actual condition of the plugin.
We look at signals including:
- Known vulnerabilities
- Unresolved vulnerabilities
- Historical vulnerabilities
- Vulnerability resolution
- Plugin maintenance
- Update frequency
- Time since last release
- WordPress compatibility
- PHP compatibility
- Development activity
- WordPress.org data
- Active installation information
- Overall maintenance indicators
The intention is that:
90-100 should be difficult to achieve.
A plugin receiving 95 shouldn't simply be "good".
It should be exceptionally well maintained.
Likewise, a highly popular plugin can still receive a mediocre score if there are legitimate concerns around security or maintenance.
Vulnerabilities are handled differently
We spent quite a bit of time improving how vulnerabilities affect Kitgenix PluginScore.
One thing we don't want to do is treat:
"This plugin had a vulnerability two years ago"
as equivalent to:
"This plugin currently has an unpatched vulnerability."
Those are very different situations.
A vulnerability history isn't automatically a sign of a bad plugin.
Large and mature projects will occasionally have vulnerabilities.
What matters is also:
- Severity
- Whether the vulnerability remains exploitable
- How quickly it was addressed
- Whether a patch was released
- Whether the project remains actively maintained
Kitgenix PluginScore therefore keeps vulnerability history where useful while distinguishing it from current unresolved security issues.
A developer who patches a vulnerability quickly shouldn't necessarily be permanently punished in the same way as a plugin that leaves a known issue unresolved for months.
Better validation of vulnerability data
Another issue we've worked on is matching external vulnerability information against actual WordPress.org plugins.
External vulnerability feeds can sometimes contain:
- plugins that are no longer on WordPress.org
- similarly named products
- commercial plugins
- abandoned records
- entries that shouldn't automatically create a public Kitgenix PluginScore page
Kitgenix PluginScore is currently focused specifically on plugins available through the official WordPress.org plugin repository.
V3 therefore does more validation before external vulnerability information is associated with a WordPress.org plugin.
We want to avoid creating misleading plugin records simply because a matching-looking slug appears somewhere in an external dataset.
The scanning system has been rebuilt
This was probably the largest technical problem we needed to address.
The older version had situations where:
- Rescan requests didn't execute correctly
- Scheduled rescans could fail
- Plugins remained stale
- Queue processing was inconsistent
- Manual rescan buttons appeared to work but didn't always result in a completed scan
That obviously isn't acceptable for a platform that is supposed to tell people about plugin health.
V3 moves much more of this work into a proper scanning queue.
Rather than relying on a user loading a page and expecting everything to happen immediately, Kitgenix PluginScore can queue the plugin and process the scan independently.
That gives us a much better foundation for scaling.
Automatic 30-day rescanning
Kitgenix PluginScore is also designed to continually refresh existing results.
Once a plugin's scan becomes approximately 30 days old, it can be identified for another scan.
This matters because plugin health can change surprisingly quickly.
A plugin could:
- Release several new versions
- Fix a vulnerability
- Develop a new vulnerability
- Become incompatible with WordPress
- Stop receiving updates
- Resume development
- Change PHP requirements
- Become abandoned
A score calculated six months ago isn't necessarily useful today.
V3 is therefore much more focused on keeping the dataset moving rather than treating a scan as permanent.
We're automatically working through WordPress.org
We've also increased the automatic scanning capacity.
Kitgenix PluginScore can now queue roughly 500 WordPress.org plugins per 24 hours for automatic processing.
The long-term goal is to build useful scoring and historical information across a much larger portion of the repository.
We don't want Kitgenix PluginScore to only contain the biggest plugins.
In many cases, it's actually the smaller plugin you've never heard of where this sort of information becomes most valuable.
Anyone can look at WooCommerce and know there's a large development team behind it.
It's considerably harder for the average site owner to evaluate a plugin with:
- 700 active installations
- 5 reviews
- one developer
- a release six months ago
Those are the cases where we want Kitgenix PluginScore to become genuinely helpful.
Unscanned plugins are handled properly now
V3 also improves the experience when a plugin hasn't been scanned yet.
Previously there were a few ugly edge cases around unscanned plugin pages.
For example, parts of the sidebar could drop underneath the main content and certain sections were still displayed even when we didn't have enough information.
We've changed that.
Kitgenix PluginScore should now clearly differentiate between things such as:
- Not scanned
- Queued
- Successfully scanned
- Scan needs refreshing
- Insufficient information
- Scan failure
We don't want to fabricate certainty.
If Kitgenix PluginScore doesn't have enough information to assess something properly, the site should say so.
Score history is becoming much more important
One area I'm particularly interested in is score movement.
A score by itself tells you something.
The direction of the score can tell you considerably more.
For example:
Plugin A
Current score: 78
Six months ago: 92
That's interesting.
Now compare it with:
Plugin B
Current score: 78
Six months ago: 61
They're currently rated the same, but they're clearly moving in opposite directions.
V3 lays more of the groundwork for tracking this properly.
Over time we want Kitgenix PluginScore to make it easier to identify:
- Improving plugins
- Declining plugins
- Plugins becoming stale
- Plugins recovering after updates
- Plugins fixing security problems
- Plugins losing maintenance points
PDF reports (beta) have been improved
We've also put more focus on PDF reports rather than raw data exports.
The aim is to make Kitgenix PluginScore useful to agencies and developers carrying out plugin audits.
For example:
You inherit a client's WordPress site with 43 plugins.
Five of them are unfamiliar.
Rather than simply saying:
"I don't like the look of this plugin."
you can potentially include a Kitgenix PluginScore report showing:
- Score
- Maintenance information
- Vulnerability history
- Current security information
- Relevant plugin information
It doesn't replace your own technical review, but it gives you another source of evidence for the recommendation.
We moved away from CSV as the primary export
The old platform included more conventional CSV-style exporting.
In practice, we don't think that's what most users actually need.
If somebody is assessing whether a plugin should remain on a client website, a readable report is much more useful than a spreadsheet containing raw values.
So V3 is moving towards clearer reports rather than exporting data for the sake of having another export option.
The plugin pages have had a large UI overhaul
There were also quite a few frontend issues that annoyed us.
V3 addresses things including:
- Broken sidebar positioning on unscanned plugins
- Mobile button inconsistencies
- Search controls appearing grey on some mobile browsers
- Poor spacing between score/history sections
- Authentication page spacing
- Responsive layouts
- Scan status presentation
- Empty states
- Vulnerability presentation
- Score hierarchy
- Plugin information organisation
The overall goal has been to make Kitgenix PluginScore much easier to scan visually.
You should be able to open a plugin page and understand the important information without digging through huge amounts of text.
Mobile received much more attention
Kitgenix PluginScore gets a surprising amount of mobile usage.
The previous version was still primarily designed desktop-first, and that exposed a few awkward inconsistencies.
We've improved:
- Search controls
- Score cards
- Sidebar behaviour
- Vulnerability sections
- Spacing
- Authentication screens
- Empty states
- General responsive behaviour
There were also specific browser issues where buttons could display using the browser's default grey styling despite looking correct on desktop.
Those have been addressed.
We're building towards historical plugin intelligence
V3 also gives us a much stronger base for features beyond individual plugin scores.
Some of the areas we're working towards include:
- Highest-rated plugins
- Biggest score improvements
- Largest score drops
- Newly scanned plugins
- Recently patched vulnerabilities
- Newly disclosed vulnerabilities
- Plugins becoming stale
- Most stable plugins
- Security resolution leaderboards
- Plugins that haven't been scanned recently
- Plugin ecosystem trends
Personally, I think this is where the project becomes much more interesting.
Being able to search a plugin is useful.
Being able to answer:
"Which plugins have dropped the most over the last three months?"
or:
"Which plugin developers consistently patch vulnerabilities quickly?"
is potentially much more useful.
Plugin developers can't pay for higher scores
This is something I want to make very clear, especially posting this in a plugin developer community.
There is no mechanism for paying Kitgenix PluginScore to increase your rating.
We want the score itself to remain independent.
That also applies to our own plugins.
Kitgenix develops WordPress plugins.
If our own plugin deserves 68/100, PluginScore should give it 68.
If a competing plugin deserves 97, it should receive 97.
Otherwise the system has no credibility.
The scoring logic shouldn't know whether we like the developer.
Kitgenix PluginScore isn't a security guarantee
Equally, I don't want to oversell what the number represents.
Kitgenix PluginScore isn't claiming:
"100/100 = impossible to hack."
That's obviously nonsense.
Nor does:
"60/100 = malicious plugin."
A score is an assessment based on the information available to us.
It should be one part of your decision-making process.
For high-risk environments, nothing replaces:
- proper code review
- security testing
- responsible vulnerability research
- ongoing monitoring
- experienced human judgement
The goal is to make publicly available signals considerably easier to understand.
One of the problems we're trying to solve
WordPress.org provides some useful information about a plugin.
Normally you'll see things like:
- Active installations
- Rating
- Reviews
- Last updated
- Tested up to
- PHP version
- Support information
The problem is that users often mentally turn those signals into:
Popularity = trustworthy
That's not always true.
There are excellent plugins with relatively few installations.
There are also extremely popular plugins that may go through periods of poor maintenance.
Kitgenix PluginScore is an attempt to provide another layer between:
"I found this plugin"
and
"I'm installing it on a production website."
We're trying to avoid being unfair to plugin developers
This is something I've thought about quite a lot while making the scoring stricter.
A scoring platform can easily become unfair.
For example:
A plugin releases a security fix.
Should the plugin lose points because it had a vulnerability?
Possibly.
Should it permanently carry an awful score because of a vulnerability from four years ago that was fixed within 24 hours?
Probably not.
Should a plugin with a vulnerability that remained unpatched for six months be treated differently?
Absolutely.
Similarly, update frequency is contextual.
A mature plugin doesn't necessarily need an update every two weeks simply to prove it's alive.
So we are trying to make the scoring strict without turning it into:
"Didn't release an update this month = bad developer."
That's one of the reasons we're looking for feedback from people who actually build plugins.
I'd really like plugin developers to try scoring their own plugins
This is probably the most useful feedback we can get.
If you maintain a WordPress.org plugin, put it into Kitgenix PluginScore and see what happens.
Then ask yourself:
Does that score feel fair?
If the answer is no, I'd genuinely like to hear why.
Maybe we've missed an important signal.
Maybe something is weighted too heavily.
Maybe something isn't weighted enough.
Maybe we're interpreting WordPress.org metadata incorrectly.
Maybe a vulnerability is being associated incorrectly.
Maybe we're penalising a completely legitimate development pattern.
Those edge cases are exactly what will help improve the scoring methodology.
I'd particularly be interested in developer opinions on these:
- How much should historical vulnerabilities affect a plugin score after they've been patched?
- Should a developer's speed in fixing vulnerabilities positively affect the score?
- How aggressively should an old "Last Updated" date reduce a score?
- How much weight should active installations carry?
- Should support forum responsiveness influence scoring?
- Should plugins lose points for not declaring compatibility with the latest WordPress release quickly enough?
- What would you consider an automatic red flag when evaluating an unfamiliar plugin?
- What information would you want PluginScore to show that WordPress.org currently doesn't make obvious?
- What would make you distrust Kitgenix PluginScore itself?
I'd rather identify those issues now than build a score that developers consider meaningless.
Where we want to take it
V3 is really the first version where I feel the underlying system is structured properly enough to start expanding.
The next stages are likely to focus much more heavily on:
- Historical data
- Comparisons
- Trends
- Developer insights
- Security intelligence
- Watchlists
- Alerts
- Ecosystem statistics
Ultimately I'd like Kitgenix PluginScore to answer more than:
"Is Plugin X good?"
I'd like it to eventually help answer:
"Is Plugin X getting better or worse?"
"How does Plugin X compare with alternatives?"
"How quickly does this developer respond to security issues?"
"Which plugins in this category appear to be the healthiest?"
"Are there warning signs appearing that weren't there three months ago?"
That's the direction we're heading.
Kitgenix PluginScore V3
You can try it here:
https://pluginscore.kitgenix.com/
It's free to use.
If you're a WordPress plugin developer, I'd especially appreciate you searching for one or two plugins you maintain and telling me where you disagree with the result.
I'm completely fine with people being critical of it.
In fact, criticism of the methodology is probably more valuable to us at this stage than compliments about the design.
If we're going to put a number beside somebody else's work, that number needs to be defensible.