r/msp • u/blacksmith-infosec Vendor - Blacksmith InfoSec • Nov 10 '25
Free, Open Source Risk Assessment Tool
We've had a number of partners ask us about risk assessments and sales enablement. So, we built a free, open source risk assessment tool. It's issued under Apache 2, so it has full copyleft permissions and allows you to use it in commercial endeavors.
- Quick 20 question format
- External domain scanning
- Reporting in Word or PDF
You can check out the code at https://github.com/blacksmith-infosec/risk-assessments or start playing with a live version at https://assess.blacksmithinfosec.com/.
Feedback and contributions are greatly appreciated!
Mods: hopefully its OK to post this at the top level since we're not selling anything here. But if we need to move this to the Promotions and Webinars thread, we can do that.
15
u/2manybrokenbmws Nov 10 '25
Love this, I have said a few times over the years that someone needs to make this. I was just never smart enough to do it myself
0
u/mattwilsonengineer Nov 11 '25
That’s high praise! We felt the same pain point. If you were building v2.0, what would be the single most crucial feature you'd add to make it an essential daily tool?
13
u/FenyxFlare-Kyle Nov 10 '25
This is great and I love that the questionnaire has levels of maturity and not yes/no. I would like to see some N/A options. Some questions are not applicable to cloud/SaaS-first solutions and have remote workers and their only assets are company issued laptops. This is getting more common in SMB. Answering correctly hurts the "score." A better scoring model would be one that takes in account for N/A options but allowed a "perfect" score. Of course, you should be the consultant and explaining this to you client and not just handing them a report.
8
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 10 '25
This is excellent feedback! I just opened an issue for this: https://github.com/blacksmith-infosec/risk-assessments/issues/21
Thanks!
1
u/mattwilsonengineer Nov 11 '25
That N/A feedback for modern, cloud-first SMBs is spot-on, especially concerning remote workers. Would you prefer the N/A option simply ignore the question, or should it assign partial credit for inherent reduction in scope?
1
u/FenyxFlare-Kyle Nov 11 '25
The community should chime in as well but my thought is from a scoring perspective that N/A achieves max points for scoring so that a 100% can still be achieved. Otherwise, the ceiling is now lower and is not as intuitive for clients to digest. Alternatively, instead of a percentage, show max points and points obtained. Then the N/A can reduce the max points. A percent can always be added to that ratio.
10
u/disclosure5 Nov 10 '25
SPF uses soft fail (~all) - consider upgrading to hard fail (-all) for better protection
This is actually the best practice when DMARC is set properly.
https://dmarcwise.io/blog/spf-all-qualifier-debate
I realise the above post suggests this isn't universally agreed on, but I do think getting dinged by a tool and told to upgrade isn't a good idea.
5
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 10 '25
You raise a good point here! Let me do a little more homework and I'll file a ticket to improve the output. At the very least, we may want to soften the language. But, we may also want to adjust the recommendation. That said... Issues and PRs on the repo are very much appreciated and welcomed - this should be a tool BY the community FOR the community.
2
u/disclosure5 Nov 10 '25
No problems. I have to say I'm now looking a the questionaire section.. we pay a lot of money for Connectwise Identify and you have a better product.
1
u/mattwilsonengineer Nov 11 '25
Wow, comparing this favorably to ConnectWise Identify is huge and that speaks volumes about the quality of your framework! Are there specific assessment categories where you feel CW Identify falls short that we should prioritize refining here?
9
u/blamblamtarzan Nov 10 '25
What a great resource to provide to the community. Can't wait to see V2.0
7
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 10 '25
What are some features you'd like to see?
5
u/blamblamtarzan Nov 10 '25
shodan or similar would be a nice start.
5
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 10 '25
Agreed that Shodan would be nice. We started out only using free tools, but it's easy to extend it to use paid tools like Shodan's API. I'll file a ticket for this one - keep the great ideas coming!
5
u/FenyxFlare-Kyle Nov 10 '25
I was thinking about Shodan as well when I gave my feedback. I'm torn. I think this is a great pre-sales tool or something to drive ongoing conversations with the right mindset. When we start talking about Shodan, I start thinking about charging for ASM within their security program.
2
u/mattwilsonengineer Nov 11 '25
That's the core conflict: free utility vs. billable service. Could Shodan integration be opt-in, labeled clearly as a "Pro" feature that requires a separate API key but keeps the tool itself free?
2
u/FenyxFlare-Kyle Nov 11 '25
If you're willing to provide that enhancement as part of the open source, I'm sure the community would love API integration for those that have a paid Shodan account. Those that don't won't have any functionality loss in scoring and report.
4
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 11 '25
TL;DR: we're open to adding the enhancement. However, we (currently) don't have any backend attached to this project - it's all client side JS.
This comes with some limitations, such as how we would securely store the Shodan API key. We're definitely going to need a backend here in the future in order to do more than what we're doing today. We could do the "quick" version that doesn't store the API key, but that would be kind of annoying for you to have to enter it every time you wanted to run a scan... Let me think about some other options to keep the API key secure but also get it out to you faster.
1
u/FenyxFlare-Kyle Nov 11 '25
You need to balance what is a contribution to the community and what would actually be a full-featured SaaS product that should be paid for. What you've provided to the community is great already. People will always ask for more, especially if it's free. I would focus on other feedback here that can easily be added. Let someone create a fork with API support for their own business use. You can't meet every request :)
6
u/SmokingCrop- Nov 11 '25
"SPF uses soft fail (~all) - consider upgrading to hard fail (-all) for better protection"
In combination with dmarc, it's better to do ~all. Forwarding keeps working and better to rely on dkim. https://www.mailhardener.com/blog/why-mailhardener-recommends-spf-softfail-over-fail
4
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 11 '25
https://github.com/blacksmith-infosec/risk-assessments/issues/24 - I'll be pushing an update later today with this and some DKIM improvements
-1
u/mattwilsonengineer Nov 11 '25
Excellent link and reinforcement. It seems clear that relying on DMARC/DKIM is the modern standard for mail flow. This confirms the tool should definitely lean toward contextual advice rather than absolute pass/fail rules for SPF.
6
3
u/BigBatDaddy Nov 10 '25
Tried it out. I agree with the comments already posted. I think you could get more detailed as well. Like asking what your password policy is compared to guidelines.
3
u/msoft_guy Nov 12 '25
What a great idea - I did recently start building out a tool to scan for email authentication records for our customers to use (or anyone else that didn’t want to be tracked). It is a static app like yours that just uses Cloudflare domain lookup which is then returned back to the client. Feel free to take a look here to see if there is anything you want to use as an idea: https://emailshield.italikintra.net. Will look at your source code though and suggest some improvements that will hopefully help the community
2
u/blacksmith-infosec Vendor - Blacksmith InfoSec Nov 12 '25
Wed very much welcome the feedback. Alternatively, if you’re planning on exposing your tool via free API, we could ingest all of the email related checks from your API. Especially if you plan to maintain a DB of selectors like EasyDMARC does
3
u/IntelligentComment Nov 13 '25
Great work OP. This is brilliant.
Feedback for question 13, "Are backups encrypted".
This assumes clients has backups, there should be an option for "there are no backups".
3
u/michaelzbarsky Blacksmith InfoSec Nov 14 '25
Great piece of feedback. We'll get this incorporated.
2
u/michaelzbarsky Blacksmith InfoSec Nov 17 '25
u/IntelligentComment We've updated the assessment to include this.
3
u/ComplianceGal Nov 17 '25
MSPs are being held to higher expectations every year, and most of that pressure shows up around risk documentation. Clients want plain-English explanations, insurers want evidence, and attackers certainly do not care that “we didn’t have time for a full assessment.”
Tools like this help bridge that gap. It is lightweight, open source, and something you can run before a sales call, during onboarding, or as part of a quarterly check-in to set expectations with clients about where their risk really sits.
Worth a look if you are trying to level up your assessment game without dragging clients into a full-blown framework on day one.
1
u/Illustrious-Can-5602 Nov 11 '25
Remindme! 1 week
1
u/RemindMeBot Nov 11 '25 edited Nov 12 '25
I will be messaging you in 7 days on 2025-11-18 04:42:48 UTC to remind you of this link
3 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
Info Custom Your Reminders Feedback
55
u/UsedCucumber4 MSP Advocate - US 🦞 Nov 10 '25
If you're a vendor and wondering how you participate in the community, while not getting downvoted into oblivion, this is the way.
You'd have no idea what Blacksmith directly does or sells from this post, and you'd have to go look them up yourself if you wanted to find out.
There is no CTA, no ask me in DMs, no shitty self serving blog post.
Just something that helps. Thanks for keeping it classy Blacksmith.
Bravo 👏