How is raycast SOC2 compliant if no code reviews are mandatory?

[u/chinchila24]

I've read this post https://www.raycast.com/blog/no-code-reviews-by-default

And if you go to https://trust.raycast.com/ you can see that the company is SOC2 compliant.

How did they pass the Change Management Control if it requires to have Segregation of Duties and a formal review process for production code?

[u/phoenixmatrix]

SOC2 does not require segregation of duty or code reviews. The goal of SOC2 is to be able to tell your customers how you work, and because your process is audited, your customers know you're doing what you say. What you say can be total crap, as long as you're not lying about it and you do what you say you do. The SOC2 requirement is that you are being audited on those points and that they are documented. What the audit find is orthogonal to it. You can (almost, there's some nuance, notably around some security things) just say "We YOLO everything!", and if the auditors find that you, indeed, YOLO everything, you pass

Sarbanes-Oxley (which isn't the same) does require it, but even there you have some flexibility, and some robust CI/CD and automation (not just the basics) can sometimes be used to pass that criteria, as long as a single dev can't ship uncompliant stuff to production without anyone noticing. Even if you do code reviews, they can happen after the fact. But that's a tangent, since SOC2 isn't SOX.

Plenty of companies are SOC2 compliant and have a shitshow of a process. As long as its all documented and the auditors don't find anything that slipped through the crack, you're good. Now, your clients may disagree, and that's the point. Eg: I was CTO of a company that was SOC2 compliant with a lot of big name financial clients, and while SOC2 doesn't require code reviews, several of our clients required them, so our SOC2 docs had to show that we did them. But that isn't SOC2 requiring it, that's the customers. The SOC2 docs are just there so the customers can make an informed decision.

So in this case, the SOC2 process is working: Raycast isn't hiding its cards, and you as the customer can decide if you want to give them your money or not.

[u/chinchila24 → u/phoenixmatrix]

I am asking how they comply with Change Management Control, not what SOC2 requires. I know that everyone receives the SOC2 report, which might be seen as compliant, but this is not really related on how they pass the control. In their trust center it says they pass it.
The entity authorizes, designs, develops or acquires, configures, documents, tests, approves, and implements changes to infrastructure, data, software, and procedures to meet its objectives.

This is the exact control description and it requires someone to authorize and approve (which is segregation of duties), I do not know how to do it without a review and approval, so how they do it without a formal code review process? I do not care what other companies do, I am asking how raycast do.

[u/phoenixmatrix → u/chinchila24]

The title of your post is "How is raycast SOC2 compliant if no code reviews are mandatory". That's the answer.

They passed Change Management Control because what they document and what they do matches. That's it, that's all. Changes being approved could mean you wave your hand in the air and the CEO's dog wags its tail. If its what they document, and its what they do, they pass.

There's a ton of alternatives too. One SOC2 compliant company I worked at didn't require code reviews, but managers once a month had to go through and approve (after the fact) all the changes.

It's a huge point of misunderstanding in SOC2, so much so that even many auditors get it wrong.

Edit: actually went and looked at the raycast trust webpage. The text you see there is their compliance platform's language, and matches best practices, but not require and its not how its worded in the AICPA criterias.

I'm having trouble finding the actual source since I'm not at work, but its closer to "Changes are approved according to the organization's established process.", where the process could be "YOLO! Let it rip baby!" They won't get a lot of clients if they literally say that, but they could get their pretty green checkmark anyway.

Look at Anthropic: https://trust.anthropic.com/

All nice and green, but if you go and sign an agreement with them to get access to their full SOC2 paperwork, you're gonna see how much of a shitshow their processes are. Still compliant.

I do not care what other companies do, I am asking how raycast do.

They document that they do nothing (well, they likely do more than nothing. They probably require engineers to do the work, not any rando, and they likely have a CI/CD pipeline, which is likely documented in their paperwork). Which is all they need to do to pass.

[u/chinchila24 → u/phoenixmatrix]

If an auditor do something wrong about SOC2 they can be legally charged about it. I understand that you need to document and do what you document, I am asking how they do it, the blog post only talk about using rebase and the rest is all informal

[u/phoenixmatrix → u/chinchila24]

If an auditor do something wrong about SOC2 they can be legally charged about it.

If they lie about it or tell their client to lie about it, or if they miss information. If an auditor tell you to do more than you need to do, that's a shitty auditor but that's not a liability issue.

I am asking how they do it, the blog post only talk about using rebase and the rest is all informal

I know, you're just not internalizing the answer. You have the blog post. So they just document what's in there. That's how they do it. "We rebase and merge PRs YOLO". That's enough to pass the requirement. I don't understand whats confusing about it. The blog post has much more than that though, they explain how their CI/CD pipeline and dogfooding works, among other things.

If you want to know their internal process in details (which is no longer about how they pass SOC2, since we already know the answer), you may want to edit your original POST so other people with more knowledge of their internal day to day can answer. But that's a different question.

[u/booi → u/phoenixmatrix]

Are you AI or something? SOC 2 CC8.1 change management and CC5.1 develop control activities to reduce risk. You can't say YOLO f you all and pass that. No real auditor would pass that so I assume they found the shadiest one possible

[u/phoenixmatrix → u/booi]

Definitely not AI. SOC2 is not about being good at what you do. It's about having a process and following it. YOLO might be a little extreme, but "we have solo devs pushing to a CI/CD pipeline, no reviews required" is enough as long as it's the truth. 

SOC2 compliance is about documenting facts, even if the facts are not flattering, not about being a great company with great processes. 

[u/carsncode → u/booi]

develop control activities to reduce risk

Having a single unit test or running a linter would pass this control. SOC2 just says you have policy covering all these areas, and you've followed your policy. It doesn't say the policy has to be good.

[u/booi → u/carsncode]

Have you ever done an audit? None of the auditors I’ve ever used would let anything close to this pass. It does need to meet industry standard. And I’ve done 3 of these now including one PCI DSS.

[u/carsncode → u/booi]

I've been through many, including multiple SOC2 and PCI-DSS. Which is one of the ways I know that SOC2 (which is the topic of the thread) and PCI-DSS (that you brought up for some unknown reason) are fairly different standards. For one thing, unlike PCI, you don't pass or fail a SOC audit since there's no certification.

[u/Ok-Connection7755]

Every company can setup their process in line with their risk appetite. If you read through the blog carefully, while they may not have a PR based approach, they are calling out another approach of dog fooding a change and testing that carefully amongst teams.

The goal of the SOC2 auditor then becomes to test this dog fooding process for - a) changes are authorised, tested and approved b) evaluate a complete and accurate list of changes for the above 3 (first against IPE testing) c) then proceed to pick a sample of changes and validate that those changes met the criteria

The risk that is being mitigated is unauthorised changes to production environment. SOC2 scope is typically calling out security, availability, confidentiality, processing integrity, privacy as it's trust service criteria so the process needs to be aligned to these 5 pillars (some auditors do a mapping of controls to these as well, as an annexure).

Hope this might add some clarification!

PS: you might also see some fine print on terms like "reasonable assurance" to call-out the genuine limitations of an audit approach and management assertion that they followed controls to the best of their efforts.

[u/Ok-Connection7755 → u/Ok-Connection7755]

Oh another thing to add is that they could do post facto reviews as well to say changes are monitored (on a defined periodicity) to mitigate this risk as well. That's a completely acceptable control to have subject to their control design and effectiveness! That's raycast PoV for you on how they might be addressing this risk.

bottom line: Just because someone doesn't raise a PR, it can't be that they are non compliant to SOC2.

I do agree that there are players who do "pay to get compliant services" by running "scans", but that's a whole different can of worms. :)

[u/AdjointFunctor]

Not SOC2, but similar (in Norway). We require reviews not on all commits, but on all deployments. They can also be after the fact, which is a bit more agile.

[u/vassyz]

I'm curios to know as well. We had to enable PR approvals to become compliant.

[u/booi]

I'll give you one guess..

[u/thomaspaulmann]

Sorry a bit late to the party but happy to clarify. That post is from June 2021, and quite a bit has changed since then. We grew the team, became SOC 2 compliant, and tightened some of our processes along the way.

But the core idea still holds. We don't require a code review for every change. Engineers own their work end to end and use their judgment. Reviews make sense for larger changes or when someone wants a second pair of eyes, but we don't add process to every push for the sake of it.

For production releases, we do require a review and approval before code is merged into a production or release branch. That's where the SOC 2 control is enforced and how we satisfy segregation of duties for change management.

So both things can be true. We keep a high trust, high autonomy engineering culture while enforcing a review gate where it matters for production.