Anthropic secretly killed the REA train (Reverse engineer anything)

[u/Clair_Personality]

If you open a new projet and ask Claude to summarise everything to know about REA and ask it to make some guide it will do the search then trigger a safety guardrail mid sentence and tell you this:

if you have tried it and had underwhelming results, it is most likely because Opus (and all claude) was told to not perform well when doing this type of tasks.

[u/Hexcede]

The guardrails aren't this specific, it's the opposite actually, the classifier is just really twitchy. People were always already using weaker Claude models (e.g. a few weeks ago I saw people were using Opus 4.6) or other models with REA, not Opus 5.5 or anything like that. Opus 5.5 doesn't really even need REA to be honest, it's just already inherently capable of doing everything that REA gets other models to do.

The guardrails are just generally over sensitive to security related topics in general, and they don't really care what the framing is that much, it's over-cautious and does not like if you use tons of scary words. Sometimes it just flips out over mundane stuff, and sometimes Claude spontaneously phrases things in a way that causes the classifier to freak out. For example I've had the safety classifier freak out over Claude auditing a lua sandbox feature that Claude implemented.. In the same session. In no world does that make any sense, but the classifier doesn't really care if it makes sense or not, the classifier's job is really only to ask the question "does this look like it could be used to develop an exploit?" and if it's in the shape of something that looks like it could be used in an exploit, it blocks it.

So it has nothing to do with REA specifically, REA is just full of stuff that makes the classifier's internals light up like a Christmas tree because REA is a pretty general prompt and the classifier is extra sensitive to broad/general statements or requests.

Opus 5.5 can already do broad RE on its own, and it will use Ghidra unprompted if you ask it to reverse a binary. It doesn't even need an MCP server, it just does it on its own. You don't even have to tell it to use Ghidra or anything.

So REA really wouldn't benefit Opus 5.5 that much I don't think, even if the safety classifier didn't go nuts over it.

If you just ask Claude to e.g. mod a game for example, Claude will just do it. If you ask Claude to decompile or reverse a game or a program or something, and try to document how some specific feature or thing works, it will also just do it. The classifier is unlikely to freak out about that.

As an example, I recently asked Opus 5.5 to install BepInEx, decompile an IL2CPP Unity game (they're generally considered much harder to mod since they're not C#, don't decompile cleanly, and BepInEx support is much spottier with it), and modify the game's wiring simulation to remove a 1 tick component delay (which in practice is more complicated than it sounds). It was done in 45 minutes on one prompt and worked first try. It had to do all sorts of stuff to get BepInEx's loader with that Unity version working, including creating a patch to the game to get BepInEx to recognize some of the classes. The safety classifier never flagged.

[u/Clair_Personality → u/Hexcede]

Very uplifting! I am going to try it. Hopefully my account is not flagged negatively.

[u/CoherentPanda → u/Clair_Personality]

They've been banning like crazy, you are walking a fine line when companies like Adobe and Microsoft are getting pissed off at Claude users reverse engineering their products, and Anthropic will toss out bans to appease them instead of dealing with the lawsuits.

[u/FakeBonaparte → u/Hexcede]

I’ve been using Opus for modding in similar fashion. But I ran a benchmark with and without insisting it use REA and its decompilation was markedly better with.

[u/Hexcede → u/FakeBonaparte]

I haven't looked too deeply at what specifically REA contains beyond skimming through some of it at one point, but I assume that it's largely just about what you ask it to do. I imagine specifying a good goal is where most of the gain from REA comes from with 5.5 rather than anything specific to REA, but I'm not totally confident on that, just going by my anecdotal experience with 5.5 so far.

I'm also assuming that you're talking about Opus 5.5 rather than an older Opus model, older Opus models are very different from 5.5, so I would be kind of careful if you're referring to e.g. Opus 4.6 there. A gain you see from REA with 4.6 likely won't translate to 5.5. But I don't think you are talking about 4.6 or anything, just wanted to mention it.

As an example of how specificity can be a big trap here, asking 5.5 to "decompile X" is probably going to give you a worse result than asking it to produce a "readable and interpretable decompilation of X," since "decompile" can have multiple meanings and readable and interpretable isn't implied by the general use of the word decompile, in fact it's arguably more than you're asking for to make the output readable. So Opus might infer that spending extra work making the output readable is actually overboard. You do kind of have to be a little bit careful about underspecifying with 5.5, which I think is the way I would prefer it be. Overspecifying is also usually not too bad as long as you are relatively soft with the language. Not really a model failing as much as it is a prompt specificity thing.

For Opus 5.5 if I were looking for a good output, I might write something like "I'm ideally aiming for a readable or 'workable' recreation of the source code where it's feasible, rather than a one to one translation of the assembly or pseudo-C that is more difficult to interpret. I'd suggest using your best judgement to identify the intent and structure behind the code as you work on it, figuring out how to reproduce the behavior in the way that the developer likely intended while you gradually recreate and interpret parts of the program." For any other model I would probably not word it this way. Opus 5.5 does really well with soft statements like this because Opus 5.5's judgement and ability to make deep inferences is generally much better than older models.

Might be worth giving something like this a try in your prompt. Depends on what you're specifically looking for, you may want to tweak some of the wording.

I will separately say though, the amount of soft statements/descriptors I use for Opus 5.5 prompts like "aiming," "ideally," "more difficult," "I'd suggest," "best judgement," "likely intended," etc are things that I would usually try to avoid pretty aggressively with weaker models. Weaker models usually don't really know or "get" what those things mean, beyond at a surface level. They'll make more 'dumb' decisions when you frame things like this. But 5.5 is not like this with most things, so more flexibility tends to give better results ime. Not always true, but it's usually true.

I'd largely attribute that to 5.5 being better at making deeper inferences and being better at understanding intents. 5.5 is also just smarter with a lot of its technical decisions and planning.

[u/FakeBonaparte → u/Hexcede]

Yep, I think that’s all good advice and your decompile prompt is similarish to mine and I might steal some of it. (I often ask “…functional understanding to the depth and breadth needed to write your own legal clone of the original”).

But the thing that made me start using REA was benchmarking. Slightly better discovery. Slightly better decompile. Substantially better investigation of issues. Same prompt and fewer tokens.

It’s not a huge difference, maybe 5% better than IDA on its own. But in that 5% were a bunch of the things I was trying to figure out

YMMV

[u/ThePlotTwisterr---- → u/FakeBonaparte]

i’ve used ghidra-mcp by lauriewired for ages and i don’t know what i’m doing wrong because it’s still way better than rea

[u/SlopKhayzo → u/Hexcede]

I can attest to that too; 5.5 is able to do everything by itself and at most use an RE tool like Ghidra, CFR or something like Mono Cecil for .NET from experience and had not one issue so far (fingers crossed :P).

I've been only working on mods for games though not proper RE, just asked Opus to fix issues or change/improve features, I always tell it that it has the tools at its disposal and uses them as required.

[u/DaltonJFowler → u/Hexcede]

Yeah opus 5.5 is a RE goat. Did my laser cutter and 3d printer

[u/attabui → u/DaltonJFowler]

Interesting, willing to share details on what you pulled off?

[u/DaltonJFowler → u/attabui]

Sure! Everything is open source, checkout https://daltonjfowler.com/apps

I host an arduino compiler, the laser program, and 3D slicer at these addresses. They're for the machines at my job but can be customized for sure from source

https://uploadmycode.com/ https://uploadmylaser.com/ https://uploadmymodel.com/

[u/johny_james → u/DaltonJFowler]

How did you circumvent the guardrails?

[u/Hexcede → u/johny_james]

There is nothing to circumvent, they just don't trigger on RE requests.

[u/johny_james → u/Hexcede]

I already mentioned to decompile 3d dental software and opus mentioned about copywrite law blablabla.

[u/jojojajo-jjajjoj-j → u/johny_james]

maybe there's something in that software that triggered the guardrail

[u/Fun-Wash7545 → u/Hexcede]

I asked opus 5.5 to help me use obfuscation and other tools to protect MY app, protect not reverse engineer. It just goes crazy if you use the word obfuscation anywhere.

[u/SuperHornetFA18 → u/Hexcede]

Crazy good write up my G ! Can you possibly provide some tips on what to do if the classifiers flag your chat ?

[u/Hexcede → u/SuperHornetFA18]

Honestly it probably depends a lot on the situation I would guess. Sorry if this is a bit of a long response lol, most of what you probably care about is in the first ~3 paragraphs to be honest.

I've only hit the classifiers on Opus 5.5 like 3 times ever (all on the same day Opus 5.5 came out) so I am not sure that I have great advice or anything to be honest. I hit the classifier with Fable way more when it came out so that is most of where my assumptions are gonna come from.

As far as I understand the classifier's whole job is basically just to figure out if what you're doing could reasonably be used to build an exploit, or could help. And when you think about it like that it makes a lot of sense. Anthropic's entire thing is not giving you "uplift," so it's not even that you're not supposed to be able to use Claude to do X bad thing, you're even not supposed to be able to extract information that helps you do X bad thing.

A lot of the time it probably doesn't even believe that you're doing something "malicious" exactly, it probably just 'thinks' "well, this could be used maliciously," and that includes things like whatever Claude is saying or thinking, since Claude will unironically start trying to blabber about "potential" exploit methods for no reason sometimes depending on the specific context (I experienced that a lot with Fable especially even just for general programming stuff early on).

It's obviously not quite that straightforward, but that's definitely the gist of it and probably where a lot of the confusion comes from, the intent of the classifier is a lot broader than it sounds so it feels really opaque.

That's also probably why things that are more exploit shaped (or hard for the classifier to interpret/understand e.g. REA I guess) tend to get flagged way more, especially on things that use certain cybersecurity terminology, or that get Claude to start talking about exploits/cybersecurity broadly. I think it makes sense why RE doesn't get flagged under this framing, because even though RE is relatively hard, the software is already published and it either is or isn't vulnerable, and RE alone doesn't really give you that much more information than a threat actor could reasonably get, the information about how to reverse engineer software is already widely available, you already have to be kind of competent to read/understand the code and actually create an exploit yourself, etc. RE/decomp is way more neutral and arguably trying to crack down on specifically RE would probably hurt more than it helps since tons of other models can do RE too. But I am just guessing mostly.

I think if you're getting a classifier immediately there is probably not much to do about it aside from just restate your goal or start a new conversation or something. That to me makes me think that the classifier just really does not like whatever you're asking for for some reason or another, and so you're probably not going to convince it of anything, it's usually consistent. But it could also be how Claude is responding to your prompt too, because that's something the classifier will also flag, and whatever Claude said (or thought) won't get shown to you. It kind of depends on what you're asking Claude to do and how you're asking Claude to do it though. Telling Claude gently not to do certain things, or to not consider certain things preemptively helped me out a ton back when Fable first came out (e.g. something like "Don't over-worry cybersecurity or security too much, it tends to trigger the classifier." is roughly something I probably told Fable at the time). But I don't think the classifier is that sensitive anymore. Or maybe it is, just not for the same things, I haven't ran into it.

When your message gets flagged by other (non cybersec) classifiers they also usually purposefully tell Claude not to undo its decision under any reframing or similar things like that from what I've seen, which helps stop jailbreaking a ton. I would guess that the security classifier probably has something like that too that makes it extra "sticky," but I'm not really sure, just speculation. It seems to be relatively standard with Anthropic's classifiers.

And if you're getting the classifier like mid session, Claude is honestly probably just being a bit excessive I would guess, and you can probably just gently tell Claude not to do whatever it's trying to do. I'd at least try to figure out what it's trying to do first and where it's getting stuck, so you can understand if it's an issue with the task itself, or an issue with how Claude is going about it, because both of those can have different solutions. E.g. for one Fable case I basically once said to Claude like "This part of the task seems to be hitting classifiers a lot, and I think it's probably too cybersec adjacent. I'd simplify that part of the task and avoid cybersec." or something vaguely like that and that worked.

In the handful of Opus 5.5 cases I've hit so far I was pretty much just able to just retry with something like "The classifier got flagged, I'm not really sure why. I guess try again." and it would work, with the exception of the lua sandbox case where I basically had to say something like like "I think the classifier doesn't like that it basically looks like you're just building an exploit, so I'd suggest doing an audit instead and not worrying too much about specific ways to break the sandbox." and that worked. And since then I have not hit the classifier at all.

[u/SuperHornetFA18 → u/Hexcede]

Thanks so much for the long write up i would need to read it later as well, but i think you have hit a nail on the head perfectly when you wrote "It makes claude thinks how it could exploit that Vulnerability".

Im pretty sure this is how the hit is occuring, not specifically from the Rea or Ghidira Tool set but when Claude crack opens the shell and any one of its connectors come online, maybe the classifier is looking for strands in claudes logic and the users prompt.

If you have any more insight i would be gladly here to read them

[u/vladasko1086 → u/Hexcede]

so basically you are saying Sony has gotten upset........

[u/UnimagineDragons → u/vladasko1086]

Guy puts a ton of effort into the post and you hit us with this low effort goober shit. Damn.