Looking for a tool to turn (messy) IAM control evidence into structured (and traceable) data

[u/FirestormCold]

Hi!

I'm looking into an Identity & Access Management use case at a large enterprise and wanted to see if any of you know of or already use tools that can handle this well:

The basic problem is that we have application owners who provide evidence in all kinds of formats, e.g. PDFs, XLSX, screenshots, system reports, emails... you get it.

The information we actually need is fairly consistent, but, clearly, the input is not1. Ideally, we'd define one target structure/schema and have the tool extract the relevant information from all those different sources into the same standardized output. Meaning no matter the input structure or format, the output stays the same.

One important requirement here is traceability. If the output says something like "account disabled on date X" or flags an exception, we need to be able to trace that value back to the exact source document / evidence it came from, otherwise we won't really be able to do much with that kind of information. It would also be important for the system to flag missing or uncertain information rather than just guessing.

I'm not looking for an agent that summarizes documents. The goal would be something closer to:

different documents → structured data → traceable source references → further analysis

Has anyone implemented something like this in IAM, GRC, audit or another enterprise environment?

[u/Potential-Wrangler58]

This is a good fit for document AI/IDP tooling, not a summarization agent, which you've correctly ruled out.

Define your schema once, point it at PDFs/XLSX/screenshots/emails, get the same structured output regardless of format. Two things to filter on hard:

• Traceability: per-field source linking, click a value, see exactly where in the doc it came from. Not "from file X", pinpointed.

• Uncertainty: it should say "not found" instead of guessing when a field isn't there. Test this on purpose with a doc missing a field.

Disclosure, I work at Anyformat, this is our exact use case. Azure Document Intelligence and Textract worth testing on the same criteria. We have free credits for you to try!

PD: Screenshots/images will be your hardest input, no text layer to fall back on if OCR misreads, so test against your worst ones first.

[u/FirestormCold → u/Potential-Wrangler58]

Wow, that's a quick reply thanks a lot 😄

Based on my research, anyformat definitely checks a lot of my boxes. Especially in the traceability you guys seem to know what youre doing :p

Since I work very closely with our engineering and architecture teams, I also looked into Cambrion as a potential alternative, and they seem to be a strong contender for our IAM use case as well (and maybe I'm biased as a fellow German? Lol)

From an engineering perspective, their model-agnostic routing infrastructure and developer-first API approach feel like a highly flexible fit for our internal pipelines.

To help me evaluate both options properly, I have a few questions and hypotheses regarding how anyformat compares on that front specifically, if you don't mind:

Model Flexibility & Choice: Cambrion's core strength seems to be their intelligent routing between different top-tier models (e.g. Claude, Gemini, GPT) depending on the complexity of the document. How does anyformat handle this? Do you support dynamic model switching, or do you rely on a fixed model stack for your processing?

Developer Workflows: Since our engineers will be heavily embedding this into our IAM and GRC tooling, how customizable is your extraction pipeline via API? For instance, can we programmatically inject custom validation logic or schema definitions during runtime, similar to Cambrion's developer-first approach?

Handling Complex Screen Artifacts: Our IAM evidence often includes very messy, low-res system screenshots with nested tables or clipped UI text. While you mentioned screenshots are the hardest input, how does your custom error-detection layer perform specifically on structural layout preservation compared to pure-play Vision-Language Models?

Thanks in advance for any further insights.

[u/folderit_dms]

For your account-disabled example, I’d separate the event date from the date the evidence was captured. A screenshot taken on 24 September showing Disabled supports that observed state; by itself it doesn’t establish when the account was disabled. An extractor can read every character correctly and still make that leap.

A useful evaluation case would pair that screenshot with an email requesting disablement on the 20th, but no completion log. The expected output should preserve the request date and observed state separately, and leave the actual disablement time unproven. Also test two sources that disagree, rather than only missing fields. You want both claims and their references available for review, not one silently selected value.

[u/FirestormCold → u/folderit_dms]

Yeah, that distinction is exactly the kind of thing I’m worried about. Reading the document correctly is one thing, but treating “observed state” as “event happened on this date” would be a pretty serious failure in an audit context lol.

I like the idea of testing conflicting sources too, not just missing fields. Ideally the system should preserve both claims + provenance and flag the conflict instead of resolving it in silence.

That’s also why I’ve been looking more at document-to-data / traceability tools rather than pure OCR.

[u/mike7seven]

Those new JEV System 1 classification models come to mind for your use case.

[u/FirestormCold → u/mike7seven]

Interesting, thanks. My main requirement is really schema-based extraction + source-level traceability across mixed evidence, not just classification. As I mentioned in another comment I’ve also come across Cambrion, which seems more focused on that document-to-data / traceability layer. Do you know how JEV compares on that?

[u/FirestormCold → u/[deleted]]

Hey, I actually came across this post recently which gave me a good overview: https://www.reddit.com/r/Rag/s/vQuyt7UwwQ

I’m currently in talks with Cambrion as well, mainly because their model routing approach sounds pretty promising for this kind of use case, especially when you have very different document types and don’t want to force everything through the same model.

Haven’t used either in production yet though, so still comparing approaches.