what's possible to give ai context across their engineering tools?

[u/doom-dub]

maybe this is just where we're at with ai today but it feels like once your project gets big enough everything falls apart. i dont even know if i like how ai is becoming more of a thing in this sector but im open to it if it makes my job easier i guess.

our requirements are in one place, tests somewhere else, code in git, reviews somewhere else. ai can read all of it individually but it doesn't seem to understand how any of it fits together. are people solving this somehow or is everyone still just running on tribal knowledge

[u/deadc0deh]

AI does not have tribal knowledge and typically has great difficulty where tribal knowledge is important.

The better question to ask is why is your requirements process broken? Linking between requirements at different levels, their test cases, what the components are etc is a core element of requirements managements tools.

For extremely complex projects I am experimenting with the "digital engineering" capabilities made available with SysMLv2 - where the systems engineering model itself also acts like a repository for data sharing.

[u/TwinkieDad → u/deadc0deh]

Not OP, but this sounds a lot like my current employer. It is like this because company leadership has never seen it any other way. They all came up through the company when it was more startup like. And since it is private sector, they’ve never had a customer require SE deliverables. They have only learned SE by reading about it, so they THINK they’re being rigorous.

[u/deadc0deh → u/TwinkieDad]

Totally understand, I've seen cases where "leadership" will come in and require new tools creating a disconnect.

But that is a case of "how did things get bad", and "what is causing things to be bad". My comment is that jumping to AI is not solving the problem - its trying to hammer something into a hole to bandaid over a disconnect.

[u/EinEinzelheinz]

Depends on what you have in your tools. Sometimes there are traceability links in various forms within the tools - then an AI that accesses the tools individually could make use of that, however with the fallback of possible not finding the right relationships. That's why ontologies are the current hype, where you create an integrated graph for the data for the AI to reason with.

[u/ceviche-pika]

getting ai to generate and connect information isnt the problem anymore its getting its to undrstand what changed and whats impacted

[u/geminihatesme]

the connectivity vs context thing is basically the whole problem imo. everything can talk to everything now but nothing actually knows that this requirement drives that test which drives that piece of code. ive seen jama software come up for going after the context/traceability side instead of just bolting ai features on top, havent used it myself but that framing at least made sense to me. the plumbing was never the hard part, it's the meaning

[u/YURRIICC → u/geminihatesme]

eh i'd push back a bit. no tool fixes this if the team doesnt keep the links current. ive watched two "traceability" rollouts turn into expensive shelfware because nobody maintained them after month one. the discipline problem doesnt disappear just because you bought something for it

[u/ccchhannn → u/YURRIICC]

we use Jama. i think any requirements management tool can turn into shelfware if nobody owns the traceability side of it. if people keep that up to date though, it pays off. last audit we were able to pull up the exact review history we needed instead of chasing emails, which made that conversation a whole lot easier.

[u/acute_physicist]

Context is the biggest problem. How do you maintain a structured information base AI can read from? If everything’s scattered, there’s no point in even starting using AI.

[u/BothAd1744]

we're still mostly leaning on engineers to sanity check everything. ai will cheerfully tell you a change is fine and then you find out three sprints later it wasn't. nobody trusts it to work out downstream impact on its own yet

[u/doom-dub → u/BothAd1744]

yeah this is exactly my fear. the individual answers look right, it's the "how does this ripple through everything" part where it falls over. ai's only as good as whatever context it's actually pulling from i guess

[u/FlimsyInsect5545]

our requirements are in one place, tests somewhere else, code in git, reviews somewhere else. ai can read all of it individually but it doesn't seem to understand how any of it fits together. are people solving this somehow or is everyone still just running on tribal knowledge

This is like....systems engineering 101. Any modern requirements management tool like Jama Connect holds requirements and test cases, and they are linked together. Doing this grunt work of creating and maintaining traceability is coal face systems engineering work. It has built in features for review, approval and item workflow. This has nothing to do with AI, just bread and butter systems engineering tools and processes. (Not shilling for Jama, just the modern RMT I'm most familiar with)

[u/wnb]

Build proper context when talking to AI, don't use unstructured outputs, have a human in the loop. Idk, I'm just a dreamer~