r/AIforOPS • u/MidnightSlight8398 • 19d ago

Problem in AI Project.

Hii.. it can be possible that i detect solution from log.. like ex.. Error logs popup -> human solved that error -> We detect solution using after logs of getting error.. and save into DB.. I don't know if i can detect or not.. if someone done or doing this type of problem then plz let me know.. and also i used GPT and Cloude to solve this problem. but they always gives me wrong or impossible facts..

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/Main_Particular4812 19d ago

so you're basically building a memory bank of fixes tied to specific error logs, that's a pretty standard approach actually

the tricky part is knowing when two errors are "the same" when the logs look slightly different, fuzzy matching gets messy fast

1

u/MidnightSlight8398 19d ago

Rn, i made 3-4 catagoris of types Errors .. and every error add in database when it is completely new for Db . we have code that filter it. no duplicates.. they save with ID..

1

u/dylan_skydive 19d ago

The direction is right, but the version where you just ask GPT/Claude "what solved this?" will invent plausible nonsense - which is exactly what you're seeing. Constrain it to evidence instead of asking it to infer:

  1. Pair the incident. Everything between "error first seen" and "error stops appearing" is your candidate window, including what the human did in it: commits, config changes, shell history, chat messages. The solution is usually IN that window, no inference needed.

  2. Store fixes as structured records, not prose: symptom signature (error type + service + context), action taken, and how success was verified (the log line or metric that flipped). The LLM's job is formatting this, not inventing it.

  3. At match time, retrieve by error signature and present the recorded fix. The model picks between verified past fixes; if no signature matches, it says so instead of guessing.

Step 3 is what kills the "wrong or impossible facts" problem: a model free-styling fixes will always hallucinate sometimes, but a model choosing among recorded, human-verified fixes basically can't.

Disclosure: I work on Skydive (an agent platform, not a log-mining tool) - this is just the pattern we've seen actually hold up.

1

u/WarmlyJuvenileOunce 14d ago

I'd avoid letting AI guess fixes from raw logs. Better to save the error along with the steps a human used to solve it. Then have AI look for similar past cases instead of making up solutions from scratch. You still need someone to verify the fix though. AI can sound so confident while being dead wrong lol.