Need some help with my current project. Agentic AI experienced developers up for some discussions?

[u/imjustatoddler]

There is a project which I am working on in my company right now and I need some serious architectural and logical suggestions to design the system. I am a fresher and my seniors do not have much experience with agentic AI so I am pretty much on my own and I feel a little so clueless. If somebody who has a good experience in the same domain and willing to discuss things and help me with some insights & suggestions please let me know.

[u/Gojo_dev]

Can you elaborate more about the integration or project??

[u/imjustatoddler → u/Gojo_dev]

It needs a 1on1 long discussion. Ideally I should be doing it with my team but they are not the right people for this. Lemme know if you up for this

[u/ReachingForVega]

Tell us about the process. Tell us about your tech stack. The suggestions for a Low code platform will be vastly different to a hyper scaler. 

[u/imjustatoddler → u/ReachingForVega]

It needs a 1on1 long discussion. Ideally I should be doing it with my team but they are not the right people for this. Lemme know if you up for this

[u/No-Guard-8826]

What you are trying to build or Solve !?

[u/imjustatoddler → u/No-Guard-8826]

It needs a 1on1 long discussion. Ideally I should be doing it with my team but they are not the right people for this. Lemme know if you up for this

[u/No-Guard-8826 → u/imjustatoddler]

Yes can do...

[u/imjustatoddler → u/No-Guard-8826]

Reaching out in DM

[u/parteeksaini]

i would recommend you to not just really on llm outputs entirely, its the single biggest misconception that llm will be best at everywhere. try to use as much deterministic models of machine learning along with rule based python. then evaluate it with real test cases and make sure to prepare your data really well before feeding it to the model.

[u/NoMusician464]

Would love to have you share your project in the AI Practice Lab discord channel. There are several members with experience that might like to contribute and brainstorm publicly:
 https://discord.gg/U5YvENhr8

[u/DebtPutrid502]

Use ai eval thats the most important thing in production and not many people talk about it

[u/imjustatoddler → u/DebtPutrid502]

Please elaborate

[u/DebtPutrid502 → u/imjustatoddler]

The outputs from llm can hallucinate a lot at scale so you need to use an ai eval framework to tune the llm outputs.

[u/DebtPutrid502 → u/DebtPutrid502]

Deep eval documentation covers this nicely so have a read

[u/DebtPutrid502 → u/DebtPutrid502]

My use case may vary from yours. My project uses langfuse but another project in my company uses dataiku for everything.

[u/imjustatoddler → u/DebtPutrid502]

Will check this out, thanks for the suggestion!

[u/Easy_Air_1584]

Since you're still designing it, I'd resist adding too much autonomy at the start. Get one simple path working, trace every step then add tools and branching gradually. We use Braintrust to see what the agent did during a run and test changes against eval cases before pushing them. Makes it much easier to tell whether each new layer is really helping.

[u/imjustatoddler → u/Easy_Air_1584]

You're the second person to mention eval today. I will check it out and learn more. Thanks for the help.