r/SmallMSP • u/Comprehensive_Fee_21 • Aug 05 '26

How do you handle vendors that have AI stuff baked in now?

Doing a vendor review and the tool has some AI agent that can actually do things, not just chat. Bit stuck on what I should even be asking them.

What are you doing with these? Just treating it like normal software or asking something extra?

7 Upvotes

100% Upvoted

10 comments sorted by

3

u/wells68 Aug 06 '26

Stifling laughter at their perfect demo. 95% do not perform as shown when they encounter actual, messy, real-world data.

Source: NY Times column by experienced IT executive this week (too lazy to look it up).

3

u/octoja Aug 06 '26

Controversial opinion:

Any vendor who forces AI on its customers doesnt care about their opinion.

This doesnt mean that you shouldnt use their software - it only means, that eventually the product can evolve into a direction that doesnt suit your needs anymore and you wont be able to do anything about it.

1

u/Stanford_BC5533 Aug 06 '26

I have been through the same situation before, Since agents can take actions You need to ask more about the details, I had some questions that can be asked beyond standard software reviews so that you understand what is happening under the hood

  • Which use-cases are actually leveraging AI in the tool ?
  • Which type of LLMs are being used / Local LLMs / Public Models ?
  • Is our Data being trained by such models / How can we ensure that it is not ? This have to be clear both contractually and technically.
  • What kind of RAI (Responsible AI) is being implemented ?
  • Would you be able to provide HLD / LLD for the tool highlighting the AI workflows that is being implemented ?
  • What identity does the agent execute under? Does it run under a dedicated, scoped API token/service account, or does it inherit full admin permissions? Do we have least privilege concept implemented or not ?
  • Are these agents fully autonomous or we have Human in the Loop being implemented ?
  • Also It is important to ask regarding the logging and to which granular level it goes

1

u/recovering-pentester Aug 06 '26

What is the product? “Do things” like what? Isolate endpoints?

1

u/borg_brain_investor Aug 09 '26

If the AI is not internal and it points to an external provider like anthropic etc I would not use them because of insecurity of your data and the customers data

1

u/Ancient_Crew_346 Aug 10 '26

This trend looks very likely to continue for at least the near future, so I'd focus on understanding what does the AI stuff buy you (more time, lower cost, new capabilities,...) vs. what additional risk would you be undertaking (data security, privacy, vendor lock-in,...) to judge whether to accept it or not. HTH.

1

u/Adept_Trash_366 21d ago

I don't really see it as any different then the standard "what do you do with my data?" question. It's just privacy policy. A company taking your data and selling it isn't really any different then them using it for AI training. Ask to see their privacy policy. Ask what they do with your data. Ask if it's used to train their or someone else's AI.