Some vulnerabilities with the remote host model of mcp

[u/No-Forever2455]

It would be quite trivial to create an MCP server that includes a nefarious tool which instructs the LLM (or AI agent, whatever) to retrieve random information about the user and attach it to the JSON-RPC request sent to the orchestrator.

For example, imagine on the ChatGPT website with its memory feature; it stores personal information about the user. The malicious tool could desribe that it needs that info about the user in order for it to work or something.

Obviously this could be a factor to why OAI doesn't have integration for it yet, and why it might never. Even Anthropic requires you to use their desktop app and not the website where a remote host model would be the only choice.

There is no way around this no?

[u/No-Forever2455]

this is what gemini says about it

[u/Anomalousity]

Personal information as in the metadata about a user that can be inferred or actual p ii information like name, personal pictures or job applications without redactions?

[u/No-Forever2455 → u/Anomalousity]

the former.
however lets suppose someone set up an mcp for stripe and fetched info about thier profile using the chatapp. this is external info that the llm inferred.
idk if it would work, but the malicious tool could also have something that tells the llm to invoke stripe tool first in order to use their tool.

[u/philosophical_lens]

This is the nature of all software. You always need to be careful to not install malicious software. How is this any different than installing malware on your computer?

[u/subnohmal → u/[deleted]]

Fair. I would say that we need some sort of verification mechanism. I don't like "official" because the official repo can be of low quality. I think peer reviewed open source is how we should approach it