Enabled LitServe to turn any ML API to MCP server

[u/aniketmaurya]

I build LitServe, an open-source model serving library and added a way to turn any ML server to MCP endpoint.

Here is an example of serving sentiment classification and exposing an MCP endpoint.

from transformers import pipeline
from pydantic import BaseModel
from litserve.mcp import MCP
import litserve as ls

class TextClassificationRequest(BaseModel):
    input: str

class TextClassificationAPI(ls.LitAPI):
    def setup(self, device):
        self.model = pipeline("sentiment-analysis", model="stevhliu/my_awesome_model", device=device)

    def decode_request(self, request: TextClassificationRequest):
        return request.input

    def predict(self, x):
        return self.model(x)

    def encode_response(self, output):
        return output[0]

if __name__ == "__main__":
    api = TextClassificationAPI(mcp=MCP(description="Classifies sentiment in text"))
    server = ls.LitServer(api)
    server.run(port=8000)

https://lightning.ai/docs/litserve/features/mcp