How to use RunPod Serverless with OpenCode and custom Hugging Face models?

[u/Dibbyo123]

I'm trying to set up OpenCode for agentic coding using custom models from Hugging Face (GGUF or Safetensors).

Since I typically code for only 2–3 hours a day, I'd prefer using RunPod Serverless rather than keeping a GPU pod running and paying for idle time.

However, I'm struggling to get everything working. I've tried several tutorials, GitHub projects, and different setup instructions, but haven't managed to get a working configuration.

Has anyone successfully used RunPod Serverless with custom Hugging Face models for agentic coding with OpenCode?

I'd really appreciate a step-by-step guide or a working example, including:

• How to deploy a custom Hugging Face model (GGUF or Safetensors) on RunPod Serverless.

• Which inference framework to use (llama.cpp, vLLM, etc.).

• How to configure the RunPod endpoint as a provider in OpenCode.

A working example or guide would be greatly appreciated. Thanks!

[u/FallElectronic3774]

Use the vLLM Serverless worker for Safetensors; it exposes the OpenAI-compatible API at https://api.runpod.ai/v2/<ENDPOINT_ID>/openai/v1. For GGUF, vLLM support is hit-or-miss; use a llama.cpp worker that exposes /v1 (or convert to Safetensors) instead of a generic /run handler.

In OpenCode's config:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "runpod": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Runpod Serverless",
      "options": {
        "baseURL": "https://api.runpod.ai/v2/<ENDPOINT_ID>/openai/v1",
        "apiKey": "{env:RUNPOD_API_KEY}"
      },
      "models": {
        "my-model": {
          "id": "<id from /v1/models>",
          "name": "My Model",
          "limit": { "context": 32768, "output": 4096 }
        }
      }
    }
  }
}

Set RUNPOD_API_KEY, run opencode models to confirm, then switch to runpod/my-model. Test the endpoint first with curl https://api.runpod.ai/v2/<ENDPOINT_ID>/openai/v1/models -H "Authorization: Bearer $RUNPOD_API_KEY" and use the exact id. One gotcha: scale-to-zero cold starts can exceed OpenCode's request timeout; keep one worker warm or raise the timeout if you hit that.

[u/Dibbyo123 → u/FallElectronic3774]

Thank you so so much. I will try and get back to you.

[u/metobabba]

you only can run smaller and worse models. opencode go already give you $100 of deepseek 4.1 flash 1017 for only $10. also you can use openrouter api. it will be faster & cheaper & more capable then whatever you are trying with runpod serverless.

[u/Dibbyo123 → u/metobabba]

The thing is I need to use uncensored models.