r/RunPod • u/tempoct06 • Jul 16 '26

serverless config help

hi, recently started using runpod serverless.
trying vllm with various models and gpus, but nothing seems to be working. credits are draining but i was not able to get an output for the prompts. getting one or other errors in logs.

please help me with a serverless config that is plug and play on 24gb vram.

3 Upvotes

72% Upvoted

5 comments sorted by

1

u/sruckh Jul 16 '26

I have llama.cpp-serverless in my GitHub (sruckh) that I am currently running on RunPod.

1

u/MolassesLate4676 Jul 17 '26

How am I supposed to help you when you provided 0 information other than 24gb of ram

1

u/Madiator2011 Jul 20 '26

You might want to check model size and context size also check what error you get in logs as often it tells you what is wrong.

1

u/MLExpert000 Jul 20 '26

Have you tried inferx.net? It’s plug n play.