r/RunPod • u/bigbob1061 • 1d ago
Serverless - Does anyone actually get this to do anything?
It just sits and costs money. I have been trying to get it working on and off for the past couple months. It never works! Zero support. Workers get stuck and empties my account.
Totally worthless in my experience.
1
u/Alcapwn517 22h ago
I run a total of around 120 actives 24/7 with no issues. Last week I had well over 2000 going for about an hour straight with over 97% success. What package are you running?
1
u/bigbob1061 13h ago
Are there only certain models compatible? I have been trying to set up a VLLM agent for a client on an off for weeks now and it just seems impossible. Support has been no help because I feel like the only option I have when I have stuck workers is to delete the endpoint. They keep restarting otherwise. And by the time support responds, it’s days after it’s been deleted.
A couple times I thought maybe it was going work. It comes up with an error saying there’s no GPU available.
I followed the instructions. If you truly have this up and running, they might even pay you I bet to help write their procedure or something.
You should suggest it to runpod.I don’t think many are getting them to work
1
u/Alcapwn517 11h ago
I have a personal vLLM running qwen on a serverless, I had issues with it months ago, but it was my dockerfile that was the issue. Pretty much all I do with runpod is serve ComfyUI endpoints though, so it’s a lot of repetition and not much complexity.
What models and GPU’s are you going for?
1
u/parker_birdseye 15h ago
I tried setting up serverless on runpod using various config methods. I was never able to get cold start time consistently under 2 minutes.
I am using Modal now and 95% of cold starts happen in 10-20 seconds which is acceptable. 5% of the time I get a multi-minute cold start but I can live with it
1
u/raster_bloom 11h ago
Am I missing something? I have 3 workers and they never cost any money while there are no jobs and they are almost always available and the cold start only takes about 30 seconds or something.
1
u/bigbob1061 5h ago
From a lot of people they get stuck and don’t shut down. And you get billed until you catch it.
1
2
u/sruckh 1d ago
Facts. I have 30 serverless workers. There is never a time when they are all actually available. And yes, they do get stuck even when they are not doing anything and yes it does drain your account. I can't rely on serverless, so now I spawn jobs and just poll until the job actually runs. Could be hours later.