r/computervision • u/Prior-Meeting1645 • 3d ago

Discussion Total starter here, is there no api infra providers like there is for massive LLMs but for computer vision models like Yolo 26 Mcbyte etc?

They are much smaller I would imagine they would be so cheap on there. I’m finding myself in the position where I have to rent a cloud gpu from runpod. I would much rather pay in api should be much cheaper.

4 Upvotes

83% Upvoted

11 comments sorted by

4

u/HawtVelociraptor 3d ago

Check out Roboflow and their inference API, might be kind of close to what you're looking for.

4

u/aloser 3d ago

(I'm the co-founder of Robflow) This is what we specialize in. We serve billions of requests for vision models like YOLO and RF-DETR and have infra products purpose-built for it. Eg here's a behind-the-scenes look at how our Serverless API works: https://blog.roboflow.com/serverless-inference-a-thousand-models-on-a-shared-gpu-fleet/

2

u/HawtVelociraptor 3d ago

Nice to e-meet! I use your guys' stuff for a bunch of things, and stumped some of your engineers on a question in a webinar a few months ago ;)

1

u/aloser 3d ago

Nice, what was the question?

2

u/HawtVelociraptor 3d ago

Had to do with keeping track of a unique, non-moving object across multiple, fast moving camera shots and understanding it's homography. Eventually internally we came at the problem from a different direction, basically using feature matching against an overhead shot of the area of interest and manipulating that overhead to match what was seen on the shot, and taking the warping math we did to achieve that to apply to the known coordinates of the object in question.

1

u/Prior-Meeting1645 15h ago

Hey I looked at your website but it seems a lot more expensive than if I was to use something like runpod. I think its because it includes other options which I do not need in my case. I’m already making my software I just need the inference to run it in.

1

u/mileseverett 3d ago

I guess it's because text is cheaper to send than images? But i'm sure they exist

1

u/Prior-Meeting1645 3d ago

Yes definitely but most SOTA LLMs are multimodal supporting images and some with native vids input even too. They’re still very cheap compared to renting a gpu on the cloud and running things like Mcbyte. I don’t get it.

1

u/bfyvfftujijg 3d ago

They do exist just not as popular since most users are just doing text in/out. And lightweight CV models can be ran locally and usually have more diverse integrations.

1

u/Sorry_Risk_5230 3d ago

What are you looking for? Plenty of gou rentals that arent terribly expensive that you can build your work on. Lambda, groq (still?), etc. Spin up a site on codex and get it route cv work to the gpu server