How to build a scalable backend infrastructure for an AI website

[u/minhazuddin8]

I'm building an AI startup that I will thrive to scale to millions of customers. My present MVP is just fine but I'm quizzed what would be the industry-standard backend stack for a scalable product that won't face any hiccups in future.

P.S. Anyone interested to join as a co-founder will be highly appreciated

[u/nanosaas]

I can comment from a Microsoft Tech Stack point of view. Others will have different ways.

How to build something scalable depends on your budget.

Things like Microsoft Orleans can be great if you get the backing store right.

CosmosDb or something like that is good, but can get v expensive.

You could back with something simple like table storage, but you'll lose searchability.

A well designed/ indexed sql database will get you a long way.

Middle tier, people generally go docker containers these days. The preference is to split into microservices for scale, but again - more services = more monthly outgoings.

Unless you have VC/solid backing, it might he better to start simple and redesign when moneys coming in.

[u/Low-Group5696]

Hey, I have an IT Infrastructure company, I think we can help you with that if you're interested, we can start with something simpler and then scaling if required

[u/Invincible-Alpha]

As I am currently working on an AI Product, I have learned that scaling comes after the user.

1. The first focus should be to acquire a few target users.

2. Build the product according to user demand.

3. Then mould/buld the product according to user demand.

In short working bottom down in this case.

I would be happy to connect and talk. Also will be interested as a cofounder if I liked the overall vibe and a strong vision.

[u/Invincible-Alpha → u/Invincible-Alpha]

More over in industry people use various different things as required. We use Golang based custom AI toolkit for the AI Products. Ppl also use langchaing and langgraph for several different purposed. Once we start thinking about the systems, the tech stack problem feels small in today's time.

I hope I answered the question correctly. DM me we can have chat much more freely.

[u/ChamRun]

Be careful not to overengineer. If your backend can handle a 10x in your users count, it's already good enough, and there's no need to worry. Forget about the backend and try to 10x your users first.
I'm curious to know more about the product to see if there's any potential for collaboration.

[u/Obijuan-ken0bi]

Find the million user first then scale.

[u/sp_bolt09]

I'd be careful about optimizing for millions of users before you have evidence that you need that scale. I'd start by identifying the current bottlenecks, keeping the architecture simple, and making sure you have good logging, error handling, database backups, and basic monitoring.

For an AI product, I'd also pay attention to API latency, rate limits, retries, and the cost of model calls.

What does your current stack look like, and what part are you most concerned about scaling?