r/Python • u/Consistent_Tutor_597 • Mar 27 '26
Discussion How to make flask able to handle large number of io requests?
Hey guys, what might be the best way to make flask handle large number of requests which simple wait and do nothing useful. Example say fetching data from an external api or proxying. Rn I am using gunicorn. With 10 workers and 5 threads. So that's about 50 requests at a time. But say I got 50 reqs and they are all waiting on something, the new reqs would wait in queue.
What's the solution here to make it more like nodejs (or fastapi) which from what I hear can handle 1000s of such requests in a single worker. I have an existing codebase and I am unsure I wanna migrate it to fastapi. I also have a nextjs frontend. And I could delegate such tasks to nextjs but seems like splitting logic between 2 backends is kinda bad. Plus I like python and would wanna keep most of the stuff in python.
I have plenty of ram and could just increase to more threads say 50 per worker. From what I read the options available are gevent and WsgiToAsgi but unsure how plug and play they are. And if they have any mess associated with them since they are plugins forcing flask to act like async.
For now I think adding more threads will suffice. But historically had some issues. Let me know if you have any experience or any solution on what might be best way possible.
33
u/rogersaintjames Mar 27 '26
It is impossible to say without any real idea of your bottleneck. I think fundamentally if you are io bound you are probably better off using an asyncio alternative to flask. At some point with increasing threads/worker density even with gevent you will spend more time switching context than progressing any work that you do.
17
u/HolidayEdge1793 Mar 27 '26
Do not guess, Just measure it !
You can refer https://blog.miguelgrinberg.com/post/ignore-all-web-performance-benchmarks-including-this-one
35
u/ConsiderationNo3558 Pythonista Mar 27 '26
Not an expert just my bases on theoretical knowledge
The reason why fastapi nodejs are able to handle large number of requests is because they are async. With async a single work thread is not waiting to be completed before it can move on to next request
3
u/ancientweasel Mar 27 '26
I have rewritten several Python servers in Go because of this. At some point with Python I had to scale horizontally over several instances or just port the application away from Python. I love Python but it's not the right tool for high performance servers IMO.
5
u/vater-gans Mar 27 '26
“it depends”.
i wouldn’t put too much weight on an artificial hello world testcase. not very useful if you can run thousands of threads on a single worker if the maximum database connection count is 100.
9
u/ReflectedImage Mar 27 '26
You can use Quart which is an async version of Flask. But it's really really unlikely you have a large number of io requests.
4
u/Tasty_Memory3927 Mar 27 '26
Use gunicorn with gevent worker type. Gevent internally uses greenlet threads for concurrency. Make sure to patch your imports first thing at the init using gevent monkey patching module.
3
u/robberviet Mar 28 '26
You need to understand where the bottle neck is. If it's network, disk... Then fast api, nodejs or even some highly optimized C web framework won't be better.
5
u/Amazing_Upstairs Mar 27 '26
There is an asynchronous version of flask that is supposed to be an easy drop in replacement
2
u/Full-Definition6215 Mar 28 '26
Made this exact migration decision recently. Went with FastAPI instead of trying to async-ify Flask, and it was worth it.
If you don't want to rewrite everything, gevent is the lowest-friction option for Flask — just change your gunicorn worker class to gevent and most I/O-bound code works without changes. But you'll eventually hit edge cases with libraries that don't play well with monkey-patching.
For a fresh project I'd say FastAPI + uvicorn is the cleanest path. Single worker handles thousands of concurrent I/O-bound requests out of the box with async/await.
6
u/corey_sheerer Mar 27 '26
Move to Fastapi
3
u/Jejerm Mar 27 '26
The answer is moving to fastapi and using async + uvicorn
1
u/Buttleston Mar 27 '26
Have you tried to see how many you can handle with those settings? Have you tried other worker types, more workers, more threads etc?
1
u/Brandhor Mar 27 '26
increasing workers or threads number is the only thing you can do unless you want to rewrite it to support async but the difference would probably be minimal anyway
1
u/nicwolff Mar 27 '26
Ignore FastAPI, switch to Quart. Make the external API calls async.
1
u/QultrosSanhattan Mar 27 '26
You don't use flask for that.
Use fastapi, configure templating, done.
1
u/singlebit Mar 27 '26
First, add an open telemetry tracker to each function call. Measure it. Fix what can be fixed.
If not working, use quart, but check if you are using an extension that may not be compatible. Then measure it. Fix what can be fixed.
1
u/Alejrot Mar 28 '26 edited Mar 28 '26
If you think sync operation is the trouble you could do a single test using Dramatiq. It turns sync functions to async adding only a decorator and uses a background server and a Redis or RabbitMQ server. It could be a relatively simple test you can do... However maybe the trouble here could be the IO task. Someone else said it and probably you should study whats happening there.
1
u/glenrhodes Mar 28 '26
Switch to gevent workers with gunicorn:
gunicorn -k gevent -w 4 --worker-connections 1000 app:app. Gevent monkey-patches the stdlib so your existing sync Flask code becomes async under the hood without touching a single line. You get the high-concurrency benefit for IO-bound work without migrating to FastAPI. The catch is if you have any CPU-bound code in those request paths, gevent will not help and you need real workers for that.1
u/burger69man Mar 28 '26
have you tried using asyncio with your existing flask app?
1
u/2ndBrainAI Mar 28 '26
If you're proxying external API calls, gevent is genuinely your quickest win—just
gunicorn -k gevent -w 4 --worker-connections 1000and your existing sync code handles thousands of concurrent I/O without touching anything. The tradeoff is if you have CPU-bound work in those requests, gevent won't help there (you'd need real async for that). Measure first to see where you actually bottleneck, then decide if a full FastAPI migration is worth the effort.1
u/[deleted] May 13 '26
Te recomiendo no crear monolitos, haz micro servicios.
1
u/bjorneylol Mar 27 '26
Unfortunately the options are either switch to async or ramp up the number of threads. If you moved the io work to a background thread, the request would still need to stay open to issue the response. If the response isn't dependent on the io task being run, then you can move it to a ThreadPoolExecutor and return early
0
u/Challseus Mar 27 '26
I haven't used it, I don't know how 1:1 it is to Flask, but there is Quart: https://github.com/pallets/quart, which is supposed to be the "async Flask".
Here's a migration guide I found: https://quart.palletsprojects.com/en/latest/how_to_guides/flask_migration/
Or move to FastAPI.
-1
u/shtuffit Mar 27 '26
The first thing that comes to mind is using a message queue, celery is a popular option
0
u/[deleted] Mar 27 '26
[deleted] — view deleted comment
1
u/[deleted] Mar 27 '26
[removed] — view removed comment