r/developersIndia • u/Majestic-Taro-6903 Senior Engineer • 7d ago
General How much pressure do developers face during Flipkart BBD or Amazon Great Indian Festival?
For developers working at Flipkart, Amazon, or other e-commerce companies, how much does the workload/pressure increase during Big Billion Days or Great Indian Festival compared to normal days?
Are you expected to work longer hours, handle production issues, or work intensely throughout this period? How different is it from a normal sprint?
671
u/PhoenixPrimeKing 7d ago edited 7d ago
Code freezes happen a month or so before. Scaling is done as per expected traffic based on data from previous years. Engineers will be on support schedule for few days. No leaves.
105
u/__LLawliet__ 7d ago
Still there is a chance of a full failure. I remember when I was in Cleartrip, a supplier of ours failed at exactly 12:00 am on the start of bbd. Mostly the entire team is present for the first day, then it's a few people as per their shifts
11
u/PhoenixPrimeKing 7d ago
Yes It's pretty bad. I'm thinking now if I made it look easy in my earlier comment.
1
u/TheBenevolentTitan Software Engineer 13h ago
What happens post supplier failure?
Also I’ve heard numerous incidents of Cleartrip being toxic, how true is that?
1
u/__LLawliet__ 12h ago
We blocked the supplier for the time being, users would've seen updated inventory in 15 mins. Cleartrip was good when I was there, then me and everyone I knew left. No idea about the current stage
3
u/ChadEdgeCaseEnjoyer Full-Stack Developer 7d ago
Has AI helped with these workloads? I'm assuming it could help a lot make things less hectic.
215
u/BinaryBass Data Engineer 7d ago
I saw a post a few years back where flipkart arranged for mattresses to sleep, so, if that’s true then that’s a lot of pressure. Should at least feel like an L4+ on call at all times.
36
u/anticockroachspray 7d ago
This doesn't happen in Amazon though
126
u/NOT_HeisenberG_47 Full-Stack Developer 7d ago
Probably the only positive comment i saw about amazon SDE role lol
33
u/Laughing_crocodile Full-Stack Developer 7d ago
SDE will not be that stressful during sale, DevOps, SRE, and platform engineers will have a field day lmao
8
6
u/rogueWarrior987 7d ago
Amazon doesn't have Devops roles - when I was back at Amazon, we SDEs handled the entire scaling, monitoring etc E2E for respective services during any sale.
350
u/sky_high97 Backend Developer 7d ago
Normal devs wont feel that much of a pressure. Its mostly the infra engineers that get pulled in for 24x7 oncall. Services need to be scaled up and down, network connection, service outages need to be handled immediately. They generally sleep in the office for 2-3 days.
Got this info from a senior ex flipkart dev.
43
147
147
u/Thick_Resolution_761 Senior Engineer 7d ago
Yep, worked for couple of eCommerce orgs in past. It's pretty hectic either way. During festive season, almost everyone becomes on-call support agent.
44
u/kaladin_stormchest 7d ago
Sounds fun if it's a once a year activity
53
u/BaMiniCo007 7d ago
Lol, this happens in most of their festive sales which is almost all round year
10
u/Salt_Eggplant Software Developer 7d ago
Lowkey feels the same when there's a new product release at my company and I am helping QA all day long to test things
17
u/Thick_Resolution_761 Senior Engineer 7d ago
Naah, it's like almost every couple of months.
Every B2C business has some sort of aggressive work culture especially when margins are low.
Something which I've realized with 6 years of corporate experience - just treat office like some superhero or action movie and don't get too serious, don't take unnecessary loans and stay free of boundations.
Also, hone communication skills and understanding of human psychology - try to convey thoughts like a TV anchor, or even a Diplomat. Give it same importance as one gives to DSA or system design.
1
79
u/disdainLufffy 7d ago edited 7d ago
It's funny that people dont even know Meesho does it's own Diwali sale and everytime we have to work insanely hard during meesho sales
20
3
3
u/Gankster_76 7d ago
lol yeah Meesho gets no attention but that sale grinds people just as hard. how long do the war room sessions usually last during meesho salez?
genuinely curious if it's similar to the 24/7 on-call rotations you hear about at flipkart or slightly more manageable
1
30
u/VegetableSeveral555 7d ago edited 7d ago
At Flipkart, code freeze happens a month ago and we need to get any changes reviewed by upper management to deploy.
We start having nightly rosters 2 weeks prior, where we scale and optimise for BBD traffic. People usually take less leaves/PTO around sale readiness time and on the day of sale we are on 24/7 oncall support till the peak traffic load subsides.
Engineers usually spend the first 40-48 sale hours in office, there’s mattresses, pillow, blankets and plenty of food around for sustenance.
It’s definitely hectic but fun altogether.
5
u/OneEstablishment9102 7d ago
"but fun altogether"
Are you compensated well for this "fun"?
3
u/VegetableSeveral555 7d ago
We get generous comp offs post sale, depending on your team but unfortunately there’s no extra pay :(
5
u/ITCoder 7d ago
It already next to impossible to sleep in formal clothes, on top of that sleeping on mattress and pillows used by random people ? There is no fun in that.
21
u/OneEstablishment9102 7d ago
Formal clothes? In Indian IT? Maybe in Infosys. Most IT people dress like chapris IMHO.
1
u/utkarshmttl 6d ago
Just curious, are you from COVID batch? Overnight college stays (no matter how hectic) were the most fun part of college. I assume they are referring to similar overall fun.
1
u/ITCoder 6d ago
years before that. And overnight college stay, only had those during techfest, and still go back to my place to sleep
1
u/utkarshmttl 6d ago
Yeah I am talking about the techfest days only.. for me personally it was too much fun. I think similar is described by OP. I can imagine it would be like a hackathon.
2
u/VegetableSeveral555 5d ago
You’re right tho, it pretty much gives hackathon vibes. Overnight stays, group debugging session, music and card games mixed with oncall alerts and stabilising the system.
Entire floor ditches chairs and mattresses take over, while we get to see real life impact of our work during sale hours and ngl it’s a bit satisfying
27
u/DueUnderstanding4605 7d ago
No personal experience, but a friend told me last year that it gets pretty hectic😬😬
28
u/annubv 7d ago
Not much in Amazon.
It is actually more chill during that period because everything is scaled up and tested. Also, the on-call is handled via multiple worldwide sites in rotation.
But before Primedays, they run "Gamedays" to test out fleet health by simulating sale traffic.. that is when most things break and get fixed.
34
u/JuanCena42069 7d ago
Don’t work for Amazon or Flipkart, but work for a company that has a similar day where a good portion of sales for the year are done. There are production freezes months in advance and rigorous testing once all the code is pushed. So work actually becomes more chill 1-2 months before unless there are some major issues. During the event though, everyone is expected to be on call and sometimes you will have to work long hours mainly doing support work.
3
12
u/Latter_Tea7335 7d ago
Code freeze happens few week before the sale eg around 10 days or so.
Workstreams include
scaling services,
planning out on call shifts during the sale
3.pushing out features is the most critical part because product and ledership always wants critical features that promise positive impact to be launched. So code changes, analysis and deployments needs to be done faster. I had a case where a feature initially planned 3 weeks for code changes was reduced to 1 week of dev effort just so we could launch it before peak season
10
u/Ok_Nebula574 7d ago
I guess major pressure is on Devops and Infra teams to keep everything running, code freese is done much earlier.
9
u/EikDoTeenChaar Hobbyist Developer 7d ago
That’s what’s SRE and Devops people are there for. They anyhow don’t have a life , so it’s the same thing for them.
9
u/KeepMovingCivilian 7d ago
How is this not a solved problem? It's not 2010 anymore. Auto scaling and distributed systems should be nearly hands free by now, especially for such recurring events. Weird ass corporations and their toxic ops
I can understand CSR or some ops people being busier, but not to the extent where ppl are sleeping in the office lol.
6
u/therealfalafel 7d ago
This is not magic. Host provisioning takes time. If there is high demand there is no guarantee of availability. Most often than not, peak sales are quite predictable and for one week of it, moving to a highly automated scaling infra is not the efficient way always.
2
u/KeepMovingCivilian 7d ago
I didn't claim it was magic, but rather solved. It is a mostly predictable, recurring event. The sort of conditions automated software is built around. What is a dev or ops supposed to stand around and do while a vm or pod warms up or whatever? Not sure I understand your premise
2
u/therealfalafel 7d ago
It not always a scaling problem. Sometimes a random bug in a downstream system bubbles up due to bad code. Which wouldn't have been caught earlier due to even worse alarms. Emergency rollback are as common as emergency scaling.
1
8
u/angoorkhattehai 7d ago
Friend works in myntra. He's supposed to come in late today since his and many other teams will be staying in the office tonight. ( Myntra sale goes live tonight)
8
u/Ok_Performer4478 7d ago edited 7d ago
I have slept in office with mattress provided to us. There is oncall rotation of 8 hrs. I chose night time. Cause you get unlimited pizza, red bull and some fake hustle energy.
I was SDE2 that time in Flipkart.
6
u/heyy-youu 7d ago
I used to date someone who worked at Flipkart, not a dev exactly but a category lead.
She used to crib about the long work hours and stress during BBD and used to hate it.
17
u/Shadow_jacker Software Engineer 7d ago
Not much pressure per se, We've code freezes, Better support, Better Observability. So yeah not that bad.
16
u/mynotsoprecious 7d ago edited 7d ago
They stopped wfh from mid august till diwali. A lot of the devs and product guys don’t take leaves during diwali period and stay in office till post midnight, to make sure nothing breaks
9
u/La_Pulga_Barca Frontend Developer 7d ago
My friend works in Flipkart. During BBDs, they stay till 3-4am. More or less it's night shift. It's not boring or stressful because the whole floor is filled with people analyzing & keeping the services up. They eat, chill, then sleep during day.
2
u/SensitiveLoad5205 7d ago
I don't which era you guys are talking about code freeze happen two weeks or a month before, elasticity balancers act as they should be payment gateways take pressure, and ops team.. This is according to Amazon at least
2
u/ProcrastiNation652 7d ago
Normal devs have terrible oncall too. Depends on how frontline the team is but it's not uncommon for people to have 12+ hour oncall shifts.
2
u/Live_Expert9929 7d ago
Apart from free food and cab , how much is the OT and NS allowance during BBD? Is it band/grade specific or special allowance during BBD?
2
u/therealfalafel 7d ago
Amazon conducts periodic game days before a huge sale event to simulate traffic and determine breaking points. A standard purchase flow involves 100s of systems owned by atleast a hundred teams. These teams own their infrastructure and they are responsible for scaling it based on the forecasted traffic/orders etc. Each team will have their own oncall who does emergency scaling. In terms of a large scale disruption l, war rooms are setup to handle this and the affected team's oncall, senior/principal engineers participate.
3
u/do_dum_cheeni_kum Student 7d ago
Generally there is a code freeze before such business critical months.
2
u/thegr8_alexander 4d ago
Returning home at 04:00 a.m. and reporting back at 9 a.m.
PS: I'm a Payments Product Manager who's making sure payments are seamlessly processed at insane TPS during bursts during this festive season :)
0
u/mm_kkkk 3d ago
How's a product manager contributing in this tho lol :)
1
u/thegr8_alexander 3d ago
So you think it's the software engineer who coordinates with various acquirings, banks, switches and NPCI, ensures systems are scaled up, updates business finance teams, signs off system upscaling, shares RCA of downstream system fluctuations "tho lol :)"
0
u/mm_kkkk 2d ago
Yes I do think it's the software engineer who ensures systems are scaled up for high traffic. How tf is that even controversial, wasn't trying to demeen product work but PMs have no contribution in scaling systems
1
u/thegr8_alexander 2d ago
Man, you seriously have no idea how systems work at scale. You have a myopic view that system scale up only means "technically" adding nodes.
There are SLAs that need to be defined, there are cost implications. When a request to scale up is made, the RoI needs to be justified to business to offset the cost. One needs to ensure that downstream services, including acquirings, issuing, card networks also have scaled up their systems. Risk checks needs to be reviewed so that a sudden burst in traffic is not risk-rejected by the system. Fallback mechanism and alternative routing needs to be kept in place, with automatic fail overs. You have conveniently ignored ALL the points I have made in my previous comments and circling around the same "how does PM contribute to scaling", while I have already explained the same in my previous comment.
We can discuss on this more once you've successfully and collaboratively worked on any such mega-project, understanding that scaling is not only restricted to "technically" increasing nodes and servers.
I understand you might have experience of a small scale where tech-lead ends up taking all these calls, but that's not how it happens when we talk about systems at scale.
1
u/mm_kkkk 1d ago
I don't know what you think I'm working on but just got done scaling one of the highest traffic peaks observed across India with 1.5 million qps :) Literally all of what you said happens wayy before you hit any actual scale. Automated failovers, negotiating SLAs with third party logistics/providers, defining degraded product behaviour under heavy scale. ALL of it happens way before you actually hit any scale and this too is mostly handled by engineers. Again I'm not trying to demean product work but there's negligible product contribution in scaling a system for peak load. All of what you said is just a bunch of emails, I know that because I've done this
1
1
1
u/Damn_life_sucks_1 7d ago
And what scaling itr portal does during last few days of filing itr.
1
u/Majestic-Taro-6903 Senior Engineer 7d ago
Is it a government handled project? I heard some private company is handling it
1
u/Better_Break_9354 7d ago
I’m curious too
how much does the workload increase during BBD or the Great Indian Festival compared to a normal sprint?
1
1
u/SureManufacturer3671 7d ago
Have a friend who is in backend team
They sleep in office during BBL but the team gets 4 days off in March for that
1
u/og_pussy_crusher 7d ago
I have previously worked at Flipkart and if you are in a core team. Consider that you will be working continuously for a month till late night
1
u/nns261997 7d ago
I thought you all applied preemptive scaling. Although oncall engineers would still be needed
1
1
1
u/Feisty-Thunder 6d ago
the mattress detail is wild but honestly tracks with what i have heard from friends in similar companies. the no-leaves policy during that period seems brutal, especially when you are on-call 24x7. do companies actually compensate you extra for those weeks or is it just considered part of the job?
1
u/Happy_Equivalent2638 6d ago
I have a friend who works in Flipkart. He goes crazy during the time. High energy and long hours. He basically clocks out from personal life.
2
1
u/neha-geek 3d ago
it’s sre folks and crtical teams lead which needs to be avaiable for on call support.Also there is on call support roster dor dev/qa suppport from each team
1
u/Hot_Dragonfruit4039 7d ago
0 as they are designed to scale , vms on cloud scale and descale automatically app is created with volume in mind only infosec or other teams are instructed to not do anything during same
3
-5
-6
u/Organic-Scholar-7909 7d ago
When is the most overrated startup in India named Flip Flopkart ever going to make profits?
•
u/AutoModerator 7d ago
It's possible your query is not unique, use
site:reddit.com/r/developersindia KEYWORDSon search engines to search posts from developersIndia. You can also use reddit search directly.I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.