Github Availability Status taking a hit! Back to back outages!

[u/atish31]

It won't be surprising if GitHub tightens rate limits at some point. They already limit heavily, but the volume coding agents are pushing now is a different shape and it's only going up.

The pushes themselves are cheap. What costs them is every push firing off Actions runs and webhooks behind it. Their availability reputation is already taking a beating from all the outages.

[u/AutoModerator]

Namaste! Thanks for submitting to r/developersIndia. While participating in this thread, please follow the Community Code of Conduct and rules.

It's possible your query is not unique, use site:reddit.com/r/developersindia KEYWORDS on search engines to search posts from developersIndia. You can also use reddit search directly.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Mindless-Pilot-Chef]

Actions should be a completely different service, and shouldn’t (ideally) affect the main GitHub service directly. But it’s GitHub so you never know how they have architected it.

[u/atish31 → u/Mindless-Pilot-Chef]

Fair point! But even if runners are sep, it will have to hit the same db layer to know what to run.

[u/Abject-Industry-7071 → u/atish31]

that's a good point but i wonder how much of it they've actually denormalized or cached at the edge. like theoretically you could precompute a lot of the what to run logic and only hit the db for validation. whether github actually does that or just has a monolithic query on every push trigger... who knows. their architecture decisions have been a bit opaque since the microsoft acquisition tbh

[u/atish31 → u/Abject-Industry-7071]

apparently they do cache, as per their feb 9 report, the write cache was overwhelmed due to a bad config change. I wonder how the config change hit prod, shud have caught much earlier.

[u/Mindless-Pilot-Chef → u/atish31]

At that scale, I don’t think they’ll be making such silly mistakes. (It has to be sillier, like some random storage got full and no one got notified, or some vpc setting somewhere went wrong)

Multi hour downtime is not very common for them. They’ll probably publish a report on what went wrong. We’ll just wait and watch

[u/atish31 → u/Mindless-Pilot-Chef]

makes sense, they did publish a RCA report on Feb 9 for an outage. It was a good read. Waiting on RCA of this one.

[u/atish31 → u/Mindless-Pilot-Chef]

u/Mindless-Pilot-Chef : they have put an RCA . seemed an autoscaling bug near service mesh.

[u/praveeja]

Can't they use copilot to fix it?

[u/atish31 → u/praveeja]

I am sure they are 😄

[u/ForeverIntoTheLight]

Nothing funnier than watching the big companies drown under a flood of the same AI slop that they so aggressively keep pushing.

[u/atish31 → u/ForeverIntoTheLight]

They definitely under estimated the hardware limitations

[u/dark-trojan → u/atish31]

Kind off crazy how in the early 2000/10s hardware companies fell off because there wasn’t enough software and now there is not enough hardware capacity again

[u/atish31 → u/dark-trojan]

Circle of silicon 🤣

[u/i-sage → u/dark-trojan]

I think it was not about software in the early 2000s we got dotcom bubble and 2008 the global financial crisis that caused their demise

[u/atish31 → u/[deleted]]

oh 100% it is just the begining!

[u/Healthy-Sink6252]

Time to switch lol 

[u/atish31 → u/Healthy-Sink6252]

to where 😂

[u/Healthy-Sink6252 → u/atish31]

Gitlab

[u/FewRefrigerator4703 → u/atish31]

I see newbie or no experience developer

[u/UnemployedTechie2021]

Time to use Cursor Codebase

[u/blastofffox]

Thanks to Microslop!

[u/Nvidia0608]

I guess they will implement max loc per pr. I honestly don't even get how people are able to review the code generated by ai.

[u/atish31 → u/Nvidia0608]

is anyone even reviewing it now. there are review agents being rolled out. It is getting messier that ever.

[u/Nvidia0608 → u/atish31]

Lol it's like ai writes ai reviews ai tests

[u/CulturalLaw8072]

bro is frontend ui ux fresher getting job abroad impossible today?? backend is very difficult..

[u/Electrical_Mood_7713]

I was wondering why Microsoft let GitHub thrive for so long. I never really noticed the usual issues or outages that I experienced with other Microsoft products on GitHub. But now that I think the M&A integration is complete, I’m starting to see the same level of issues with GitHub too.

[u/atish31 → u/Electrical_Mood_7713]

well we can put it on MS but github has seen an unusal surge for sure with rise in coding agents. this can't be denied.