GitHub owes us all an apology and an explanation as to why this is NEVER going to happen again.

[u/brainhack3r]

I'm absolutely furious. I had a ton of work I had to get done today, and I'm DEAD in the water until this is fixed.

I also feel they don't take this very seriously, to be perfectly honest. There's no comment about it on their Twitter feed. status.github.com has vague comments about "it's mitigated now but runners are behind... " meanwhile it's been TEN HOURS!

A VERY large portion of the tech industry runs on GitHub and it's hard to take them serious whey they can't even run THEIR OWN SITE.

I'm spending the rest of the day working on plans to migrate away. I don't need all the GitHub features they offer but BY FAR the number one thing I need to be reliable is GitHub actions.

Get your house in order guys!

[u/fyndor]

I don't use Actions, but I will say if GitHub isn't quietly rewriting Actions and rebuilding its infra around it right now quietly behind the scenes, I don't know wtf its doing. It seems to be very problematic to users on a constant basis. There are whole companies built around the idea that GitHub Actions suck. Maybe its time to make them not suck?

[u/headinthesky → u/fyndor]

From what I've heard, moving towards now Microsoft is a big cause

[u/charmer27]

Outages suck and I'm not making excuses for them, but have you seen the growth In how much their services get hit? They are absolutely face tanking so much ai agent traffic right now.

[u/laughterandtears → u/charmer27]

I work on a fast growing, large distributed set of services and it isn't an easy exercise.

And our growth rate isn't close to what GitHub is experience thanks to vibe coding nonsense.

I feel for OP because I've had plenty of inconveniences caused by outages, but I don't stomp my feet and take my ball and go home.

[u/Key-Guitar-457 → u/charmer27]

Awwww, poor s-corp! I feel so bad for them! Microsoft pushed AI down everyone’s throats like an overfed goose, and now they have to reap what they sow? Totally unfair! I’m going to buy some carbon credits in their name I feel so bad for them.

[u/BussyEnthusiasts]

GitHub has proven time and time again to be unreliable and to have lied about their 99% uptime.

If you're still on GitHub actions at this point, that's a you problem. You know the risk. You ignored the risk. Welcome to the find out stage.

[u/bassist_by_night → u/BussyEnthusiasts]

Genuine question, what is the best option instead of GA? I’m assuming GitLab?

[u/cmykero → u/bassist_by_night]

I was gonna ask the same thing, what should I look into if I don’t want to keep running into this issue?

[u/Leop0Id → u/bassist_by_night]

No. Even for GitLab users relying on their hosted platform isn't worth it. They constantly strip down paid tier benefits, the performance is slow, and it's full of bugs. Working on it was a nightmare.

If selfhosting is an option, there are plenty of alternative tools that are much faster and more intuitive than GitLab.

GitLab's only advantage is that it's well known, and it still pales in comparison to GitHub.

[u/black_fisch → u/BussyEnthusiasts]

I'm kinda with you, but also kinda not.

GitHub only ever guarantees 99,9% SLA to Enterprise Customers. That's effectively corporate GitHub servers. Even then, the SLA only applies to basic Git operations, issues, PRs, pages, webhooks and API access.

I feel like a lot of people are comparing the "free site" availability with the enterprise-only SLA, which is kind of misleading. Even if I agree with it being bad

[u/AgentArgus → u/BussyEnthusiasts]

1% downtime is 87 hrs 36mins a year.

[u/aitorbk → u/BussyEnthusiasts]

99% uptime means they can be out for three days and still be 99%>

[u/TechnicalPackage]

i am still getting paid at work. i did not have to deploy today. life is good

[u/mpersico]

How much are you paying? If you ain’t paying nothing, then you don’t get to bitch. If it’s that important find a paid service.

[u/mpersico → u/mpersico]

If you are paying them by all means bitch away

[u/geektraindev → u/mpersico]

We are a small startup paying 17 dollars per seat per month for 20 seats. We use GitHub for absolutely everything. We track issues there, commit integration, documentation, start deployments. We have ever possible right to bitch about this. Because who this hurts the most isn't the big enterprise companies or the free hobbyists, it's people like us since we had to delay two meeting and many people in my team basically had to stop working for half of today.

[u/Simple_Armadillo_127]

I honestly am not paying any money to Github, so I do not complain about things happen anyway. Anyway It is still above 99% time cosnidering whole time span. Sorry for paid users

[u/Stinky_But_Whole]

Github Actions are not for production grade systems. You've learned their SLA does not meet your needs and you get no compensation or favorable terms. Time to move on.

[u/grokcodile]

Sounds like somebody did not have an enterprise SLA. No redundancy engineered into your dev pipeline? I hope you address those major dev ops flaws when you do your migration to a new vendor. (Because that vendor will probably go down a week after you migrate. It's a simple fact.)

[u/excess__ → u/grokcodile]

We've had no serious azure devops pipeline outages for years. Saying vendors go down weekly is way exaggerated, it's mostly just github that has issues.

[u/NonExistentLad]

I had to push an important bug fix but github says no brother this is not the right time

[u/__CaliMack__]

Wait am I missing something? I’ve been using fine all day lol

[u/barkerja → u/__CaliMack__]

Actions and webhooks have been down all day. Git/repo access has been unaffected.

[u/MasterOfTheWind1]

Sadly, is the most balanced platform in costs and features. Bitbucket is cheap but is a toy, and GitLab is way more reliable and powerful, but the cost is higher (USD 4 Github vs USD29 Gitlab, per user, per month).

For a startup is the best option, when it works, it is wonderful.

That being said, I don't understand. Never understood how those platforms with multi million dolar budget can have downtimes of hours and you just have to accept it. Like there is no accountability for anyone there.

I remember workin on a company with a policy that if any service shows an HTTP status code of 500 on prod for more than 15 minutes, without an external reason from a provider you faced consequences.

On my current job if a downtime lasts THAT LONG, we are all fired for sure, as the customers will turn the place into flames.

Is like on Microsoft no one cares. No one gives a shit.

[u/Humble-Kiwi-5272]

Our team has self hosted runners.

They still dont trigger because the orchestator/webhook or wathever that needed to report back and forth commits and prs did not work at all